[ORCJIT] Embed liborc_rt into the extension and add orc_rt selector#660
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the ORC JIT session to embed the liborc_rt archive directly into the shared library's .rodata section on Linux/ELF, eliminating the need for on-disk path lookups. It updates the Python and C++ APIs to support a flexible orc_rt selector (supporting embedded, custom path, in-memory bytes, or no platform). The review feedback identifies two issues: first, SetUpOrcPlatform is incorrectly invoked on Windows due to an overly broad preprocessor check (#ifndef __APPLE__), which could cause runtime errors; second, subtracting the two distinct extern pointers orc_rt_archive_end and orc_rt_archive_start to calculate the archive size constitutes undefined behavior in C++ and should be resolved by casting to std::uintptr_t first.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
The ORC runtime shipped as a separate liborc_rt*.a data file resolved at startup via a filesystem glob (the tvm_ffi_orcjit.DefaultOrcRuntimePath Python hook). That coupled two artifacts and let the runtime be lost or relocated by repackaging. Bake the archive into the .so's .rodata with a generated .incbin stub (orc_rt_embed.S.in) and hand it to ExecutorNativePlatform as a zero-copy MemoryBuffer. The wheel now ships one artifact, startup does no path lookup, and the runtime cannot be separated from the extension. The embedded start/end symbols are hidden, adding no interposition surface. default_session() is always the embedded runtime and is deliberately not user-configurable: a shared process-wide singleton with a hidden runtime knob would let the first caller pick the platform for everyone. User-created sessions gain an orc_rt selector (was orc_rt_path): - "auto" (default): the embedded runtime; - str/Path: a custom liborc_rt archive on disk; - bytes: a custom liborc_rt archive in memory; - None: no ORC platform. C++ takes Optional<Variant<String, Bytes>>; empty String is "auto", nullopt is None. Linux/ELF only, matching where the platform is wired up; macOS/Windows ignore the selector as before. Custom runtimes must match the LLVM the extension was built against. Implements Solution A of the embedded-orcrt design.
7e000c2 to
cc956d1
Compare
Motivation
The ORC runtime shipped as a separate
liborc_rt*.adata file next to the extension.soand was resolved at startup via a filesystem glob (through thetvm_ffi_orcjit.DefaultOrcRuntimePathPython hook). That couples two artifacts, lets the runtime be lost/relocated by repackaging, and does a filesystem glob to find its own runtime on first session.What this does
Bakes
liborc_rt.ainto the extension.so's.rodatavia a generated.incbinstub (orc_rt_embed.S.in) and hands the bytes toExecutorNativePlatformas a zero-copyMemoryBuffer. The wheel now ships one artifact, startup does no path lookup, and the runtime cannot be separated from the extension. The embeddedstart/endsymbols are.hidden, so they add no interposition surface and don't affect the existing--exclude-libssymbol hiding.This is Solution A of the embedded-orcrt design.
default_session()Always uses the embedded runtime and is deliberately not user-configurable: a shared process-wide singleton with a hidden runtime knob would let whichever caller runs first silently pick the platform for everyone. The old
DefaultOrcRuntimePathdiscovery hook is dropped.User-created
ExecutionSession— neworc_rtselectorRenamed
orc_rt_path→orc_rt, now a 4-state selector:orc_rt="auto"(default)str/Pathliborc_rtarchive on diskbytesliborc_rtarchive in memoryNoneC++ takes
Optional<Variant<String, Bytes>>; emptyString=="auto",nullopt==None. Linux/ELF only, matching where the ORC platform is wired up today; macOS/Windows ignore the selector as before. A custom runtime must match the LLVM/compiler-rt the extension was built against.Testing
orc_rttests: custom path (str +Path), in-memory bytes,None(no platform), and bad-type rejection. Path/bytes cases are@skipifnon-Linux (the selector only takes effect on ELF)..socarries the hiddenorc_rt_archivesymbols in.rodata, absent from the dynamic table; noliborc_rt*.abundled; C++ exception unwinding across a JIT frame works underauto/path/bytes (provingELFNixPlatformis genuinely installed).