Skip to content

[ORCJIT] Embed liborc_rt into the extension and add orc_rt selector#660

Merged
cyx-6 merged 1 commit into
apache:mainfrom
cyx-6:orcjit-embed-orc-rt
Jul 11, 2026
Merged

[ORCJIT] Embed liborc_rt into the extension and add orc_rt selector#660
cyx-6 merged 1 commit into
apache:mainfrom
cyx-6:orcjit-embed-orc-rt

Conversation

@cyx-6

@cyx-6 cyx-6 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Motivation

The ORC runtime shipped as a separate liborc_rt*.a data file next to the extension .so and was resolved at startup via a filesystem glob (through the tvm_ffi_orcjit.DefaultOrcRuntimePath Python hook). That couples two artifacts, lets the runtime be lost/relocated by repackaging, and does a filesystem glob to find its own runtime on first session.

What this does

Bakes liborc_rt.a into the extension .so's .rodata via a generated .incbin stub (orc_rt_embed.S.in) and hands the bytes to ExecutorNativePlatform as a zero-copy MemoryBuffer. The wheel now ships one artifact, startup does no path lookup, and the runtime cannot be separated from the extension. The embedded start/end symbols are .hidden, so they add no interposition surface and don't affect the existing --exclude-libs symbol hiding.

This is Solution A of the embedded-orcrt design.

default_session()

Always uses the embedded runtime and is deliberately not user-configurable: a shared process-wide singleton with a hidden runtime knob would let whichever caller runs first silently pick the platform for everyone. The old DefaultOrcRuntimePath discovery hook is dropped.

User-created ExecutionSession — new orc_rt selector

Renamed orc_rt_pathorc_rt, now a 4-state selector:

orc_rt= meaning
"auto" (default) the embedded runtime
str / Path a custom liborc_rt archive on disk
bytes a custom liborc_rt archive in memory
None no ORC platform

C++ takes Optional<Variant<String, Bytes>>; empty String == "auto", nullopt == None. Linux/ELF only, matching where the ORC platform is wired up today; macOS/Windows ignore the selector as before. A custom runtime must match the LLVM/compiler-rt the extension was built against.

Testing

  • New orc_rt tests: custom path (str + Path), in-memory bytes, None (no platform), and bad-type rejection. Path/bytes cases are @skipif non-Linux (the selector only takes effect on ELF).
  • Verified locally (aarch64 Linux, LLVM 22): the .so carries the hidden orc_rt_archive symbols in .rodata, absent from the dynamic table; no liborc_rt*.a bundled; C++ exception unwinding across a JIT frame works under auto/path/bytes (proving ELFNixPlatform is genuinely installed).
  • Full addon suite: 164 passed, 11 skipped. clang-format / ruff / cmake-format / ASF-header all clean.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the ORC JIT session to embed the liborc_rt archive directly into the shared library's .rodata section on Linux/ELF, eliminating the need for on-disk path lookups. It updates the Python and C++ APIs to support a flexible orc_rt selector (supporting embedded, custom path, in-memory bytes, or no platform). The review feedback identifies two issues: first, SetUpOrcPlatform is incorrectly invoked on Windows due to an overly broad preprocessor check (#ifndef __APPLE__), which could cause runtime errors; second, subtracting the two distinct extern pointers orc_rt_archive_end and orc_rt_archive_start to calculate the archive size constitutes undefined behavior in C++ and should be resolved by casting to std::uintptr_t first.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread addons/tvm_ffi_orcjit/src/ffi/orcjit_session.cc
Comment thread addons/tvm_ffi_orcjit/src/ffi/orcjit_session.cc
The ORC runtime shipped as a separate liborc_rt*.a data file resolved at
startup via a filesystem glob (the tvm_ffi_orcjit.DefaultOrcRuntimePath
Python hook). That coupled two artifacts and let the runtime be lost or
relocated by repackaging.

Bake the archive into the .so's .rodata with a generated .incbin stub
(orc_rt_embed.S.in) and hand it to ExecutorNativePlatform as a zero-copy
MemoryBuffer. The wheel now ships one artifact, startup does no path
lookup, and the runtime cannot be separated from the extension. The
embedded start/end symbols are hidden, adding no interposition surface.

default_session() is always the embedded runtime and is deliberately not
user-configurable: a shared process-wide singleton with a hidden runtime
knob would let the first caller pick the platform for everyone.

User-created sessions gain an orc_rt selector (was orc_rt_path):
  - "auto" (default): the embedded runtime;
  - str/Path: a custom liborc_rt archive on disk;
  - bytes: a custom liborc_rt archive in memory;
  - None: no ORC platform.
C++ takes Optional<Variant<String, Bytes>>; empty String is "auto",
nullopt is None. Linux/ELF only, matching where the platform is wired up;
macOS/Windows ignore the selector as before. Custom runtimes must match
the LLVM the extension was built against.

Implements Solution A of the embedded-orcrt design.
@cyx-6 cyx-6 force-pushed the orcjit-embed-orc-rt branch from 7e000c2 to cc956d1 Compare July 10, 2026 20:20
@cyx-6 cyx-6 merged commit 329ec83 into apache:main Jul 11, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants