dotnet: in-process FFI runtime hosting (InProcess transport)#1901
Draft
SteveSandersonMS wants to merge 5 commits into
Draft
dotnet: in-process FFI runtime hosting (InProcess transport)#1901SteveSandersonMS wants to merge 5 commits into
SteveSandersonMS wants to merge 5 commits into
Conversation
Add a new InProcess transport that loads the runtime cdylib (runtime.node)
in-process and speaks JSON-RPC over its C ABI instead of spawning a CLI
subprocess and talking over stdio/TCP. The Rust host_start call spawns the
residual Node worker (napi-oop) itself; the C# side only drives opaque
JSON-RPC blobs over the FFI boundary.
- FfiRuntimeHost.cs: shared host body (library resolution, host_start/
connection_open handshake, duplex Channel-backed streams bridging the native
outbound callback to the SDK's JsonRpc framing) with two interop backends:
* net8.0/net10.0: source-generated [LibraryImport] P/Invoke with a
DllImportResolver mapping the logical library name to the runtime.node
absolute path and an [UnmanagedCallersOnly] cdecl function-pointer callback
routed via a GCHandle. Trim/NativeAOT-compatible (IsAotCompatible).
* netstandard2.0: classic delegate-based P/Invoke over a hand-rolled
dlopen/LoadLibrary loader (netstandard2.0 has neither LibraryImport,
NativeLibrary, nor UnmanagedCallersOnly), with the outbound callback held as
an instance delegate for the connection's lifetime.
- Types.cs: InProcessRuntimeConnection + RuntimeConnection.ForInProcess(path, args).
- Client.cs: select the FFI transport for InProcessRuntimeConnection (or the
COPILOT_SDK_FFI_HOST override), send a null connection token in FFI mode.
- csproj: AllowUnsafeBlocks for the function-pointer callback.
- ClientE2ETests: Should_Start_And_Connect_Over_InProcess_Ffi.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e-bin The in-process FFI transport no longer resolves or passes a `copilot-runtime-bin` provider path. The child manifest is delivered over the napi-oop handshake, so host_start only needs the CLI entrypoint and env. - FfiRuntimeHost: drop the provider field/param and the provider argument from the C ABI bindings (modern LibraryImport + netstandard2.0 delegate); accept a binary-or-.js entrypoint. A .js entrypoint is launched via `node` (dev/dist-cli); the packaged single-file CLI binary is invoked directly as `copilot --embedded-host` (it embeds its own Node). - Client: FFI path resolution falls back to the bundled CLI (runtimes/<rid>/native/copilot) the same way stdio discovers it, still overridable via COPILOT_CLI_PATH. Renamed ResolveCliJsPathForFfi -> ResolveCliPathForFfi. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The in-process FFI host loads the Rust cdylib in the .NET process, so the build must emit it next to the CLI binary. The MSBuild targets copy the tarball's napi-style prebuilds/<npm-platform>/runtime.node to a flat, natural shared-library name in runtimes/<rid>/native/: win -> copilot_runtime.dll linux -> libcopilot_runtime.so osx -> libcopilot_runtime.dylib (these are what the Rust cdylib would be called if napi didn't rename it to .node; the .NET host loads it by absolute path, so the name is ours). This avoids shipping a doubly-non-obvious prebuilds/<node-platform>/runtime.node tree in .NET output. Emission is conditioned on the source existing, so stdio-only consumers on tarballs without the cdylib are unaffected. FfiRuntimeHost resolves the cdylib relative to the entrypoint, preferring the flat renamed sibling and falling back to the dev dist-cli/prebuilds/<node-platform>-<arch>/runtime.node layout (for COPILOT_CLI_PATH overrides). The prebuilds folder uses the napi-rs <node-platform>-<arch> convention (win32-x64/darwin-x64/linux-x64), not the .NET RID (win-x64/osx-x64) which only matched on Linux; Client now computes this via GetNapiPrebuildsFolder. Validated: FFI E2E passes against a locally-built SEA copilot binary using both the flat renamed libcopilot_runtime.so and the dev prebuilds fallback; the targets emit copilot + libcopilot_runtime.so in runtimes/<rid>/native/ and skip the cdylib gracefully when absent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| if (ctx.FfiHost is { } ffiHost) | ||
| { | ||
| try { ffiHost.Dispose(); } | ||
| catch (Exception ex) { AddCleanupError(errors, ex, _logger); } |
| ?? throw new InvalidOperationException($"Could not determine directory for '{cliEntrypoint}'."); | ||
|
|
||
| // Bundled .NET layout: flat, natural shared-library name next to the CLI. | ||
| var flatLibraryPath = Path.Combine(distDir, GetRuntimeLibraryFileName()); |
| // Bundled .NET layout: flat, natural shared-library name next to the CLI. | ||
| var flatLibraryPath = Path.Combine(distDir, GetRuntimeLibraryFileName()); | ||
| // Dev/tarball layout: dist-cli/prebuilds/<node-platform>-<arch>/runtime.node. | ||
| var prebuildsLibraryPath = Path.Combine(distDir, "prebuilds", prebuildsFolder, "runtime.node"); |
Comment on lines
+227
to
+230
| catch (Exception ex) | ||
| { | ||
| _logger.LogDebug(ex, "FfiRuntimeHost: connection_close failed"); | ||
| } |
Comment on lines
+240
to
+243
| catch (Exception ex) | ||
| { | ||
| _logger.LogDebug(ex, "FfiRuntimeHost: host_shutdown failed"); | ||
| } |
| public static IntPtr Sym(IntPtr handle, string name) | ||
| { | ||
| try { return Libdl2.dlsym(handle, name); } | ||
| catch (DllNotFoundException) { return Libdl1.dlsym(handle, name); } |
| private static class Libdl2 | ||
| { | ||
| [DllImport("libdl.so.2", EntryPoint = "dlopen", CharSet = CharSet.Ansi, BestFitMapping = false, ThrowOnUnmappableChar = true)] | ||
| public static extern IntPtr dlopen([MarshalAs(UnmanagedType.LPStr)] string fileName, int flags); |
| public static extern IntPtr dlopen([MarshalAs(UnmanagedType.LPStr)] string fileName, int flags); | ||
|
|
||
| [DllImport("libdl.so.2", EntryPoint = "dlsym", CharSet = CharSet.Ansi, BestFitMapping = false, ThrowOnUnmappableChar = true)] | ||
| public static extern IntPtr dlsym(IntPtr handle, [MarshalAs(UnmanagedType.LPStr)] string symbol); |
| private static class Libdl1 | ||
| { | ||
| [DllImport("libdl", EntryPoint = "dlopen", CharSet = CharSet.Ansi, BestFitMapping = false, ThrowOnUnmappableChar = true)] | ||
| public static extern IntPtr dlopen([MarshalAs(UnmanagedType.LPStr)] string fileName, int flags); |
| public static extern IntPtr dlopen([MarshalAs(UnmanagedType.LPStr)] string fileName, int flags); | ||
|
|
||
| [DllImport("libdl", EntryPoint = "dlsym", CharSet = CharSet.Ansi, BestFitMapping = false, ThrowOnUnmappableChar = true)] | ||
| public static extern IntPtr dlsym(IntPtr handle, [MarshalAs(UnmanagedType.LPStr)] string symbol); |
The native connection_write copies its bytes synchronously, so the C# side need not own the buffer past the call. Thread frames through as ReadOnlySpan instead of allocating+copying a byte[] per write (LibraryImport marshals the span pointer; the netstandard2.0 delegate pins it). Also coalesce the LSP Content-Length header and JSON body into one pooled buffer so each message is a single write — one native boundary crossing instead of two. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
BuildFrame stackalloc'd a header scratch then copied it into the rented frame. Over-rent by the fixed <=30B header bound instead and write the header directly into the frame, dropping the scratch buffer and header copy. The single JSON copy remains (its length must precede it as Content-Length). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
Cross-SDK Consistency ReviewScope: All 7 changed files are in New public API surfaceThis PR introduces a new transport option to the RuntimeConnection.ForInProcess() // → InProcessRuntimeConnectionCross-SDK transport parityFor reference, here's how transport options currently map across SDKs:
AssessmentThis is an intentional, .NET-specific transport that uses .NET platform capabilities ( As the PR notes, this is also strictly opt-in and leaves all existing stdio/TCP/URI transports unchanged. Suggestion (non-blocking)Once the native in-process host capability ships and this graduates out of draft, it may be worth opening a tracking issue to evaluate whether other language SDKs could benefit from similar in-process transport options — even if the implementations will differ significantly per language. No cross-SDK consistency changes are needed for this PR as drafted.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In-process FFI runtime hosting for the .NET SDK
Adds a new InProcess transport to the .NET SDK. Instead of spawning the CLI as a subprocess and talking JSON-RPC over stdio/TCP, the SDK loads the runtime's native shared library in-process and drives JSON-RPC over its C ABI. The native host spawns the residual Node worker itself; the C# side only pumps opaque JSON-RPC blobs across the FFI boundary.
What's here
FfiRuntimeHost.cs— shared host body (native library resolution,host_start/connection_openhandshake, duplexChannel-backed streams bridging the native outbound callback to the SDK's JSON-RPC framing) with two interop backends:[LibraryImport]P/Invoke with aDllImportResolvermapping the logical library name to the absolute path of the native lib, and an[UnmanagedCallersOnly]cdecl function-pointer callback routed via aGCHandle. Trim/NativeAOT-compatible (IsAotCompatible).dlopen/LoadLibraryloader (netstandard2.0 has neitherLibraryImport,NativeLibrary, norUnmanagedCallersOnly).Types.cs—InProcessRuntimeConnection+RuntimeConnection.ForInProcess(path, args).Client.cs— selects the FFI transport forInProcessRuntimeConnection(or theCOPILOT_SDK_FFI_HOSToverride); resolves the entrypoint fromCOPILOT_CLI_PATHand falls back to the bundled CLI the same way stdio discovers it.MSBuild targets — emit the native shared library next to the bundled CLI as a natural, platform-obvious name in
runtimes/<rid>/native/:copilot_runtime.dlllibcopilot_runtime.solibcopilot_runtime.dylibThe .NET host loads it by absolute path, so the filename is ours. Emission is conditioned on the source existing, so stdio-only consumers are unaffected.
ClientE2ETests—Should_Start_And_Connect_Over_InProcess_Ffi.Status — draft
This depends on native in-process-host support that ships in a future
@github/copilotCLI release. It should not be merged until the pinned CLI version is bumped to a build that ships the runtime shared library and the in-process host capability. Kept as a draft until then.The default stdio/TCP transports are unchanged; InProcess is strictly opt-in.