Add ConfidentialLLM: one-call Oblivious HTTP chat client#303
Merged
Conversation
Adds a high-level client that mirrors the chat app's OHTTP flow (lib/api/ohttp.ts): auto-resolve an OHTTP-capable TEE from the on-chain registry, POST HPKE-encrypted requests to the relay's confidential-inference path (/api/v1/chat/ohttp), and verify the enclave's RSA-PSS signature before returning any content. Previously the SDK exposed only the low-level OhttpRelayClient, which forced callers to build a TEERegistry, resolve the TEE, and know/append the endpoint path by hand. ConfidentialLLM wraps all of that behind a single constructor and a chat() convenience that builds the OpenAI request body (normalizing the "provider/model" id to the gateway model name, like the chat app). It needs no wallet on the caller's side — the untrusted relay holds the x402 account and only ever sees ciphertext. - New module src/opengradient/client/confidential_llm.py (ConfidentialLLM, from_tee, chat/chat_completion/stream_chat_completion, OHTTP_CHAT_ENDPOINT) - Export from client and top-level package - Test tests/confidential_llm_test.py (full encrypt/sign/verify round trip) - Example examples/confidential_chat.py + README entry - Regenerated pdoc docs Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Z4QrXF4R4VGKia6hhTomc
Add a Key Features bullet and a "Confidential Inference (Oblivious HTTP)" section under Core Functionality, covering ConfidentialLLM: end-to-end encrypted chat through an untrusted relay, client-side signature verification, no wallet needed on the caller, auto-resolution of an OHTTP-capable TEE, and the from_tee escape hatch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Z4QrXF4R4VGKia6hhTomc
Implements the CLI-auth device flow the chat app exposes at /cli-auth
(app/(standalone)/cli-auth, CliAuth.tsx). login_chat_account() starts a
short-lived loopback HTTP listener, opens the authorization page pointed at it,
and returns the session the page posts back once the user signs in. The page
only ever posts to a loopback (127.0.0.1) address, so the token never leaves the
user's machine.
The listener answers the browser's CORS preflight (OPTIONS) as well as the JSON
POST — without the preflight response the browser blocks the callback and falls
back to manual copy/paste.
The returned ChatAccountAuth carries the access token plus the client config
(chat_api_base_url, TEE registry) and drops straight into ConfidentialLLM:
auth = og.login_chat_account()
client = og.ConfidentialLLM(
relay_url=auth.chat_api_base_url,
auth_headers=auth.auth_headers,
)
- New module src/opengradient/client/chat_auth.py (login_chat_account,
ChatAccountAuth)
- Export from client and top-level package
- Test tests/chat_auth_test.py (bundle parsing + end-to-end loopback flow with
CORS preflight)
- examples/confidential_chat.py now logs in via the browser by default
- README + examples README + regenerated pdoc docs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015Z4QrXF4R4VGKia6hhTomc
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
ConfidentialLLM, a high-level client for verified, private chat completions over Oblivious HTTP (OHTTP). This is the same confidential inference path the OpenGradient browser chat app uses, enabling end-to-end encrypted requests to TEEs through an untrusted relay without requiring a wallet on the caller's side.Key Changes
New module
opengradient.client.confidential_llm: High-level convenience wrapper that:/api/v1/chat/ohttp)ConfidentialLLMclass with:from_tee()class method to skip registry lookup for self-hosted or pinned enclaveschat()method for argument-driven chat completions (mirrors OpenAI API shape)chat_completion()andstream_chat_completion()escape hatches for raw request bodiesHelper functions:
_gateway_model(): Reduces"provider/model"IDs to gateway model names (e.g.,"anthropic/claude-haiku-4-5"→"claude-haiku-4-5")_confidential_inference_url(): Appends/api/v1/chat/ohttppath to relay URLsDocumentation: Full API docs in
docs/opengradient/client/confidential_llm.mdwith usage examplesExample script
examples/confidential_chat.py: Demonstrates end-to-end encrypted chat with TEE verificationComprehensive tests in
tests/confidential_llm_test.py: Full encrypt/sign/verify round trip through a self-contained OHTTP recipient fixtureUpdated module exports: Added
ConfidentialLLMtoopengradient.__init__.pyand client module docsImplementation Details
OhttpRelayClientandTEERegistryinfrastructure, focusing on the high-level convenience layerDEFAULT_RPC_URL,DEFAULT_TEE_REGISTRY_ADDRESS) are duplicated fromllm.pyto keep the confidential path free of x402/wallet dependenciesbuildInnerChatRequestlogicstream_framesare safe to replay to end usersauth_headersproviderhttps://claude.ai/code/session_015Z4QrXF4R4VGKia6hhTomc