feat(gateway): use server-minted scoped tokens for model calls - #1130
Draft
brandonleung wants to merge 7 commits into
Draft
feat(gateway): use server-minted scoped tokens for model calls#1130brandonleung wants to merge 7 commits into
brandonleung wants to merge 7 commits into
Conversation
Each run now asks the PostHog backend for a scoped gateway token instead of sending the user's OAuth token upstream, and speaks the new gateway's contract: run metadata as one X-PostHog-Properties blob, no explicit Bedrock-fallback opt-in. Any non-200 from the mint endpoint falls back to today's posture, so rollout is entirely server-side. Also clamps xhigh effort to high on anthropic models, which reject it while extended thinking is off. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
🧙 Wizard CIRun the Wizard CI and test your changes against wizard-workbench example apps by replying with a GitHub comment using one of the following commands: Test all apps:
Test all apps in a directory:
Test an individual app:
Show more apps
Results will be posted here when complete. |
Review round on the scoped-token flow. buildGatewayProvider re-derived thinkingLevel from the raw override, so the xhigh clamp sat on a signature no production caller used and the provider 400 still shipped. The override now resolves through modelCapabilities. Also: the subprocess takes the run's resolved credential as a value rather than re-reading process.env, which every concurrent task run overwrites; the mint is single-flighted; the refresh margin is capped at a fifth of the token's life, so a short TTL is still served from cache; a returned gateway url is checked for scheme and host before any bearer goes to it; fallbacks say why in the log; and the tutorial stream, the fifth SDK site, joins the same posture instead of staying on per-key headers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second review round; the first round's own fixes are the subject. The tutorial stream adopted the resolved gateway url and header edition but kept sending the OAuth token, so once the mint endpoint ships that surface would have presented a legacy credential to the new gateway. The url, the bearer and the edition travel together now, and its subprocess reads them from the run's auth rather than the globals a sibling run also writes. The 401 screen's region and url come from the same place. An explicit `off` effort is applied instead of dropped: mapping it to undefined fell through to the model table's own effort, which is the opposite of what the frontmatter asked for. The refresh margin gains a floor, and a token arriving with less life than one request is refused rather than cached, since the subprocess holds its credential for the whole session. A trusted gateway url must now be a bare origin, because every consumer appends a route to it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Third review round. The clamp was inert end to end: pi clamps every level a model spec does not declare, and the gateway spec declares no thinkingLevelMap, so xhigh already became high before any request was built. Its tests asserted a path production cannot reach. The xhigh-with-thinking-disabled rejection lives on the anthropic-harness path, which sets effort through the agent SDK, so it is not this PR's to fix. The token-lifetime guard now takes the server's own relative lifetime when it offers one, because judging a server expiry against this machine's clock let skew disable the new gateway outright. A token is adopted only when the refresh margin still leaves a usable cache window, so a short one falls back rather than being re-minted per request. The 401 screen's gateway url has a test, and the unreachable bare-IPv6 branch is gone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Final verification round: tests and comments only. The auth-error case set an env var the function never reads and then checked it echoed back its own argument, so it passed on a revert of the fix it was named for. It now pins what is actually load-bearing: the region the 401 screen shows comes from whichever url it is handed. The off-effort comment claimed a spend saving pi does not deliver: with no thinkingLevelMap on the spec, pi omits reasoning_effort rather than disabling reasoning, so the provider default applies. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Overengineering pass on this PR's own work. The expires_in handling was the client half of a protocol the server does not speak: nothing in the mint response carries that field, so the branch and its test covered a path no real response can take. If clock skew turns out to matter, the fix starts with the endpoint. Four constants decided one question. Two do: a token is adopted with at least two minutes of life, and re-resolved once four fifths of that life is gone. That drops refreshSlackMs and its suite, and the remaining fixtures pin the same behaviour with less machinery. Also drops the bare-IPv6 host from the trusted set, which WHATWG parsing makes unreachable. The two loopback forms and host.docker.internal stay: urls.ts really emits them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A run today puts the user's OAuth token in
ANTHROPIC_AUTH_TOKENand points the agent at the gateway's/wizardpath. On the new gateway a run should instead hold a credential scoped to it: capped, expiring, and carrying attribution the run cannot change.Changes
Adds
gateway-session.tsas the single place a run resolves its gateway credential, and teaches the request path the new gateway's contract./wizardpath), so the flip is server-controlled and rolling it back needs no release.X-PostHog-Propertiesblob on the new path; the per-keyX-POSTHOG-PROPERTY-*headers stay on the old one. Flag variants land aswizard_flag_<key>, since the gateway strips$-prefixed keys.x-posthog-use-bedrock-fallbackopt-in on the new path, where fallback is part of the routing chain.process.env, which every concurrent task run overwrites.Test plan
vitest run: 2009 cases pass. New coverage sits on the wire contract: the v2 header shape on both producers, single-flight under three concurrent callers, a short-TTL cache hit, per-field rejection of a malformed mint response, and the untrusted-URL refusal. Typecheck adds no errors against the pre-change baseline. Not exercised against a live mint endpoint, which ships in PostHog/posthog#87269 and is unset until deployed.Known gap: a run outliving its token's TTL still fails at the request, because the anthropic harness's subprocess captures the credential once at spawn. The pi harness re-resolves per task and is unaffected.
Out of scope: the
xhigh-with-thinking-disabled combination the old gateway rewrites. It originates on the anthropic harness's agent-SDK path, so a clamp in the model switchboard would not reach it.