feat: offline context usage estimation for sessions - #448
Open
111-test-111 wants to merge 1 commit into
Open
Conversation
Estimate a session's context token usage (percent of the model's context window) by reading the .jsonl session file offline — no live agent session needed. Mirrors AgentSession.getContextUsage(): same compaction gate, same last-valid-usage + trailing-estimate formula, with the model resolved from the branch's model_change entries. The session detail and context API routes now return contextUsage using the shared models cache; the models registry loader is extracted from /api/models into lib/models-loader.ts so both can use it. The client hook consumes the file-derived value in loadSession/loadContext so the top-bar and session-stats panel show usage for sessions that are not running, and the system panel fetches live state on demand.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Estimates a session's context-window usage (percent + token counts) by reading the session
.jsonlfile directly — no live AgentSession required — and serves it from the session API routes:GET /api/sessions/[id]andGET /api/sessions/[id]/context?leafId=responses gain a best-effortcontextUsagefield (failures never fail the response)contextUsageon session/context load, so non-running sessions show usage immediately; the system-prompt panel is loaded on demand viaget_stateand refreshes live usage/api/modelsintolib/models-loader.ts(shared, addscontextWindowtomodelList) since the estimator needs model context windowsNotes
tsc --noEmitand eslint clean.app/api/sessions/[id]/route.tsimports/response, as does the companion branch-labels PR — the second one to land may need a trivial rebase.