fix(config): Fix inquiry ceiling inheritance and fields() gaps - #928
Open
JeanMertz wants to merge 7 commits into
Open
fix(config): Fix inquiry ceiling inheritance and fields() gaps#928JeanMertz wants to merge 7 commits into
fields() gaps#928JeanMertz wants to merge 7 commits into
Conversation
A query now aborts once a response's generated content passes `assistant.request.max_response_bytes` (default 1 MiB, `0` disables it). The turn ends with an error, but any content already streamed stays in the conversation, and the response is not retried, since retrying would just regenerate the same runaway. This bounds the cost of a model that gets stuck generating without end while nobody is watching the terminal, and applies equally to interactive queries and to no-terminal collect calls such as title generation and summarization. The ceiling wraps the stream outside any provider-level continuation logic, so a response Anthropic assembles from several chained `MaxTokens` continuations counts as one response against the limit rather than resetting per link. Anthropic's own continuation chaining is separately bounded to 5 links (`MAX_CHAIN_DEPTH`), so a model that keeps hitting the token ceiling can no longer chain requests forever. Bytes are counted rather than tokens, since tokens can't be counted locally; roughly four bytes per token is a reasonable guide when tuning the value. Set it from the CLI, e.g. `jp config set assistant.request.max_response_bytes 4096`. Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
Signed-off-by: Jean Mertz <git@jeanmertz.com>
`conversation.inquiry.assistant.request.max_response_bytes` no longer silently disables the output ceiling when only a sibling field (e.g. `cache`) is set in that block. The inquiry `assistant` block now mirrors the full `assistant` config and inherits every unset field from it during resolution, rather than resolving unset fields to Rust's `Default`. Users can still disable the ceiling for inquiries alone with an explicit `0`/`"disabled"`, independent of the parent assistant's setting. `max_response_bytes` itself is now a `MaxResponseBytes` value instead of a bare `u32`, accepting a byte count or `0`, `"disabled"`, `"off"`, `false` to turn the ceiling off, e.g. `jp config set assistant.request.max_response_bytes disabled`. This removes the previous ambiguity where `0` had a different meaning depending on which config layer it appeared in. `AppConfig::fields()` (and therefore `envs()`) now descends into `Option<T>` nested configs and into unions that describe one value written multiple ways (a shorthand plus its expanded table form), so keys like `style.reasoning.summary_model.id`, `editor.cmd.program`/`args`/`shell`, and `conversation.tools.*.enable.state` are reported and assignable via env vars, where before the containing block was reported as an opaque leaf. `editor.cmd` and `conversation.tools.*.enable` keep their shorthand form addressable too: writing `editor.cmd.args` after `editor.cmd = "code --wait"` now expands the shorthand and keeps the named program instead of being rejected. Schema support for this (`schema_union_with`, `UnionType::expanded_index`) lives in `schematic_macros`/`schematic_types`, letting a derived struct schema be unioned with hand-declared shapes for types whose `Deserialize` accepts more than their fields describe. Signed-off-by: Jean Mertz <git@jeanmertz.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
conversation.inquiry.assistant.request.max_response_bytesno longer silently disables the output ceiling when only a sibling field (e.g.cache) is set in that block. The inquiryassistantblock now mirrors the fullassistantconfig and inherits every unset field from it during resolution, rather than resolving unset fields to Rust'sDefault. Users can still disable the ceiling for inquiries alone with an explicit0/"disabled", independent of the parent assistant's setting.max_response_bytesitself is now aMaxResponseBytesvalue instead of a bareu32, accepting a byte count or0,"disabled","off",falseto turn the ceiling off, e.g.jp config set assistant.request.max_response_bytes disabled. This removes the previous ambiguity where0had a different meaning depending on which config layer it appeared in.AppConfig::fields()(and thereforeenvs()) now descends intoOption<T>nested configs and into unions that describe one value written multiple ways (a shorthand plus its expanded table form), so keys likestyle.reasoning.summary_model.id,editor.cmd.program/args/shell, andconversation.tools.*.enable.stateare reported and assignable via env vars, where before the containing block was reported as an opaque leaf.editor.cmdandconversation.tools.*.enablekeep their shorthand form addressable too: writingeditor.cmd.argsaftereditor.cmd = "code --wait"now expands the shorthand and keeps the named program instead of being rejected.Schema support for this (
schema_union_with,UnionType::expanded_index) lives inschematic_macros/schematic_types, letting a derived struct schema be unioned with hand-declared shapes for types whoseDeserializeaccepts more than their fields describe.