Skip to content

bug: failed reporter calls leave orphaned prompts in history #33

Description

@JNK234

Problem

Several reporters append the user prompt to per-agent history before the provider call succeeds, but they do not roll that mutation back on failure.

Current behavior

This pattern appears in multiple places:

  • llm:chat
  • llm:chat-async
  • llm:choose
  • llm:chat-with-template (already problematic for separate reasons)

In each case, the user-side prompt is added to the mutable history buffer before the provider response is known. If the request fails because of timeout, invalid credentials, network error, or parsing error, the reporter throws an exception but the partial prompt remains in history.

Why this matters

This makes failures stateful in a way the caller cannot easily see:

  • a retry runs with extra prompt context that came from the failed attempt
  • llm:choose can pollute history with the internal constrained-choice prompt text
  • one transient provider error can permanently alter later conversation behavior for that agent

This is especially confusing because the call appears to have failed "cleanly" from NetLogo’s perspective, but the extension has already mutated state.

Affected code

  • src/main/LLMExtension.scala (ChatReporter.report)
  • src/main/LLMExtension.scala (ChatAsyncReporter.report)
  • src/main/LLMExtension.scala (ChooseReporter.report)
  • src/main/LLMExtension.scala (ChatWithTemplateReporter.report)

Suggested fix directions

  1. Treat provider calls as transactional with respect to history.
  2. Build request context from a snapshot and only commit user/assistant messages after success.
  3. Add failure-path tests proving that history is unchanged after provider exceptions or timeouts.
  4. Consider whether internal prompts used by llm:choose should be hidden from persistent user-visible history entirely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions