fix(hitl): approve/reject continues conversation, persists trace, multi-tool-call decisions

README.md

@@ -249,7 +249,7 @@ Every agent is defined by a single YAML file validated against the `AgentConfig`
 | `system_prompt` | `string` | `null` | Inline system prompt. Mutually exclusive with `system_prompt_file`. |
 | `system_prompt_file` | `string` | `null` | Path to a text file containing the system prompt (resolved relative to the YAML file). Mutually exclusive with `system_prompt`. |
 | `tools` | `list[string]` | `[]` | Python tool references in `module.path:attribute` format. |
-| `backend` | `BackendConfig` | `{"type": "state", "store_backend": "memory", "checkpoint_backend": "memory"}` | Persistence backend. See [Backends](#backends). |
+| `backend` | `BackendConfig` | `{"type": "state", "store_backend": "memory", "checkpoint_backend": "postgres"}` | Persistence backend. See [Backends](#backends). |
 | `hitl` | `HITLConfig` | `{"rules": {}}` | Human-in-the-loop interrupt rules. |
 | `memory` | `list[string]` | `[]` | Paths to memory files (e.g. `"./AGENTS.md"`). |
 | `skills` | `list[string]` | `[]` | Paths to skill directories (e.g. `"./skills/"`). |
@@ -509,7 +509,7 @@ The `BackendConfig` schema controls where agent state and checkpoints are persis
 |---|---|---|---|
 | `type` | `BackendType` (`state` \| `store`) | `state` | Backend kind. `state` = in-memory LangGraph state, `store` = LangGraph store-backed. |
 | `store_backend` | `Literal["memory", "postgres"]` | `memory"` | Where the LangGraph store lives. `memory` = in-process, `postgres` = PostgreSQL-backed (singleton reused across all agent builds). |
-| `checkpoint_backend` | `Literal["memory", "postgres"]` | `memory"` | Where the LangGraph checkpointer lives. `memory` = in-process, `postgres` = PostgreSQL-backed (singleton reused across all agent builds). |
+| `checkpoint_backend` | `Literal["memory", "postgres"]` | `"postgres"` | Where the LangGraph checkpointer lives. `memory` = in-process, `postgres` = PostgreSQL-backed (singleton reused across all agent builds). Defaults to `postgres` for durability; falls back to `memory` with a warning if Postgres is unreachable at agent-build time. |
 
 ### Supported `type` values
 
@@ -562,7 +562,7 @@ All endpoints are prefixed appropriately. The server runs on `http://localhost:8
 | `GET` | `/api/v1/threads/{thread_id}/messages` | List messages in a thread (projection from `trace_events`: `HUMAN_MESSAGE` + `AI_MESSAGE` only, backward-compat) | `200` |
 | `POST` | `/api/v1/chat/{thread_id}` | Send a message and get the full response | `200` |
 | `POST` | `/api/v1/chat/{thread_id}/stream` | Send a message and stream the response (SSE) | `200` |
-| `POST` | `/api/v1/threads/{thread_id}/hitl` | Submit a human-in-the-loop decision | `200` |
+| `POST` | `/api/v1/chat/{thread_id}` | Submit a human-in-the-loop decision (approve/reject/edit, single or multi `decisions`) | `200` |
 | `GET` | `/api/v1/agents` | List all agent configs from `agents/` directory | `200` |
 | `GET` | `/api/v1/agents/{agent_name}` | Get a specific agent configuration | `200` |
 | `GET` | `/api/v1/store/files` | List file paths in the store (optional `prefix` query param) | `200` |
@@ -917,15 +917,18 @@ Response (`200`):
 
 ### 10. HITL -- Approve a Pending Tool Call
 
-When the agent is configured with HITL rules and a tool call is interrupted, submit a decision:
+When the agent is configured with HITL rules and one or more tool calls are
+interrupted, submit decisions via `POST /api/v1/chat/{thread_id}`. The preferred
+payload is a `decisions` list (one entry per interrupted tool call):
 
 ```bash
-curl -X POST http://localhost:8000/api/v1/threads/a1b2c3d4-e5f6-7890-abcd-ef1234567890/hitl \
+curl -X POST http://localhost:8000/api/v1/chat/a1b2c3d4-e5f6-7890-abcd-ef1234567890 \
   -H "Content-Type: application/json" \
   -H "X-API-Key: <API_KEY>" \
   -d '{
-    "tool_call_id": "call_abc123",
-    "action": "approve"
+    "decisions": [
+      {"tool_call_id": "call_abc123", "action": "approve"}
+    ]
   }'
 ```
 
@@ -937,33 +940,53 @@ Response (`200`):
   "content": "Action approved. Proceeding with file write.",
   "timestamp": "2025-01-15T10:31:00.000000",
   "tool_calls": null,
-  "tool_call_id": null
+  "status": "completed"
 }
 ```
 
+A legacy single-decision shape (`tool_call_id` + `action`) is still accepted and
+internally converted to a one-element `decisions` list.
+
 ### 11. HITL -- Reject a Pending Tool Call
 
 ```bash
-curl -X POST http://localhost:8000/api/v1/threads/a1b2c3d4-e5f6-7890-abcd-ef1234567890/hitl \
+curl -X POST http://localhost:8000/api/v1/chat/a1b2c3d4-e5f6-7890-abcd-ef1234567890 \
   -H "Content-Type: application/json" \
   -H "X-API-Key: <API_KEY>" \
   -d '{
-    "tool_call_id": "call_abc123",
-    "action": "reject",
-    "reason": "This operation is too risky for production."
+    "decisions": [
+      {"tool_call_id": "call_abc123", "action": "reject", "reason": "This operation is too risky for production."}
+    ]
   }'
 ```
 
 ### 12. HITL -- Edit and Approve a Pending Tool Call
 
 ```bash
-curl -X POST http://localhost:8000/api/v1/threads/a1b2c3d4-e5f6-7890-abcd-ef1234567890/hitl \
+curl -X POST http://localhost:8000/api/v1/chat/a1b2c3d4-e5f6-7890-abcd-ef1234567890 \
+  -H "Content-Type: application/json" \
+  -H "X-API-Key: <API_KEY>" \
+  -d '{
+    "decisions": [
+      {"tool_call_id": "call_abc123", "action": "edit", "edits": {"filename": "safe_output.txt", "content": "sanitized content"}}
+    ]
+  }'
+```
+
+### 12b. HITL -- Multiple Tool Calls in One Resume
+
+When several tool calls are interrupted in the same turn, provide one decision
+per tool call (positional order matches the interrupted actions):
+
+```bash
+curl -X POST http://localhost:8000/api/v1/chat/a1b2c3d4-e5f6-7890-abcd-ef1234567890 \
   -H "Content-Type: application/json" \
   -H "X-API-Key: <API_KEY>" \
   -d '{
-    "tool_call_id": "call_abc123",
-    "action": "edit",
-    "edits": {"filename": "safe_output.txt", "content": "sanitized content"}
+    "decisions": [
+      {"tool_call_id": "call_001", "action": "approve"},
+      {"tool_call_id": "call_002", "action": "reject", "reason": "Not safe"}
+    ]
   }'
 ```
 

src/application/requests/chat.py

@@ -2,22 +2,38 @@
 
 from pydantic import BaseModel, Field, model_validator
 
+from src.domain.entities.hitl_decision import HitlDecision
+
 
 class ChatRequest(BaseModel):
-    """Request body for sending a chat message or an HITL decision."""
+    """Request body for sending a chat message or an HITL decision.
+
+    Exactly one of the following must be provided:
+        * ``message`` — a human message, or
+        * ``decisions`` — a non-empty list of HITL decisions (new contract), or
+        * ``tool_call_id`` + ``action`` — a single legacy HITL decision.
+    """
 
     message: str | None = Field(default=None, min_length=1)
     tool_call_id: str | None = Field(default=None, min_length=1)
     action: Literal["approve", "reject", "edit"] | None = None
     reason: str | None = None
     edits: dict | None = None
+    decisions: list[HitlDecision] | None = None
 
     @model_validator(mode="after")
     def validate_input(self) -> Self:
         has_message = self.message is not None
         has_hitl = self.tool_call_id is not None
-        if has_message == has_hitl:
-            raise ValueError("Provide either 'message' or HITL fields (tool_call_id + action), not both.")
+        has_decisions = self.decisions is not None
+        if has_message and (has_hitl or has_decisions):
+            raise ValueError("Provide either 'message' or HITL fields (tool_call_id + action / decisions), not both.")
+        if has_decisions and has_hitl:
+            raise ValueError("'decisions' is mutually exclusive with legacy 'tool_call_id' + 'action'.")
+        if not has_message and not has_decisions and not has_hitl:
+            raise ValueError("Provide either 'message', 'decisions', or HITL fields (tool_call_id + action).")
+        if has_decisions and len(self.decisions) == 0:  # type: ignore[arg-type]
+            raise ValueError("'decisions' must be a non-empty list.")
         if has_hitl and self.action is None:
             raise ValueError("'action' is required for HITL decisions.")
         if self.action == "edit" and self.edits is None:

src/application/routes/chat.py

@@ -54,6 +54,7 @@ async def send_message(
         tool_call_id=body.tool_call_id,
         reason=body.reason,
         edits=body.edits,
+        decisions=body.decisions,
     )
     logger.info(LogMessage.CHAT_RESPONSE, thread_id, result.status, len(result.content or ""))
     return result

src/application/use_cases/send_message.py

@@ -1,17 +1,19 @@
-"""SendMessageUseCase — send a message or an HITL decision to the agent.
-
-Ticket 3 rewrite: the use case now depends on TraceEventRepository + the new
-runner API ``invoke(thread_id, message, turn_id) -> (Message, list[TraceEvent])``.
-The full trace is persisted in a single batch via ``trace_repo.add_batch``.
-The HITL path (approve/reject/edit) returns the runner Message directly
-without persisting trace events.
+"""SendMessageUseCase — send a message or HITL decisions to the agent.
+
+The use case depends on TraceEventRepository + the runner API
+``invoke(thread_id, message, turn_id) -> (Message, list[TraceEvent])`` and the
+unified HITL resume method
+``resume_hitl(thread_id, decisions, turn_id) -> (Message, list[TraceEvent])``.
+The full trace is persisted in a single batch via ``trace_repo.add_batch`` for
+both the human-message and the HITL resume paths.
 """
 
 import logging
 import time
 import uuid
 from typing import Any
 
+from src.domain.entities.hitl_decision import HitlDecision
 from src.domain.entities.message import Message
 from src.domain.errors.hitl import InvalidHitlActionError
 from src.domain.errors.messages import ErrorMessage
@@ -24,14 +26,17 @@
 
 
 class SendMessageUseCase:
-    """Send a human message or an HITL decision to the agent and return the response.
+    """Send a human message or HITL decisions to the agent and return the response.
 
     For a human message: generates a fresh ``turn_id``, invokes the runner,
     persists the full trace in a batch, and returns the final AI Message.
 
-    For HITL decisions (approve/reject/edit): calls the corresponding runner
-    method and returns the Message directly (no trace persistence — HITL does
-    not currently emit trace events).
+    For HITL decisions (approve/reject/edit, single or multiple): generates a
+    fresh ``turn_id``, calls ``runner.resume_hitl`` with the decisions list,
+    persists the returned trace in a batch, and returns the final AI Message.
+    The legacy single-decision shape (``action`` + ``tool_call_id`` +
+    optional ``reason``/``edits``) is converted into a one-element decisions
+    list for backward compatibility.
     """
 
     def __init__(self, registry: AgentRegistry, threads: ThreadRepository, trace_repo: TraceEventRepository) -> None:
@@ -45,19 +50,21 @@ async def execute(
         *,
         message: str | None = None,
         action: str | None = None,
-        tool_call_id: str | None = None,  # noqa: ARG002
-        reason: str | None = None,  # noqa: ARG002
-        edits: dict[str, Any] | None = None,  # noqa: ARG002
+        tool_call_id: str | None = None,
+        reason: str | None = None,
+        edits: dict[str, Any] | None = None,
+        decisions: list[HitlDecision] | None = None,
     ) -> Message:
         """Execute the use case.
 
         Args:
             thread_id: Conversation thread identifier.
             message: Human message text (mutually exclusive with HITL fields).
-            action: HITL action ("approve", "reject", "edit").
-            tool_call_id: Tool call id targeted by the HITL decision.
-            reason: Optional reject reason.
-            edits: Edited args for the "edit" action.
+            action: Legacy single HITL action ("approve", "reject", "edit").
+            tool_call_id: Tool call id targeted by a legacy single HITL decision.
+            reason: Optional reject reason (legacy single decision).
+            edits: Edited args for the "edit" action (legacy single decision).
+            decisions: List of HITL decisions (new multi-decision contract).
 
         Returns:
             The final AI Message.
@@ -67,13 +74,9 @@ async def execute(
             AgentError: On runner failure.
             ThreadNotFoundError: If the thread does not exist.
         """
-        # Validate HITL action name up-front to keep the 422 contract intact.
-        if message is None:
-            match action:
-                case "approve" | "reject" | "edit":
-                    pass
-                case _:
-                    raise InvalidHitlActionError(ErrorMessage.INVALID_HITL_ACTION.format(action=action))
+        is_hitl = message is None
+        if is_hitl and decisions is None and action not in {"approve", "reject", "edit"}:
+            raise InvalidHitlActionError(ErrorMessage.INVALID_HITL_ACTION.format(action=action))
 
         thread = await self._threads.get(thread_id)
         runner = await self._registry.get_runner(thread.agent_name)
@@ -83,7 +86,6 @@ async def execute(
             turn_id = str(uuid.uuid4())
             start = time.monotonic()
             final_message, trace = await runner.invoke(thread_id, message, turn_id)
-            # Persist all trace events of the turn in a single batch.
             await self._trace_repo.add_batch(thread_id, trace)
             elapsed = time.monotonic() - start
             logger.info(
@@ -96,19 +98,14 @@ async def execute(
             )
             return final_message
 
-        # HITL path — returns the runner Message directly, no trace persistence.
-        logger.info(LogMessage.CHAT_HITL_RECEIVED, thread_id, thread.agent_name, action, tool_call_id)
+        if decisions is None:
+            decisions = [HitlDecision(tool_call_id=tool_call_id, action=action, reason=reason, edits=edits)]  # type: ignore[arg-type]
+
+        logger.info(LogMessage.CHAT_HITL_RECEIVED, thread_id, thread.agent_name, "decisions", len(decisions))
+        turn_id = str(uuid.uuid4())
         start = time.monotonic()
-        match action:
-            case "approve":
-                response = await runner.approve_hitl(thread_id, tool_call_id)  # type: ignore[arg-type]
-            case "reject":
-                response = await runner.reject_hitl(thread_id, tool_call_id, reason)  # type: ignore[arg-type]
-            case "edit":
-                response = await runner.edit_hitl(thread_id, tool_call_id, edits)  # type: ignore[arg-type]
-            case _:
-                # Defensive — already validated above, but keeps mypy happy.
-                raise InvalidHitlActionError(ErrorMessage.INVALID_HITL_ACTION.format(action=action))
+        final_message, trace = await runner.resume_hitl(thread_id, decisions, turn_id)
+        await self._trace_repo.add_batch(thread_id, trace)
         elapsed = time.monotonic() - start
-        logger.info(LogMessage.CHAT_HITL_COMPLETE, thread_id, thread.agent_name, elapsed, response.status)
-        return response
+        logger.info(LogMessage.CHAT_HITL_COMPLETE, thread_id, thread.agent_name, elapsed, final_message.status)
+        return final_message

src/domain/entities/agent_config.py

@@ -12,7 +12,7 @@ class BackendType(StrEnum):
 
 class BackendConfig(BaseModel):
     type: BackendType = BackendType.STORE
-    checkpoint_backend: Literal["memory", "postgres"] = "memory"
+    checkpoint_backend: Literal["memory", "postgres"] = "postgres"
 
 
 class InterruptRule(BaseModel):

src/domain/entities/hitl_decision.py

@@ -0,0 +1,25 @@
+"""HITL decision domain entity.
+
+A ``HitlDecision`` represents a single human decision (approve / reject / edit)
+applied to one interrupted tool call during a Human-In-The-Loop resume flow.
+"""
+
+from typing import Literal
+
+from pydantic import BaseModel
+
+
+class HitlDecision(BaseModel):
+    """A single human decision for an interrupted tool call.
+
+    Attributes:
+        tool_call_id: The id of the interrupted tool call this decision targets.
+        action: The decision kind: ``"approve"``, ``"reject"`` or ``"edit"``.
+        reason: Optional reject reason (ignored for approve/edit).
+        edits: Edited args dict, required for ``action="edit"``.
+    """
+
+    tool_call_id: str
+    action: Literal["approve", "reject", "edit"]
+    reason: str | None = None
+    edits: dict | None = None

src/domain/entities/trace_event.py

@@ -25,6 +25,7 @@ class TraceEventType(StrEnum):
     CONTENT = "content"
     TOOL_CALL = "tool_call"
     TOOL_RESULT = "tool_result"
+    HITL_DECISION = "hitl_decision"
 
 
 class TraceEvent(BaseModel, frozen=True):

src/domain/errors/messages.py

@@ -44,6 +44,18 @@ class ErrorMessage(StrEnum):
     AGENT_HITL_APPROVE_ERROR = "HITL approve error: {error}"
     AGENT_HITL_REJECT_ERROR = "HITL reject error: {error}"
     AGENT_HITL_EDIT_ERROR = "HITL edit error: {error}"
+    AGENT_HITL_RESUME_ERROR = "HITL resume error: {error}"
+    AGENT_HITL_NO_PENDING_INTERRUPT = (
+        "HITL resume failed: no pending interrupt to resume (nothing to resume) for thread {thread_id}."
+    )
+    AGENT_HITL_UNKNOWN_TOOL_CALL = (
+        "HITL resume failed: unknown tool_call_id {tool_call_id} (not found in pending interrupts) "
+        "for thread {thread_id}."
+    )
+    AGENT_HITL_MISSING_DECISION = (
+        "HITL resume failed: missing decision for {pending} interrupted tool call(s) "
+        "(mismatch with provided {provided}) for thread {thread_id}."
+    )
     AGENT_STREAM_IDLE_TIMEOUT = (
         "Agent stream idle for {timeout}s (thread={thread_id}); aborting — a tool result "
         "was likely lost (flaky transport)."

src/domain/logging/messages.py

@@ -172,6 +172,11 @@ class LogMessage(StrEnum):
     HITL_EDIT = "[thread=%s] HITL edit, tool_call_id=%s"
     HITL_EDIT_COMPLETE = "[thread=%s] HITL edit complete, elapsed=%.2fs"
     HITL_EDIT_ERROR_LOG = "HITL edit error"
+    HITL_RESUME = "[thread=%s] HITL resume, decisions=%d"
+    HITL_RESUME_COMPLETE = "[thread=%s] HITL resume complete, elapsed=%.2fs, status=%s"
+    HITL_RESUME_ERROR_LOG = "HITL resume error"
+    HITL_NO_PENDING_INTERRUPT = "[thread=%s] HITL resume aborted: no pending interrupt"
+    CHECKPOINTER_POSTGRES_UNAVAILABLE = "Postgres checkpointer unavailable, falling back to MemorySaver: %s"
 
     # --- DeepAgent runner / ToolNode patching ---
     TOOLS_NODE_MISSING = "No 'tools' node found in graph; cannot patch handle_tool_errors"