diff --git a/docs/actions/structured_output.md b/docs/actions/structured_output.md index eaf6de92..271400a7 100644 --- a/docs/actions/structured_output.md +++ b/docs/actions/structured_output.md @@ -20,7 +20,7 @@ Two JSON response formats: | Provider | `json_object` | `json_schema` | Notes | |:---------------|:-------------:|:-------------:|:------| | **OpenAI** | 🟩 | 🟩 | Native support with strict mode (Responses API only for json_schema) | -| **Anthropic** | 🟦 | ❌ | Emulated via prompt engineering technique | +| **Anthropic** | 🟦 | 🟩 | json_object emulated; json_schema native via output_config.format on supported Claude models | | **OpenRouter** | 🟩 | 🟩 | Native support, depends on underlying model | | **Ollama** | 🟨 | 🟨 | Model-dependent, support varies by model | | **RubyLLM** | 🟨 | 🟨 | Depends on underlying provider/model | diff --git a/docs/providers/anthropic.md b/docs/providers/anthropic.md index c5698fbd..08d224cd 100644 --- a/docs/providers/anthropic.md +++ b/docs/providers/anthropic.md @@ -119,12 +119,65 @@ Anthropic provides access to the Claude model family. For the complete list of a ### Response Format -- **`response_format`** - Control output format (see [Emulated JSON Object Support](#emulated-json-object-support)) +- **`response_format`** - Control output format (see [JSON Object Support](#emulated-json-object-support) and [JSON Schema Support](#json-schema-support)) ### Streaming - **`stream`** - Enable streaming responses (boolean, default: false) +## JSON Schema Support + +ActiveAgent maps `response_format: :json_schema` (or the hash form) to Anthropic's native `output_config.format` API. JSON Schema Hashes are passed through as-is; ActiveAgent does not convert them to `Anthropic::BaseModel`. + +### Usage + +```ruby +class ColorsAgent < ApplicationAgent + generate_with :anthropic, model: "claude-haiku-4-5" + + def primary_colors + prompt( + "Return the three primary colors.", + response_format: :json_schema + ) + end +end +``` + +Place a schema file at `app/views/agents/colors_agent/primary_colors.json`: + +```json +{ + "schema": { + "type": "object", + "properties": { + "colors": { "type": "array", "items": { "type": "string" } } + }, + "required": ["colors"], + "additionalProperties": false + } +} +``` + +ActiveAgent serializes the request as: + +```ruby +{ + output_config: { + format: { + type: "json_schema", + schema: { ... } + } + } +} +``` + +### Notes + +- `name` and `strict` from the common format are not forwarded; Anthropic's `output_config.format` does not use them. +- JSON Schema support availability depends on model version. Check [Anthropic's documentation](https://platform.claude.com/docs/en/build-with-claude/structured-outputs) for supported models. +- `json_object` emulation via prompt engineering is separate and unchanged. + ## Emulated JSON Object Support While Anthropic does not natively support structured response formats like OpenAI's `json_object` mode, ActiveAgent provides emulated support through a prompt engineering technique. @@ -155,7 +208,7 @@ Unlike OpenAI's native JSON mode: - **Prompt-dependent reliability**: Success depends on clear prompt instructions - **No strict mode**: Cannot guarantee specific field requirements -For applications requiring guaranteed schema conformance, consider using the [Structured Output](/actions/structured_output) feature with providers that support native JSON schema validation. +For applications requiring guaranteed schema conformance, use the [Structured Output](/actions/structured_output) feature with `response_format: :json_schema`. Anthropic natively supports JSON schema validation via `output_config.format` on supported Claude models. ## Constitutional AI diff --git a/lib/active_agent/providers/anthropic/request.rb b/lib/active_agent/providers/anthropic/request.rb index ccc33b91..8ab7ce29 100644 --- a/lib/active_agent/providers/anthropic/request.rb +++ b/lib/active_agent/providers/anthropic/request.rb @@ -71,7 +71,9 @@ class Request < SimpleDelegator # @raise [ArgumentError] when gem model validation fails def initialize(**params) # Step 1: Extract custom fields that gem doesn't support - @response_format = params.delete(:response_format) + # Read response_format without deleting - normalize_params will delete and convert it + # to output_config for json_schema, or drop it for other types. + @response_format = params[:response_format] @stream = params.delete(:stream) anthropic_beta = params.delete(:anthropic_beta) diff --git a/lib/active_agent/providers/anthropic/transforms.rb b/lib/active_agent/providers/anthropic/transforms.rb index 747ed2e0..43fb85d1 100644 --- a/lib/active_agent/providers/anthropic/transforms.rb +++ b/lib/active_agent/providers/anthropic/transforms.rb @@ -31,6 +31,12 @@ def normalize_params(params) params[:tools] = normalize_tools(params[:tools]) if params[:tools] params[:tool_choice] = normalize_tool_choice(params[:tool_choice]) if params[:tool_choice] + # Normalize json_schema response_format → output_config (Anthropic's native structured output field) + if params[:response_format] + output_config = normalize_response_format(params.delete(:response_format)) + params[:output_config] = output_config if output_config + end + # Handle mcps parameter (common format) -> transforms to mcp_servers (provider format) if params[:mcps] params[:mcp_servers] = normalize_mcp_servers(params.delete(:mcps)) @@ -159,6 +165,52 @@ def normalize_tool_choice(tool_choice) end end + # Normalizes response_format to Anthropic output_config structure. + # + # Supported (ActiveAgent common format): + # { type: "json_schema", json_schema: { name: "...", schema: {...}, strict: true } } + # → { format: { type: "json_schema", schema: {...} } } + # + # Notes: + # - Anthropic does not use OpenAI's `name` or `strict` fields in output_config.format. + # - json_object is not handled here; it remains prompt-emulated. + # - text is not handled here; Anthropic returns plain text by default. + # + # @param format [Hash, Symbol, String] ActiveAgent common response_format + # @return [Hash] Anthropic output_config hash, or nil if not applicable + def normalize_response_format(format) + case format + when Hash + format_hash = format.deep_symbolize_keys + + if format_hash[:type].to_s == "json_schema" + { + format: { + type: "json_schema", + schema: format_hash[:json_schema]&.dig(:schema) + } + } + elsif format_hash[:type].to_s == "json_object" + # json_object is not handled here; it remains prompt-emulated. + nil + elsif format_hash[:type].to_s == "text" + # text is not handled here; it remains prompt-emulated. + nil + else + # Pass through (already properly structured or Anthropic native format) + format_hash + end + when Symbol, String + if format.to_s == "json_schema" + { format: { type: "json_schema" } } + else + nil + end + else + format + end + end + # Merges consecutive same-role messages into single messages with multiple content blocks. # # Required by Anthropic API - consecutive messages with the same role must be combined. diff --git a/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_implicit.yml b/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_implicit.yml new file mode 100644 index 00000000..211756f1 --- /dev/null +++ b/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_implicit.yml @@ -0,0 +1,36 @@ +--- +http_interactions: +- request: + method: post + uri: https://api.anthropic.com/v1/messages + body: + encoding: UTF-8 + string: '{"model":"claude-haiku-4-5","messages":[{"content":"Return the three + primary colors.","role":"user"}],"max_tokens":1024,"output_config":{"format":{"type":"json_schema","schema":{"type":"object","properties":{"colors":{"type":"array","items":{"type":"string"}}},"required":["colors"],"additionalProperties":false}}}}' + headers: + Accept-Encoding: + - gzip;q=1.0,deflate;q=0.6,identity;q=0.3 + Accept: + - application/json + User-Agent: + - Ruby + Host: + - api.anthropic.com + Content-Type: + - application/json + Anthropic-Version: + - '2023-06-01' + X-Api-Key: + - ACCESS_TOKEN + response: + status: + code: 200 + message: OK + headers: + Content-Type: + - application/json + body: + encoding: ASCII-8BIT + string: '{"model":"claude-haiku-4-5-20251001","id":"msg_01ImplicitJsonSchema","type":"message","role":"assistant","content":[{"type":"text","text":"{\"colors\":[\"red\",\"yellow\",\"blue\"]}"}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":25,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0},"output_tokens":18,"service_tier":"standard"}}' + recorded_at: Sat, 31 May 2026 00:00:00 GMT +recorded_with: VCR 6.3.1 diff --git a/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_implicit_bare.yml b/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_implicit_bare.yml new file mode 100644 index 00000000..5e092c16 --- /dev/null +++ b/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_implicit_bare.yml @@ -0,0 +1,36 @@ +--- +http_interactions: +- request: + method: post + uri: https://api.anthropic.com/v1/messages + body: + encoding: UTF-8 + string: '{"model":"claude-haiku-4-5","messages":[{"content":"Return the three + primary colors.","role":"user"}],"max_tokens":1024,"output_config":{"format":{"type":"json_schema","schema":{"type":"object","properties":{"colors":{"type":"array","items":{"type":"string"}}},"required":["colors"],"additionalProperties":false}}}}' + headers: + Accept-Encoding: + - gzip;q=1.0,deflate;q=0.6,identity;q=0.3 + Accept: + - application/json + User-Agent: + - Ruby + Host: + - api.anthropic.com + Content-Type: + - application/json + Anthropic-Version: + - '2023-06-01' + X-Api-Key: + - ACCESS_TOKEN + response: + status: + code: 200 + message: OK + headers: + Content-Type: + - application/json + body: + encoding: ASCII-8BIT + string: '{"model":"claude-haiku-4-5-20251001","id":"msg_01ImplicitBareJsonSchema","type":"message","role":"assistant","content":[{"type":"text","text":"{\"colors\":[\"red\",\"yellow\",\"blue\"]}"}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":25,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0},"output_tokens":18,"service_tier":"standard"}}' + recorded_at: Sat, 31 May 2026 00:00:00 GMT +recorded_with: VCR 6.3.1 diff --git a/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_inline.yml b/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_inline.yml new file mode 100644 index 00000000..e7bcb690 --- /dev/null +++ b/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_inline.yml @@ -0,0 +1,36 @@ +--- +http_interactions: +- request: + method: post + uri: https://api.anthropic.com/v1/messages + body: + encoding: UTF-8 + string: '{"model":"claude-haiku-4-5","messages":[{"content":"Return the three + primary colors.","role":"user"}],"max_tokens":1024,"output_config":{"format":{"type":"json_schema","schema":{"type":"object","properties":{"colors":{"type":"array","items":{"type":"string"}}},"required":["colors"],"additionalProperties":false}}}}' + headers: + Accept-Encoding: + - gzip;q=1.0,deflate;q=0.6,identity;q=0.3 + Accept: + - application/json + User-Agent: + - Ruby + Host: + - api.anthropic.com + Content-Type: + - application/json + Anthropic-Version: + - '2023-06-01' + X-Api-Key: + - ACCESS_TOKEN + response: + status: + code: 200 + message: OK + headers: + Content-Type: + - application/json + body: + encoding: ASCII-8BIT + string: '{"model":"claude-haiku-4-5-20251001","id":"msg_01InlineJsonSchema","type":"message","role":"assistant","content":[{"type":"text","text":"{\"colors\":[\"red\",\"yellow\",\"blue\"]}"}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":25,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0},"output_tokens":18,"service_tier":"standard"}}' + recorded_at: Sat, 31 May 2026 00:00:00 GMT +recorded_with: VCR 6.3.1 diff --git a/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_named.yml b/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_named.yml new file mode 100644 index 00000000..98e48cb9 --- /dev/null +++ b/test/fixtures/vcr_cassettes/integration/anthropic/common_format/response_format_test/test_agent_response_json_schema_named.yml @@ -0,0 +1,36 @@ +--- +http_interactions: +- request: + method: post + uri: https://api.anthropic.com/v1/messages + body: + encoding: UTF-8 + string: '{"model":"claude-haiku-4-5","messages":[{"content":"Return the three + primary colors.","role":"user"}],"max_tokens":1024,"output_config":{"format":{"type":"json_schema","schema":{"type":"object","properties":{"colors":{"type":"array","items":{"type":"string"}}},"required":["colors"],"additionalProperties":false}}}}' + headers: + Accept-Encoding: + - gzip;q=1.0,deflate;q=0.6,identity;q=0.3 + Accept: + - application/json + User-Agent: + - Ruby + Host: + - api.anthropic.com + Content-Type: + - application/json + Anthropic-Version: + - '2023-06-01' + X-Api-Key: + - ACCESS_TOKEN + response: + status: + code: 200 + message: OK + headers: + Content-Type: + - application/json + body: + encoding: ASCII-8BIT + string: '{"model":"claude-haiku-4-5-20251001","id":"msg_01NamedJsonSchema","type":"message","role":"assistant","content":[{"type":"text","text":"{\"colors\":[\"red\",\"yellow\",\"blue\"]}"}],"stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":25,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0},"output_tokens":18,"service_tier":"standard"}}' + recorded_at: Sat, 31 May 2026 00:00:00 GMT +recorded_with: VCR 6.3.1 diff --git a/test/integration/anthropic/common_format/response_format_test.rb b/test/integration/anthropic/common_format/response_format_test.rb index 9183f61c..ce811c5d 100644 --- a/test/integration/anthropic/common_format/response_format_test.rb +++ b/test/integration/anthropic/common_format/response_format_test.rb @@ -73,7 +73,23 @@ def response_json_object content: "Return the three primary colors." } ], - max_tokens: 1024 + max_tokens: 1024, + output_config: { + format: { + type: "json_schema", + schema: { + type: "object", + properties: { + colors: { + type: "array", + items: { type: "string" } + } + }, + required: [ "colors" ], + additionalProperties: false + } + } + } } def response_json_schema_inline @@ -141,10 +157,10 @@ def response_json_schema_named end [ - # :response_json_schema_inline, - # :response_json_schema_implicit, - # :response_json_schema_named, - # :response_json_schema_implicit_bare + :response_json_schema_inline, + :response_json_schema_implicit, + :response_json_schema_named, + :response_json_schema_implicit_bare ].each do |action_name| test_request_builder(TestAgent, action_name, :generate_now, TestAgent::REQUEST_JSON_SCHEMA) end @@ -175,19 +191,19 @@ def response_json_schema_named end end - # test "response format: json_schema (implicit bare)" do - # agent_name = TestAgent.name.demodulize.underscore - # action_name = "response_json_schema_implicit_bare" - # cassette_name = [ self.class.name.underscore, "#{agent_name}_#{action_name}" ].join("/") + test "response format: json_schema (implicit bare)" do + agent_name = TestAgent.name.demodulize.underscore + action_name = "response_json_schema_implicit_bare" + cassette_name = [ self.class.name.underscore, "#{agent_name}_#{action_name}" ].join("/") - # VCR.use_cassette(cassette_name) do - # response = TestAgent.response_json_schema_implicit_bare.generate_now + VCR.use_cassette(cassette_name) do + response = TestAgent.response_json_schema_implicit_bare.generate_now - # assert_equal "json_schema", response.format.type - # assert_not_nil response.message.parsed_json - # assert_kind_of Array, response.message.parsed_json[:colors] - # end - # end + assert_equal "json_schema", response.format.type + assert_not_nil response.message.parsed_json + assert_kind_of Array, response.message.parsed_json[:colors] + end + end end end end