Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/actions/structured_output.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Two JSON response formats:
| Provider | `json_object` | `json_schema` | Notes |
|:---------------|:-------------:|:-------------:|:------|
| **OpenAI** | 🟩 | 🟩 | Native support with strict mode (Responses API only for json_schema) |
| **Anthropic** | 🟦 | | Emulated via prompt engineering technique |
| **Anthropic** | 🟦 | 🟩 | json_object emulated; json_schema native via output_config.format on supported Claude models |
| **OpenRouter** | 🟩 | 🟩 | Native support, depends on underlying model |
| **Ollama** | 🟨 | 🟨 | Model-dependent, support varies by model |
| **RubyLLM** | 🟨 | 🟨 | Depends on underlying provider/model |
Expand Down
57 changes: 55 additions & 2 deletions docs/providers/anthropic.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,12 +119,65 @@ Anthropic provides access to the Claude model family. For the complete list of a

### Response Format

- **`response_format`** - Control output format (see [Emulated JSON Object Support](#emulated-json-object-support))
- **`response_format`** - Control output format (see [JSON Object Support](#emulated-json-object-support) and [JSON Schema Support](#json-schema-support))

### Streaming

- **`stream`** - Enable streaming responses (boolean, default: false)

## JSON Schema Support

ActiveAgent maps `response_format: :json_schema` (or the hash form) to Anthropic's native `output_config.format` API. JSON Schema Hashes are passed through as-is; ActiveAgent does not convert them to `Anthropic::BaseModel`.

### Usage

```ruby
class ColorsAgent < ApplicationAgent
generate_with :anthropic, model: "claude-haiku-4-5"

def primary_colors
prompt(
"Return the three primary colors.",
response_format: :json_schema
)
end
end
```

Place a schema file at `app/views/agents/colors_agent/primary_colors.json`:

```json
{
"schema": {
"type": "object",
"properties": {
"colors": { "type": "array", "items": { "type": "string" } }
},
"required": ["colors"],
"additionalProperties": false
}
}
```

ActiveAgent serializes the request as:

```ruby
{
output_config: {
format: {
type: "json_schema",
schema: { ... }
}
}
}
```

### Notes

- `name` and `strict` from the common format are not forwarded; Anthropic's `output_config.format` does not use them.
- JSON Schema support availability depends on model version. Check [Anthropic's documentation](https://platform.claude.com/docs/en/build-with-claude/structured-outputs) for supported models.
- `json_object` emulation via prompt engineering is separate and unchanged.

## Emulated JSON Object Support

While Anthropic does not natively support structured response formats like OpenAI's `json_object` mode, ActiveAgent provides emulated support through a prompt engineering technique.
Expand Down Expand Up @@ -155,7 +208,7 @@ Unlike OpenAI's native JSON mode:
- **Prompt-dependent reliability**: Success depends on clear prompt instructions
- **No strict mode**: Cannot guarantee specific field requirements

For applications requiring guaranteed schema conformance, consider using the [Structured Output](/actions/structured_output) feature with providers that support native JSON schema validation.
For applications requiring guaranteed schema conformance, use the [Structured Output](/actions/structured_output) feature with `response_format: :json_schema`. Anthropic natively supports JSON schema validation via `output_config.format` on supported Claude models.

## Constitutional AI

Expand Down
4 changes: 3 additions & 1 deletion lib/active_agent/providers/anthropic/request.rb
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,9 @@ class Request < SimpleDelegator
# @raise [ArgumentError] when gem model validation fails
def initialize(**params)
# Step 1: Extract custom fields that gem doesn't support
@response_format = params.delete(:response_format)
# Read response_format without deleting - normalize_params will delete and convert it
# to output_config for json_schema, or drop it for other types.
@response_format = params[:response_format]
@stream = params.delete(:stream)
anthropic_beta = params.delete(:anthropic_beta)

Expand Down
52 changes: 52 additions & 0 deletions lib/active_agent/providers/anthropic/transforms.rb
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ def normalize_params(params)
params[:tools] = normalize_tools(params[:tools]) if params[:tools]
params[:tool_choice] = normalize_tool_choice(params[:tool_choice]) if params[:tool_choice]

# Normalize json_schema response_format → output_config (Anthropic's native structured output field)
if params[:response_format]
output_config = normalize_response_format(params.delete(:response_format))
params[:output_config] = output_config if output_config
end

# Handle mcps parameter (common format) -> transforms to mcp_servers (provider format)
if params[:mcps]
params[:mcp_servers] = normalize_mcp_servers(params.delete(:mcps))
Expand Down Expand Up @@ -159,6 +165,52 @@ def normalize_tool_choice(tool_choice)
end
end

# Normalizes response_format to Anthropic output_config structure.
#
# Supported (ActiveAgent common format):
# { type: "json_schema", json_schema: { name: "...", schema: {...}, strict: true } }
# → { format: { type: "json_schema", schema: {...} } }
#
# Notes:
# - Anthropic does not use OpenAI's `name` or `strict` fields in output_config.format.
# - json_object is not handled here; it remains prompt-emulated.
# - text is not handled here; Anthropic returns plain text by default.
#
# @param format [Hash, Symbol, String] ActiveAgent common response_format
# @return [Hash] Anthropic output_config hash, or nil if not applicable
def normalize_response_format(format)
case format
when Hash
format_hash = format.deep_symbolize_keys

if format_hash[:type].to_s == "json_schema"
{
format: {
type: "json_schema",
schema: format_hash[:json_schema]&.dig(:schema)
}
}
elsif format_hash[:type].to_s == "json_object"
# json_object is not handled here; it remains prompt-emulated.
nil
elsif format_hash[:type].to_s == "text"
# text is not handled here; it remains prompt-emulated.
nil
else
# Pass through (already properly structured or Anthropic native format)
format_hash
end
when Symbol, String
if format.to_s == "json_schema"
{ format: { type: "json_schema" } }
else
nil
end
else
format
end
end

# Merges consecutive same-role messages into single messages with multiple content blocks.
#
# Required by Anthropic API - consecutive messages with the same role must be combined.
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 32 additions & 16 deletions test/integration/anthropic/common_format/response_format_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,23 @@ def response_json_object
content: "Return the three primary colors."
}
],
max_tokens: 1024
max_tokens: 1024,
output_config: {
format: {
type: "json_schema",
schema: {
type: "object",
properties: {
colors: {
type: "array",
items: { type: "string" }
}
},
required: [ "colors" ],
additionalProperties: false
}
}
}
}

def response_json_schema_inline
Expand Down Expand Up @@ -141,10 +157,10 @@ def response_json_schema_named
end

[
# :response_json_schema_inline,
# :response_json_schema_implicit,
# :response_json_schema_named,
# :response_json_schema_implicit_bare
:response_json_schema_inline,
:response_json_schema_implicit,
:response_json_schema_named,
:response_json_schema_implicit_bare
].each do |action_name|
test_request_builder(TestAgent, action_name, :generate_now, TestAgent::REQUEST_JSON_SCHEMA)
end
Expand Down Expand Up @@ -175,19 +191,19 @@ def response_json_schema_named
end
end

# test "response format: json_schema (implicit bare)" do
# agent_name = TestAgent.name.demodulize.underscore
# action_name = "response_json_schema_implicit_bare"
# cassette_name = [ self.class.name.underscore, "#{agent_name}_#{action_name}" ].join("/")
test "response format: json_schema (implicit bare)" do
agent_name = TestAgent.name.demodulize.underscore
action_name = "response_json_schema_implicit_bare"
cassette_name = [ self.class.name.underscore, "#{agent_name}_#{action_name}" ].join("/")

# VCR.use_cassette(cassette_name) do
# response = TestAgent.response_json_schema_implicit_bare.generate_now
VCR.use_cassette(cassette_name) do
response = TestAgent.response_json_schema_implicit_bare.generate_now

# assert_equal "json_schema", response.format.type
# assert_not_nil response.message.parsed_json
# assert_kind_of Array, response.message.parsed_json[:colors]
# end
# end
assert_equal "json_schema", response.format.type
assert_not_nil response.message.parsed_json
assert_kind_of Array, response.message.parsed_json[:colors]
end
end
end
end
end
Expand Down
Loading