Storyboard fixtures as requirements, not injected rows: satisfy them through the seller's normal AdCP surface instead of runner-chosen `seed_*` IDs

[torian257x]

Problem

Every seed_* scenario of comply_test_controller shares one contract: the runner picks the entity ID, and the seller must store it verbatim, surface it through its read tools, and accept it back in buyer calls — most controller storyboards reference the seeded literal directly (e.g. governance-delivery-monitor seeds outdoor_display_q2, then sends packages[].product_id: "outdoor_display_q2" to create_media_buy). SeedSuccess is message-only, so the seller has no channel to answer with its own ID.

We operate a multi-tenant ad-management platform (goTom) with an AdCP sales agent on top, and for sellers like us this contract is unimplementable — for every seeded entity type, not just products:

• Our entities are not rows in a fixture store. A product is a valid, priced combination derived at read time from an underlying inventory model (rate card, publisher, placement, pricing option, …). A media buy is the outcome of a real booking transaction; an account is a real customer record.
• Our IDs are derived keys, not free-form strings. create_media_buy decodes the product ID and resolves its components against real entities before booking. A runner-chosen string is not "a product we haven't created yet" — it is unparseable. The same holds for pricing_option_id, format IDs, media_buy_id, account_id.
• Inventory is publisher-owned. Publishers provision it through the platform; the AdCP agent deliberately has no write path into the catalog.

The only workaround is an alias table plus interception of every read and write path to translate runner IDs into real ones — a parallel, test-only code path. That is exactly the teach-to-test trap the docs warn against, and it means the graded surface is the alias layer, not the production pipeline. (TestControllerBridge doesn't close the gap either: a fixture merged into a read response still cannot be bought.)

Opting out is not viable when the goal is full coverage: UNKNOWN_SCENARIO on a required seed grades the whole storyboard not_applicable, the media_buy_seller baseline itself declares controller_seeding: true, and Stateful compliance requires seed_product + seed_pricing_option for sales-* specialisms.

Proposal: requirement-based fixture satisfaction

Keep the storyboard fixtures: blocks exactly as they are, but define them as requirements ("a non-guaranteed display product with a CPM/USD pricing option must be available") rather than rows to inject. The runner satisfies each requirement through an ordered strategy ladder and binds the fixture's ID as a session-scoped handle to whatever entity satisfied it — using the substitution machinery that already exists (context_outputs / $context.*). Storyboard YAML does not change.

Strategy	Applies to	Mechanism
1. Seed (today's behavior)	Sellers advertising the matching seed_* scenario	Push as today; handle = seeded ID verbatim. Zero change for existing implementers.
2. Discover	Catalog entities: products, pricing options, creative formats	Enumerate the normal read path (get_products under the sandbox account, list_creative_formats), select the first entity satisfying the requirement fields in spec-defined order, bind the handle to its real IDs
3. Construct	Lifecycle entities: creatives, media buys	Drive the real buyer flow (sync_creatives; create_media_buy against an already-bound product), then reach the required state via force_*; bind the handle to the seller-minted ID
4. Coverage gap	Anything unsatisfied	New skip reason (e.g. fixture_unsatisfied) graded not_applicable — same honest-coverage philosophy as UNKNOWN_SCENARIO today

For a platform seller the fixture surface becomes the platform itself: we pre-provision deterministic test inventory in the sandbox tenant through our own provisioning path, and the runner finds it the way any buyer would. The only controller surface still needed is the one that genuinely requires a control channel — force_* / simulate_* — which we are implementing regardless (force_media_buy_status, simulate_delivery, simulate_budget_spend, force_create_media_buy_arm / force_task_completion).

Determinism. Requirement fields are exactly the fields fixtures carry today; selection order is spec-defined; within a session a handle stays pinned to the same entity. Storyboards that need exact seeded values (not shapes) keep strategy 1 as their only path and otherwise grade not_applicable — accurate coverage reporting, same philosophy as requires_capability gates.

Anti-gaming gets stronger. Under strategies 2–3 the graded flows exercise the seller's production catalog and booking pipeline end-to-end, and there are no magic ID literals a seller could special-case, because the seller never sees them.

This is not new machinery. The suite and SDK already use every ingredient: pagination_integrity_list_accounts bootstraps via sync_accounts; the content-standards / collection-lists / property-lists pagination storyboards bootstrap via real create_* calls; sales-non-guaranteed already binds create_media_buy to a discovered $context.first_product_id; and the @adcp/sdk conformance fuzzer's auto-seed tier discovers seller IDs instead of pushing them ("Seeded IDs are agent-generated and differ per run"). Meanwhile media buys and creative formats are seeded by ID — entities that could bootstrap exactly like accounts and content standards do today.

Alternative mechanics we'd also accept

• Seller-resolved seed IDs (smaller runner change): keep the seed push, but let the seed response carry the seller's real ID and have the runner bind to it (SeedSuccess variants are additionalProperties: true, so this is additive). Works for us too, but pushes per-entity matching logic into every platform seller instead of once into the runner.
• Statically declared fixtures in get_adcp_capabilities / adagents.json: simple, but covers only catalog entities and risks drift against per-version fixture shapes.

What we are NOT asking to change

• force_* / simulate_* / query_* — unchanged; strategy 3 depends on them, and we are implementing them.
• seed_* — unchanged; it remains strategy 1 for sellers with a fixture store.
• Sandbox gating, the production ban, comply-controller-mode-gate, observational-mode grading — unchanged.
• Storyboard YAML — unchanged; fixture IDs are reinterpreted as session-scoped handles.

Affected areas

• Storyboard runner — discover/construct strategies and handle→entity binding (reuses existing $context machinery); fixture_unsatisfied skip reason in the runner output contract
• Docs — L3 seeding semantics; a platform-seller row in the test-surfaces table; Stateful-compliance wording ("satisfiable fixtures + full force_*/simulate_*" rather than seed_* specifically)
• No wire-schema change is required for the lead proposal

Related: #2584 (seed scenarios spec), #2743 (fixtures blocks), #4226 (UNKNOWN_SCENARIO grading), #4379 ((Sandbox) framing), #5168 (open scenario strings).

Offer

We are building this seller now and are happy to prototype against a spec draft and contribute — in particular the discovery/selection logic, which the SDK's conformance seeder already implements in miniature.

[bokelley]

Triage

Classification: Feature request
Bucket(s): compliance suite (label absent — noted in run summary), spec / protocol
Status: ready-for-human
Milestone: Spec Backlog → 3.1.x

What the experts said:

• ad-tech-protocol-expert: Proposal is protocol-sound; runner-side discovery is strictly stronger anti-gaming; seed_* MUST relaxation is a spec-owner call with WG notification; routes to 3.1.x; one open question — if create_media_buy fails during Construct fixture setup, that must grade as a conformance failure, not silently absorb as fixture_unsatisfied.
• code-reviewer: Significant runner work for the full ladder; real substitution gap in existing storyboards (runtime tokens like governance_context are not fixture IDs and won't be auto-substituted); discover selection-order rule must be pinned; plan and media_buy entities have no discovery API so Construct is their only non-seed path.
• ad-tech-protocol-expert-deep: Three normative surfaces must change together (runner-output-contract, storyboard-schema, L3 MUST language); SeedSuccess alternative is immediately actionable with no schema edit; full discover/construct tier requires a WG decision record before implementation begins.

My take: goTom is genuinely blocked — derived-key IDs are architecturally incompatible with the runner-picks-the-literal contract, and the alias-table workaround is exactly the teach-to-test trap the docs warn against. The SeedSuccess enhancement can unblock them immediately with a minimal, non-breaking runner change; the full strategy ladder needs three explicit decisions before spec language is written.

Thanks for filing!

---

Interim unblock — ready to implement (no human decision needed):

Extend strategy 1: after any seed_* succeeds, the runner binds the fixture handle to a seller-returned ID when present, falling back to the pushed literal for existing implementers. SeedSuccess is already additionalProperties: true — no schema edit required.

Option A — SeedSuccess ID echo:

# seed_product SeedSuccess response — additionalProperties: true already:
 { "success": true,
+  "entity_id": "prod-cat-4729|placement-88|cpm",  // seller's real derived key, optional
   "message": "Fixture seeded" }
# Runner: bind handle "outdoor_display_q2" → response.entity_id if present
# Fallback: bind to pushed literal — zero change for existing seed_* implementers

Implementation scope: define entity_id field name under SeedSuccess shape in L3 doc, update runner binding logic. Ships as minor on 3.1.x. Limitation: platforms that cannot implement seed_* at all still need Discover (products) or Construct (media buys).

---

@bokelley, three open questions before the full strategy ladder can be specced:

1. Discover-tier selection rule when multiple products match the fixture requirement shape:

Option B1 — First match in response order (consistent with sales-non-guaranteed $context.first_product_id precedent):

# Runner selects get_products(sandbox_account)[0]
# where product attrs ⊇ fixture {delivery_type, channels, format_ids}
# Bind handle "outdoor_display_q2" → seller's first matching product_id

Deterministic within a session. Anti-gaming-weak: sellers control catalog sort order.

Option B2 — Fail on ambiguity:

# Runner: >1 product matches predicate → fixture_unsatisfied → storyboard not_applicable
# Forces sellers to provision exactly one matching entity in sandbox

Anti-gaming-strong. Breaks sellers with dense catalogs containing multiple valid test products.

2. Scope of seed_* MUST relaxation in L3 certification tier docs:

Option C1 — Universal (changes the conformance contract for all controller_seeding: true specialisms):

- MUST implement seed_product + seed_pricing_option for sales-* specialisms.
+ MUST satisfy all declared fixtures (via Seed, Discover, or Construct).

Requires a decision record in governance/decisions/ — changes who is eligible for sales-non-guaranteed Stateful compliance.

Option C2 — Targeted (adds a platform-seller row to the test-surfaces table; MUST text unchanged):

+ | Platform seller (derived IDs) | Discover/Construct | seed_* graded not_applicable |

Narrower. Doesn't change the existing MUST for sellers with a fixture store.

3. Construct path disposition:

• Spec now — normatively define per-entity creation sequences (which tasks, which force_* transition). Resolves plan and media_buy types; larger spec work; necessary if the MUST removal is universal (Option C1).
• Defer — ship SeedSuccess + Discover first; plan/media_buy remain seed-only or not_applicable. goTom's primary blocker (products/pricing options) is resolved by Discover + SeedSuccess without needing Construct.

Related: #2584, #2743, #4226, #4379, #5168.

---

Triaged by Claude Code. Session: https://claude.ai/code/session_01SWRAPxKVBMkdK4TuAJbDkH

---

Generated by Claude Code

[bokelley]

Lineage decision: keep current fixture injection stable for 3.1.x. Move the runner-owned ID versus seller-surface fixture contract redesign to 3.2.0 so IDs, setup lifecycle, and compatibility can change coherently.

[sangilish]

The 3.2 lineage decision makes sense. I would use the decision record to separate fixture requirement identity from resolution evidence before choosing runner mechanics:

• the authored fixture ID is a session handle, not an ID the seller must persist;
• resolution records the strategy, bound seller IDs, matched requirement fields, and setup steps;
• Discover uses a protocol-defined deterministic ordering rather than response order or ambiguity failure;
• fixture_unsatisfied means no qualifying capability or entity exists, while a supported setup operation that fails remains a conformance failure;
• current literal injection remains the compatibility strategy for existing implementations.

I can help turn those invariants into a focused 3.2 decision-record sketch without changing the 3.1 runner.