Skip to content

oci-extract: full-SDK sweep — classification, per-service gate, egress/async inventory - #15

Merged
sinkog merged 1 commit into
develfrom
feature/oci-extract-full-sweep
Aug 3, 2026
Merged

oci-extract: full-SDK sweep — classification, per-service gate, egress/async inventory#15
sinkog merged 1 commit into
develfrom
feature/oci-extract-full-sweep

Conversation

@sinkog

@sinkog sinkog commented Aug 3, 2026

Copy link
Copy Markdown
Member

Follow-up to #14, which made the extractor service-agnostic. This runs it across
the whole pinned SDK and settles the three questions that only a full-surface
measurement could answer.

Build-time registry (task B)

A new tools/oci-extract/cmd/oci-sweep loops the resolver over every service
directory of the pinned SDK — no hand-curated file lists — and classifies each
resource by its create surface:

Class Count Meaning
A — read model + resolvable create 685 mechanical; the pipeline handles these
B — read model, no usable create mostly legitimate (WorkRequest, Trace, Log): read-only byproducts
C — no read model not resources; explicitly listed exclusions, not silent skips
D — polymorphic create reported and exit 5; discriminator semantics is a modelling decision, deliberately out of scope

Per-service integrity gate (task C)

The single extracted_schema_hash becomes an extracted_schema_hashes map, so
a diff stays reviewable at full-SDK scale. Two new tests in
tests/test_oci_sdk_lock.py cover that every committed service is present and
that the hashes match. The semantic oci-extract -diff still fails non-zero on
a real break: deleting cidrBlock from module/schemas/core/vcn.json yields
{"breaking":[{"field":"cidrBlock","kind":"removed"}]} and exit 3.

Embedding measured, not estimated (task D)

Built, not guessed: the 685 class-A schema bundles were embedded into a
throwaway copy of module/, compiled with TinyGo, and compared against a
freshly built 2-resource baseline in the same container.

module.wasm, full surface (685 resources) 6 334 147 B (6.04 MiB)
module.wasm, baseline (vcn+subnet) 1 152 869 B
relay default MaxWASMModuleSize 16 777 216 B
full surface as % of the cap 37.8%

The whole class-A surface would fit with 62% headroom. "Fits" is not a reason
to embed services a module never imports
— embedding stays selective by
design; this measurement removes size as the reason to hesitate, nothing more.

Note: the measurement ran on a /tmp scratch that was deleted, so the number is
not reproducible from this branch. The command is documented.

Async inventory — unblocks poll (task E)

poll has been blocked because CreateVcn/UpdateVcn are synchronous. The
sweep looks for a response field tagged presentIn:"header" name:"opc-work-request-id", cross-checked against the SDK source.

2238 async candidates across 117 of 168 services — and 73 of them are in
core, the package the module already imports, including the VCN CIDR actions.
poll can be exercised without pulling in a new service. Not yet run against
real OCI; that needs tenancy access and was out of scope.

Egress inventory → R5 (task F)

Of 168 service packages, 162 build their host from a realm template ending
in {region}.{secondLevelDomain} (21 realms in common/regions.go), and all
162 are expressible today, with no relay change, as one glob per realm —
hostGlobMatch matches a leading * label against a fixed literal suffix,
which is exactly this shape (verified in CIC-Relay/cmd/relay/cic_flow_manifest_test.go,
TestHostGlobMatch). Four more hardcode a fixed host; also expressible.

R5 is therefore scoped narrowly to the one genuine gap — identitydomains,
whose endpoint resolves from tenant configuration at runtime and so cannot be
declared in a build-time-signed manifest at all. It does not block the
current core/network PoC.

Roadmap now follows the evidence (task G)

P3.1/P3.2/P3.3 move to verified, each citing the specific
manual-verification.md row. P3.4 becomes partialplan and execute
verified, poll blocked, with the concrete unblocking candidate named. P3.5
stays todo, but for a corrected reason: the hand-picked PoC list is a
deliberate scope subset of the mechanical registry, not a workaround for what
the extractor could reach.

Control group

vcn and subnet moved to module/schemas/core/ and are byte-identical
after the move; regression_test.go continues to freeze their field set,
policies, state size and operations.

Review

Orchestrator review, with independent re-verification of the schema move and the
relay glob contract, and an explicit list of what was not re-verified, is in
the cic-factory repo at jobs/oci-extract-full-sweep/review.md.

…ice gate, egress/async inventory

Runs the P2.5 extractor+resolver across all 168 pinned-SDK service
packages via a new build-time tool (tools/oci-extract/cmd/oci-sweep):
8048/8048 operations resolved (unchanged from P2.5), 685 resources
classified as mechanically handled (read model + resolvable create),
621 read-only byproducts, 62 non-resources, 106 polymorphic.

Splits the P2.4 integrity gate from one repo-wide extracted_schema_hash
into extracted_schema_hashes, a map keyed by SDK service directory
(module/schemas/<service>/), so a reviewer sees which service changed
instead of "something did". module/schemas/{vcn,subnet}.json move to
module/schemas/core/ to give the existing two resources a service home;
the semantic oci-extract -diff gate is unchanged (already per-resource).

Measures (not estimates) the WASM binary cost of embedding the full
685-resource surface with the current go:embed approach: 6.04 MiB,
37.8% of the relay's default 16 MiB MaxWASMModuleSize — no relay
change needed today, selective per-module embedding stays the right
default regardless.

Names core:LaunchInstance as the poll-unblocking candidate (real
opc-work-request-id, unlike the synchronous CreateVcn/UpdateVcn already
proven against real OCI) and inventories the SDK's three egress-host
shapes (162 realm-templated, 4 hardcoded to oracle.com, 2 — objectstorage,
identitydomains — genuinely dynamic). Raises relay-requirements.md R5 for
the identitydomains case (a runtime-resolved endpoint no static manifest
can express); the realm and hardcoded-host cases need no relay change.

Syncs roadmap.md Phase 3 status to docs/design/manual-verification.md's
actual coverage (P3.1-P3.3 verified, P3.4 partial/poll blocked, P3.5
reworded from "hand-picked because the extractor couldn't reach further"
to "a deliberate PoC subset of a now-mechanical 685-resource registry").

---
[signing-metadata]
key = cic-my-sign-key
signature = vault:v1:MEYCIQD77KGTCC87gle5V+F6JkydGCryNtoFF9iZoT+R/DSfQAIhAN6286Gv9C8cYYNEzwf2QQht7ykvyUzmfl8dLRYxCXMr
hash-algorithm = sha256
digest = 2UUKAfmDX5VjB3tXyZr9He9vdX/hbbv6rH9cUr83MVM=

[certificate]
-----BEGIN CERTIFICATE-----
MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw
QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV
BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy
MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb
MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe
i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU
yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS
WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0
lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW
-----END CERTIFICATE-----
@sinkog
sinkog merged commit ec4f79e into devel Aug 3, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant