oci-extract: full-SDK sweep — classification, per-service gate, egress/async inventory - #15
Merged
Merged
Conversation
…ice gate, egress/async inventory
Runs the P2.5 extractor+resolver across all 168 pinned-SDK service
packages via a new build-time tool (tools/oci-extract/cmd/oci-sweep):
8048/8048 operations resolved (unchanged from P2.5), 685 resources
classified as mechanically handled (read model + resolvable create),
621 read-only byproducts, 62 non-resources, 106 polymorphic.
Splits the P2.4 integrity gate from one repo-wide extracted_schema_hash
into extracted_schema_hashes, a map keyed by SDK service directory
(module/schemas/<service>/), so a reviewer sees which service changed
instead of "something did". module/schemas/{vcn,subnet}.json move to
module/schemas/core/ to give the existing two resources a service home;
the semantic oci-extract -diff gate is unchanged (already per-resource).
Measures (not estimates) the WASM binary cost of embedding the full
685-resource surface with the current go:embed approach: 6.04 MiB,
37.8% of the relay's default 16 MiB MaxWASMModuleSize — no relay
change needed today, selective per-module embedding stays the right
default regardless.
Names core:LaunchInstance as the poll-unblocking candidate (real
opc-work-request-id, unlike the synchronous CreateVcn/UpdateVcn already
proven against real OCI) and inventories the SDK's three egress-host
shapes (162 realm-templated, 4 hardcoded to oracle.com, 2 — objectstorage,
identitydomains — genuinely dynamic). Raises relay-requirements.md R5 for
the identitydomains case (a runtime-resolved endpoint no static manifest
can express); the realm and hardcoded-host cases need no relay change.
Syncs roadmap.md Phase 3 status to docs/design/manual-verification.md's
actual coverage (P3.1-P3.3 verified, P3.4 partial/poll blocked, P3.5
reworded from "hand-picked because the extractor couldn't reach further"
to "a deliberate PoC subset of a now-mechanical 685-resource registry").
---
[signing-metadata]
key = cic-my-sign-key
signature = vault:v1:MEYCIQD77KGTCC87gle5V+F6JkydGCryNtoFF9iZoT+R/DSfQAIhAN6286Gv9C8cYYNEzwf2QQht7ykvyUzmfl8dLRYxCXMr
hash-algorithm = sha256
digest = 2UUKAfmDX5VjB3tXyZr9He9vdX/hbbv6rH9cUr83MVM=
[certificate]
-----BEGIN CERTIFICATE-----
MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw
QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV
BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy
MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb
MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe
i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU
yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS
WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0
lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW
-----END CERTIFICATE-----
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #14, which made the extractor service-agnostic. This runs it across
the whole pinned SDK and settles the three questions that only a full-surface
measurement could answer.
Build-time registry (task B)
A new
tools/oci-extract/cmd/oci-sweeploops the resolver over every servicedirectory of the pinned SDK — no hand-curated file lists — and classifies each
resource by its create surface:
WorkRequest,Trace,Log): read-only byproductsexit 5; discriminator semantics is a modelling decision, deliberately out of scopePer-service integrity gate (task C)
The single
extracted_schema_hashbecomes anextracted_schema_hashesmap, soa diff stays reviewable at full-SDK scale. Two new tests in
tests/test_oci_sdk_lock.pycover that every committed service is present andthat the hashes match. The semantic
oci-extract -diffstill fails non-zero ona real break: deleting
cidrBlockfrommodule/schemas/core/vcn.jsonyields{"breaking":[{"field":"cidrBlock","kind":"removed"}]}and exit 3.Embedding measured, not estimated (task D)
Built, not guessed: the 685 class-A schema bundles were embedded into a
throwaway copy of
module/, compiled with TinyGo, and compared against afreshly built 2-resource baseline in the same container.
module.wasm, full surface (685 resources)module.wasm, baseline (vcn+subnet)MaxWASMModuleSizeThe whole class-A surface would fit with 62% headroom. "Fits" is not a reason
to embed services a module never imports — embedding stays selective by
design; this measurement removes size as the reason to hesitate, nothing more.
Note: the measurement ran on a
/tmpscratch that was deleted, so the number isnot reproducible from this branch. The command is documented.
Async inventory — unblocks
poll(task E)pollhas beenblockedbecauseCreateVcn/UpdateVcnare synchronous. Thesweep looks for a response field tagged
presentIn:"header" name:"opc-work-request-id", cross-checked against the SDK source.2238 async candidates across 117 of 168 services — and 73 of them are in
core, the package the module already imports, including the VCN CIDR actions.pollcan be exercised without pulling in a new service. Not yet run againstreal OCI; that needs tenancy access and was out of scope.
Egress inventory → R5 (task F)
Of 168 service packages, 162 build their host from a realm template ending
in
{region}.{secondLevelDomain}(21 realms incommon/regions.go), and all162 are expressible today, with no relay change, as one glob per realm —
hostGlobMatchmatches a leading*label against a fixed literal suffix,which is exactly this shape (verified in
CIC-Relay/cmd/relay/cic_flow_manifest_test.go,TestHostGlobMatch). Four more hardcode a fixed host; also expressible.R5is therefore scoped narrowly to the one genuine gap —identitydomains,whose endpoint resolves from tenant configuration at runtime and so cannot be
declared in a build-time-signed manifest at all. It does not block the
current core/network PoC.
Roadmap now follows the evidence (task G)
P3.1/P3.2/P3.3 move to verified, each citing the specific
manual-verification.mdrow. P3.4 becomes partial —planandexecuteverified,
pollblocked, with the concrete unblocking candidate named. P3.5stays todo, but for a corrected reason: the hand-picked PoC list is a
deliberate scope subset of the mechanical registry, not a workaround for what
the extractor could reach.
Control group
vcnandsubnetmoved tomodule/schemas/core/and are byte-identicalafter the move;
regression_test.gocontinues to freeze their field set,policies, state size and operations.
Review
Orchestrator review, with independent re-verification of the schema move and the
relay glob contract, and an explicit list of what was not re-verified, is in
the cic-factory repo at
jobs/oci-extract-full-sweep/review.md.