Skip to content

Release: service-agnostic extraction, full-SDK sweep, and the measurements it unblocks - #16

Merged
sinkog merged 6 commits into
mainfrom
devel
Aug 3, 2026
Merged

Release: service-agnostic extraction, full-SDK sweep, and the measurements it unblocks#16
sinkog merged 6 commits into
mainfrom
devel

Conversation

@sinkog

@sinkog sinkog commented Aug 3, 2026

Copy link
Copy Markdown
Member

Six commits, all CI-green. Tooling and contract work — the two resource types
the module ships (vcn, subnet) are byte-identical to what is on main.

What changed

Commit
4c79605 keep the read-only harness read-only; forward OCI_REALM_DOMAIN
ec4f79e 33a0c62 full-SDK sweep: classification, per-service gate, egress/async inventory, roadmap alignment (#15)
0adf578 5c1e035 resolve operations and models from the HTTP surface, not Go names (#14)
6331e76 record the cic-primitives alignment direction

The extractor was service-specific — it no longer is

P2.2-P2.4 were proven on vcn and subnet, both in core/network. Measured
across all 171 services of the pinned SDK, four assumptions failed: operations
without a request object, create/delete verbs that are not Create/Delete
(570 of 1217 resources have no Create<R>Details), resources addressed by
something other than one id path parameter (323 of 1481 Get operations),
and models declared as interfaces (1014).

The worst of these exited 0 with a valid draft-07 schema that had no required
list and neither a create nor a delete operation — a schema that validates and
is wrong. Lifecycle is now derived structurally from the HTTP surface, and
unresolved surfaces exit non-zero instead of emitting silence.

Result: 8048/8048 operations resolved, 0 missing method/path, across 319
client files.

Measurements that unblock pending decisions

  • Embedding is not size-limited. The full 685-resource class-A surface
    compiles to a 6.04 MiB WASM — 37.8% of the relay's 16 MiB default. Embedding
    stays selective by design; this only removes size as a reason to hesitate.
  • poll can finally be exercised. 2238 async candidates across 117 of 168
    services, and 73 of them are in core — the package the module already
    imports, including the VCN CIDR actions. No new service needed.
  • Egress is expressible today. 162 of 168 services fit the existing
    hostGlobMatch contract with one glob per realm, no relay change. R5 is
    scoped to the single genuine gap (identitydomains, tenant-resolved endpoint)
    and does not block the current core/network PoC.

Verification

  • CI green on every commit; headSha matched against each tested commit
  • vcn and subnet moved to module/schemas/core/ and are byte-identical
    after the move; regression_test.go freezes their field set, policies, state
    size and operations
  • Re-verified against a live OCI tenancy after the rebuild (oc19 realm,
    read-only): Observe, Validate and Plan all pass, confirming the
    go:embed path change did not break schema loading on the real API path

Known limits, deliberately

  • Polymorphic create models (interfaces) are reported and exit 5, not
    expanded — discriminator semantics is a modelling decision, not extraction
  • No new resource types ship; module/schemas/ is still vcn + subnet
  • No cic-primitives / YANG binding — that layer carries behavior, not shape
  • The 6.04 MiB figure was measured on a scratch copy that was deleted, so it is
    not reproducible from this branch. The command is documented

Orchestrator reviews, including what was not re-verified, are in the
cic-factory repo under jobs/oci-extract-generalize/review.md and
jobs/oci-extract-full-sweep/review.md.

Gábor Z Sinkó and others added 6 commits August 3, 2026 09:54
…onsistency

No code-level connection exists between this module and cic-primitives (the
repo GitHub still hosts as "cic-yang" — its own tag history shows the
yang -> primitives rebrand). Its own roadmap marks the domain-specialization
compiler binding (Phase 6) as unimplemented concept, not even validated on
its one in-repo example (KubernetesPod).

Recommends staying API-first (tools/oci-extract against the real OCI SDK,
now real-OCI-verified) rather than hand-modeling resources from an unproven
aggregate-primitive schema; if/when cic-primitives Phase 6 lands, project
the already-correct extracted contract onto ManagedEntity mechanically
instead of hand-authoring a second, divergence-prone source of truth.

Also records: cic-yang/README.md claims CIC-Relay runs primitives-derived
schemas — zero references to cic-primitives/managed-entity exist anywhere in
CIC-Relay's actual source. And records how the first read of cic-yang's own
"not a YANG descriptor" disclaimer led to underselling it — every aggregate
primitive actually carries an explicit semantic_mapping.yang block.

---
[signing-metadata]
key = cic-my-sign-key
signature = vault:v1:MEYCIQDzaZRj3m5HLv76pZc576LN2OnXI6ToS2YioDWH9Kbe7wIhAKy0e5WJHv1WTOrch3PT8nUGfReaoiD7+BGaQBcPnK+b
hash-algorithm = sha256
digest = Qn7uBFP6OKM47c4UeNWhZj17Us+fqSMhQe/CiM6R8Jg=

[certificate]
-----BEGIN CERTIFICATE-----
MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw
QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV
BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy
MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb
MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe
i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU
yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS
WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0
lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW
-----END CERTIFICATE-----
…e, not Go names

P2.2-P2.4 were proven on vcn and subnet. Both live in core/network, so they
established that the pipeline is not resource-specific; they could not establish
that it is not service-specific. Measured against all 171 services of the pinned
SDK, it was. Four assumptions held for core/network and not for OCI:

1. An operation has a request object. IdentityClient.ListRegions(ctx) has none,
   and therefore builds its wire call with common.MakeDefaultHTTPRequest rather
   than request.HTTPRequest. It was dropped silently — 8047 of 8048 operations
   resolved and nothing named the missing one.
2. The create verb is "Create" and the delete verb is "Delete". core.Instance
   uses LaunchInstance/TerminateInstance; 570 of 1217 resources have no
   Create<R>Details. The name lookup found nothing and emitted a config schema
   with no required list and a create surface built from Update ∩ Read — a
   schema that validates and is wrong.
3. A resource is addressed by one path parameter and it is its id.
   objectstorage.Bucket is /n/{namespaceName}/b/{bucketName}: two, neither an
   id. 323 of 1481 Get operations (21.8%), across 65 of 158 services, have a
   count other than one, and nothing recorded them.
4. Models are structs. OCI declares 1014 models as interfaces (470 *Details or
   *Base); those parsed as absent, giving the same silent empty create surface.

Resolution now derives the lifecycle structurally: read = GET readPath, create =
POST on readPath minus its trailing /{param}, update = PUT readPath else POST
readPath, delete = DELETE readPath, actions = POST under readPath/. Body models
come from the request struct's own contributesTo:"body" tag. One convention
survives as a fallback (Get<Resource>, only when no response models are given)
and the resolution records which route it took.

Silence became signal: -audit counts candidates separately from resolved and
exits 4; -schema/-policy report unresolved surfaces and exit 5, so oci.generate
cannot commit a schema whose create surface was never derived.

Verified on the real pinned SDK v65.121.0: 8048/8048 operations resolved, 0
missing method/path across 319 client files. Per service — core/network 271/271,
core/compute 129/129, identity 145/145, objectstorage 56/56, database 456/456,
dns 54/54.

vcn and subnet are unchanged: oci-extract -diff reports neither a breaking nor a
compatible change to their config schemas, and regression_test.go freezes their
field set, policies, state size and operations against a written-out expectation.
The bundles differ only by the additive path_params, hence the re-pinned
extracted_schema_hash and the rebuilt guest.

CI also now triggers on feature/** branches, which it did not before.

---
[signing-metadata]
key = cic-my-sign-key
signature = vault:v1:MEQCIGLo+rBakUqxWsKcaqtYS9eeXqgewbOPnhNu4xdh91MLAiBcKkd4R8M6ixoX1WgIrcxw7CFiD6khsarF3KwmcAV83w==
hash-algorithm = sha256
digest = M2tsElmF0suuGzMLzT8tx+dVYyCN60pUEReVVqcnwpM=

[certificate]
-----BEGIN CERTIFICATE-----
MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw
QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV
BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy
MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb
MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe
i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU
yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS
WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0
lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW
-----END CERTIFICATE-----
…ralize

oci-extract: resolve operations and models from the HTTP surface, not Go names
…ice gate, egress/async inventory

Runs the P2.5 extractor+resolver across all 168 pinned-SDK service
packages via a new build-time tool (tools/oci-extract/cmd/oci-sweep):
8048/8048 operations resolved (unchanged from P2.5), 685 resources
classified as mechanically handled (read model + resolvable create),
621 read-only byproducts, 62 non-resources, 106 polymorphic.

Splits the P2.4 integrity gate from one repo-wide extracted_schema_hash
into extracted_schema_hashes, a map keyed by SDK service directory
(module/schemas/<service>/), so a reviewer sees which service changed
instead of "something did". module/schemas/{vcn,subnet}.json move to
module/schemas/core/ to give the existing two resources a service home;
the semantic oci-extract -diff gate is unchanged (already per-resource).

Measures (not estimates) the WASM binary cost of embedding the full
685-resource surface with the current go:embed approach: 6.04 MiB,
37.8% of the relay's default 16 MiB MaxWASMModuleSize — no relay
change needed today, selective per-module embedding stays the right
default regardless.

Names core:LaunchInstance as the poll-unblocking candidate (real
opc-work-request-id, unlike the synchronous CreateVcn/UpdateVcn already
proven against real OCI) and inventories the SDK's three egress-host
shapes (162 realm-templated, 4 hardcoded to oracle.com, 2 — objectstorage,
identitydomains — genuinely dynamic). Raises relay-requirements.md R5 for
the identitydomains case (a runtime-resolved endpoint no static manifest
can express); the realm and hardcoded-host cases need no relay change.

Syncs roadmap.md Phase 3 status to docs/design/manual-verification.md's
actual coverage (P3.1-P3.3 verified, P3.4 partial/poll blocked, P3.5
reworded from "hand-picked because the extractor couldn't reach further"
to "a deliberate PoC subset of a now-mechanical 685-resource registry").

---
[signing-metadata]
key = cic-my-sign-key
signature = vault:v1:MEYCIQD77KGTCC87gle5V+F6JkydGCryNtoFF9iZoT+R/DSfQAIhAN6286Gv9C8cYYNEzwf2QQht7ykvyUzmfl8dLRYxCXMr
hash-algorithm = sha256
digest = 2UUKAfmDX5VjB3tXyZr9He9vdX/hbbv6rH9cUr83MVM=

[certificate]
-----BEGIN CERTIFICATE-----
MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw
QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV
BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy
MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb
MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe
i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU
yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS
WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0
lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW
-----END CERTIFICATE-----
…-sweep

oci-extract: full-SDK sweep — classification, per-service gate, egress/async inventory
Two defects found while running the read-only harness against a live oc19
tenancy to verify the rebuilt guest after the schemas/core/ move.

The Makefile target never forwarded OCI_REALM_DOMAIN. Dual-realm support landed
in 899ffe0, but the wrapper still passed only the pre-dual-realm variable set,
so any non-oc1 tenancy would resolve to oraclecloud.com — a host that does not
exist for that realm.

manual-verification.md claimed that -run TestManualRealOCI 'does not match
TestManualRealOCIExecute'. Go's -run is an unanchored regexp, so it does match,
and running it demonstrated that: the mutating test executed and failed only
because its OCI_EXEC_* guard was unset. The env guard is the real protection,
not the pattern. The target now anchors the pattern to the three read-only
tests, and the document says what actually keeps a bare run safe.

Verified: with the anchored pattern the package passes with no Execute run;
Observe, Validate and Plan all pass against the live tenancy.

---
[signing-metadata]
key = cic-my-sign-key
signature = vault:v1:MEQCIBp5dSQ9cvS9zxAQgBEn7lSGMuQgLo7xCW1E1XzS+AzGAiAzxLRyI8dQVx9AJKj5RBdtrBxrZWn1fo2TK++USGDO+A==
hash-algorithm = sha256
digest = RsU7HgerjMd8Tj/PV94/AoZ98U6ngzeUsO6Sl2UD7Hk=

[certificate]
-----BEGIN CERTIFICATE-----
MIICBjCCAaygAwIBAgIUSnRMR6RPnEbg296XWPOqq/u5PCwwCgYIKoZIzj0EAwIw
QzELMAkGA1UEBhMCSFUxGTAXBgNVBAoMEENlbnRyYWxJbmZyYUNvcmUxGTAXBgNV
BAMMEENJQyBEZXZlbG9wZXIgQ0EwHhcNMjYwMzIwMTMyMjU5WhcNMjYxMjMxMTMy
MjU5WjBFMQswCQYDVQQGEwJIVTEZMBcGA1UECgwQQ2VudHJhbEluZnJhQ29yZTEb
MBkGA1UEAwwSR2Fib3IgWm9sdGFuIFNpbmtvMFkwEwYHKoZIzj0CAQYIKoZIzj0D
AQcDQgAEIG2CVmTfmLB9pLLclj7YmP2eedAjklpy4LGrU2ijoiy6Xqpuybv7OgJe
i+ez31s65NEV8+X/ByeX1cstR988z6N8MHowCQYDVR0TBAIwADAdBgNVHQ4EFgQU
yZN6AIX/TNnIJ9GwAa/NRN3ujHAwHwYDVR0jBBgwFoAUXn6CHYzPUqU4JVP8g+OS
WeDYjhcwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMCBggrBgEF
BQcDBDAKBggqhkjOPQQDAgNIADBFAiEA+bFzXRoJ4PCQbhAAtpkcMjt0vNj5rEW0
lOMBGDNyaWkCIB1vmM7PcZzv/c9bIrxF5kqv6QXomouhByUfeNUTbpKW
-----END CERTIFICATE-----
@sinkog
sinkog merged commit f002698 into main Aug 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant