Skip to content

feat(tfhost): host real providers — protocol 5+6, msgpack, nested blocks#121

Merged
astrojerms merged 1 commit into
mainfrom
feat/tfhost-real-providers
Jul 11, 2026
Merged

feat(tfhost): host real providers — protocol 5+6, msgpack, nested blocks#121
astrojerms merged 1 commit into
mainfrom
feat/tfhost-real-providers

Conversation

@astrojerms

Copy link
Copy Markdown
Owner

What

Closes the four gaps that blocked openctl's Terraform host (tfhost) from driving a real terraform-provider-* binary. It could previously only round-trip opaque state and send JSON config against an in-repo JSON fake.

Gap Now
msgpack state decoding decodeState reads a provider's DynamicValue (msgpack or JSON) via the public tfprotov6.DynamicValue.Unmarshal(type)
nested-block / typed config encoding encodeConfig builds a tftypes.Value conforming to the schema's implied type (nested blocks included) → NewDynamicValue (msgpack)
protocol v5 negotiated alongside v6 via go-plugin VersionedPlugins; SDKv2 (proto 5) providers driven through v5adapter converting field-identical v5⇄v6 messages
real-binary e2e drives the published hashicorp/time provider (proto 5) end-to-end, plus an upgraded msgpack/nested-block tf-fake (proto 6)

How it stays correct

  • Public codec, no deprecated API. The earlier slice sent JSON + stored opaque bytes specifically to avoid terraform-plugin-go's deprecated low-level helpers. The fix uses the library's intended value codec — tfprotov6.NewDynamicValue / DynamicValue.Unmarshal — the same functions provider authors use. Full msgpack fidelity, zero SA1019.
  • Adapter stays on tfplugin6 types. provider.go is unchanged in shape; a protocol-5 provider is bridged at the client boundary (client_v5.go). tfplugin5 gRPC stubs vendored in pkg/tfplugin5 (mirrors pkg/tfplugin6, generated from the upstream proto).
  • provider_state stays opaque — stored DynamicValue bytes round-trip untouched into Plan/Apply.

Verification

  • plugins/tf-fake upgraded to a nested-block, msgpack-emitting framework provider — existing tests now exercise the real wire encoding; a new TestProviderAdapterNestedBlock proves a nested network block round-trips through the adapter.
  • TestRealProviderTimeStatic (gated on TFHOST_E2E_PROVIDER_TIME) drives the real hashicorp/time v0.12.1 binary: create/read/delete time_static, decoding computed msgpack outputs (year=2020, unix, id). Verified locally:
    TFHOST_E2E_PROVIDER_TIME=/path/to/terraform-provider-time go test ./internal/controller/providers/tfhost/ -run TestRealProviderTimeStatic
    
  • Tests assert the negotiated protocol (fake→6, time→5).

Local CI

gofmt / go vet / staticcheck / golangci-lint / modernize (generated-filtered) / go test ./... / go test -race (tfhost) all green. No new external deps (uses already-present terraform-plugin-go).

Follow-ups

UpgradeResourceState wiring; making the real-provider e2e run in CI (currently gated/local); then wire a real provider (e.g. the Terraform Cloudflare provider) through config.

Closes the four gaps that blocked openctl's Terraform host from driving a
real terraform-provider-* binary. It could previously only round-trip
opaque state and send JSON config against an in-repo JSON fake.

- msgpack state decoding + nested-block/typed config encoding: encodeConfig
  builds a tftypes.Value from an openctl spec conforming to the schema's
  implied type (nested blocks included) and encodes it via the PUBLIC
  tfprotov6 codec (NewDynamicValue -> msgpack); decodeState reads a
  provider's DynamicValue (msgpack or JSON) back into a map via
  DynamicValue.Unmarshal. No deprecated tftypes helpers. New values.go;
  splitState now classifies decoded fields into spec vs computed-only
  status; isNullDynamicValue also recognizes msgpack null (0xc0).

- protocol 5 + 6 negotiation: Launch offers both via go-plugin
  VersionedPlugins; SDKv2 providers (proto 5) and framework providers
  (proto 6) both connect. The adapter stays on tfplugin6 types — a proto-5
  provider is driven through v5adapter (client_v5.go), converting the
  field-identical v5<->v6 messages at the boundary. tfplugin5 gRPC stubs
  vendored in pkg/tfplugin5 (mirrors pkg/tfplugin6).

- real-binary e2e: plugins/tf-fake upgraded to a nested-block,
  msgpack-emitting framework provider (proto 6) — so existing tests now
  exercise the real wire encoding, plus a new nested-block round-trip test
  through the adapter. A gated TestRealProviderTimeStatic drives the
  PUBLISHED hashicorp/time provider (proto 5) end-to-end
  (create/read/delete time_static, decoding its computed msgpack outputs),
  run with TFHOST_E2E_PROVIDER_TIME=<binary>. Tests assert the negotiated
  protocol (fake=6, time=5).

Docs: plugin-architecture.md updated (the earlier "avoid msgpack" decision
is superseded by using the public codec); ROADMAP entry added.
@astrojerms astrojerms merged commit c13da11 into main Jul 11, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant