Skip to content

Kms preflight sandbox interceptor - #2381

Closed
p0lyn0mial wants to merge 4 commits into
openshift:masterfrom
p0lyn0mial:kms-preflight-sandbox-interceptor
Closed

Kms preflight sandbox interceptor#2381
p0lyn0mial wants to merge 4 commits into
openshift:masterfrom
p0lyn0mial:kms-preflight-sandbox-interceptor

Conversation

@p0lyn0mial

@p0lyn0mial p0lyn0mial commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

xref: #2377

Summary by CodeRabbit

  • New Features
    • KMS preflight checks now generate the required encryption configuration before deploying preflight workloads.
    • Encryption configuration is adapted to use the preflight KMS endpoint, supporting key creation and provider changes.
  • Bug Fixes
    • Improved handling when encryption configuration is still converging, allowing reconciliation to retry safely.
    • Enhanced detection of encryption keys and configuration resources during preflight processing.

bertinatto and others added 4 commits July 25, 2026 08:10
The preflight sandbox previously used testing/fstest.MapFS and
manifestclient.NewTestingHTTPClient to simulate dry-run controller
execution, importing a test-only package in production code.

Replace it with a thin interceptingSecretsGetter that wraps the real
corev1client.SecretsGetter. Reads fall through to the live cluster;
writes matching a caller-supplied predicate are captured in an in-memory
map and never reach the real API. A shared interceptor instance between
the KeyController and StateController dry-runs means the intercepted key
secret is visible to the StateController's List call without any
pre-seeding or YAML serialization.

staticEncryptionDeployer is removed from production code; both dry-run
controllers now use the real encryptionDeployer directly, which is
already known to be converged at the call site.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@p0lyn0mial
p0lyn0mial requested a review from ardaguclu July 27, 2026 09:46
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 27, 2026
@openshift-ci

openshift-ci Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci

openshift-ci Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: p0lyn0mial

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 27, 2026
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Walkthrough

KMS preflight now computes its encryption-config Secret through dry-run key and state controller execution, intercepts Secret writes, rewrites KMS endpoints for the preflight socket, and supplies the result during deployment. Key and state controllers are also exported with factory-compatible interfaces.

Changes

KMS preflight encryption flow

Layer / File(s) Summary
Exported encryption controller interfaces
pkg/operator/encryption/controllers/key_controller.go, pkg/operator/encryption/controllers/state_controller.go, pkg/operator/encryption/controllers/*_test.go
KeyController and StateController expose concrete factory-compatible Run, Name, and Sync methods.
Intercepted Secret client
pkg/operator/encryption/controllers/kms_preflight_interceptor.go
Selected Secret creates and updates are stored in memory and returned through reads and filtered lists.
Dry-run encryption configuration generation
pkg/operator/encryption/controllers/kms_preflight_sandbox.go, pkg/operator/encryption/controllers/kms_preflight_sandbox_test.go
Dry-run controllers generate an encryption-config Secret without API mutations, and KMS endpoints are rewritten for the preflight socket.
Preflight deployment integration
pkg/operator/encryption/controllers/kms_preflight_controller.go, pkg/operator/encryption/controllers/kms_preflight_controller_test.go
The preflight controller computes configuration when the pod is absent, requeues until convergence, watches configuration Secrets, and passes the generated Secret to the deployer.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant KMSPreflightController
  participant KeyController
  participant StateController
  participant SecretClient
  KMSPreflightController->>KeyController: Run dry-run Sync
  KeyController->>SecretClient: Intercept key Secret write
  KMSPreflightController->>StateController: Run dry-run Sync
  StateController->>SecretClient: Intercept encryption-config write
  KMSPreflightController->>KMSPreflightController: Rewrite KMS endpoint
  KMSPreflightController->>KMSPreflightController: Deploy preflight workload with Secret
Loading

Possibly related PRs

  • openshift/library-go#2377: Covers overlapping KMS preflight dry-run encryption-config computation and Secret interception work.

Suggested reviewers: ardaguclu

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Test Structure And Quality ⚠️ Warning kms_preflight_sandbox_test.go:87 still logs the full action with %#v, which can print Secret data; that’s an unsafe assertion message. Replace %#v with a redacted message (verb/resource/namespace only), as suggested in review, to avoid leaking generated secret material.
✅ Passed checks (14 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: adding a KMS preflight sandbox interceptor.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed No new Ginkgo titles or subtest names use dynamic data; added test names are static literals.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the changed tests are standard testing.T unit tests, so the MicroShift check is inapplicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The added tests are plain Go unit tests, not Ginkgo e2e tests, and I found no SNO-unsafe multi-node/HA assumptions or skip gaps.
Topology-Aware Scheduling Compatibility ✅ Passed Touched files only refactor encryption controllers and add in-memory secret interception; no nodeSelector/affinity/topologySpread/replica logic was added.
Ote Binary Stdout Contract ✅ Passed No touched file adds main/init/TestMain/BeforeSuite/RunSpecs stdout writes; logging stays inside ordinary controller/test methods.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the changed tests are plain testing.T unit tests and show no IPv4-only or external connectivity assumptions.
No-Weak-Crypto ✅ Passed No MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret/token compares were added in the changed files.
Container-Privileges ✅ Passed PASS: PR diff only touches Go controllers/tests; no manifests or privilege-related settings (privileged, hostPID/Network/IPC, allowPrivilegeEscalation, SYS_ADMIN) appear.
No-Sensitive-Data-In-Logs ✅ Passed The PR diff adds no new logs exposing secrets; the only %#v secret-dump line is pre-existing and not part of this change.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
pkg/operator/encryption/controllers/kms_preflight_controller_test.go (1)

846-860: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cover dry-run convergence and error paths.

The scenario table cannot configure encryptionDeployer, so it only verifies successful deployment. Add cases where dry-run computation requires convergence (assert no Deploy) and where it fails (assert the controller reports degradation).

Also applies to: 877-885

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/operator/encryption/controllers/kms_preflight_controller_test.go` around
lines 846 - 860, Extend the scenario table and test setup around
NewKMSPreflightController to allow configuring encryptionDeployer behavior. Add
a dry-run convergence case that verifies Deploy is not called, and a dry-run
failure case that verifies the controller reports degradation, while preserving
the existing successful deployment coverage.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/operator/encryption/controllers/kms_preflight_sandbox_test.go`:
- Around line 85-87: Update the failure message in the fakeKubeClient action
loop to avoid formatting the entire action object. Log only the action verb,
resource, and namespace, using the corresponding action accessors, so Secret
data and generated credentials cannot appear in CI output.

---

Nitpick comments:
In `@pkg/operator/encryption/controllers/kms_preflight_controller_test.go`:
- Around line 846-860: Extend the scenario table and test setup around
NewKMSPreflightController to allow configuring encryptionDeployer behavior. Add
a dry-run convergence case that verifies Deploy is not called, and a dry-run
failure case that verifies the controller reports degradation, while preserving
the existing successful deployment coverage.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 98825896-905d-414b-9a80-8a432cb70354

📥 Commits

Reviewing files that changed from the base of the PR and between e4053a9 and ea282be.

📒 Files selected for processing (8)
  • pkg/operator/encryption/controllers/key_controller.go
  • pkg/operator/encryption/controllers/key_controller_test.go
  • pkg/operator/encryption/controllers/kms_preflight_controller.go
  • pkg/operator/encryption/controllers/kms_preflight_controller_test.go
  • pkg/operator/encryption/controllers/kms_preflight_interceptor.go
  • pkg/operator/encryption/controllers/kms_preflight_sandbox.go
  • pkg/operator/encryption/controllers/kms_preflight_sandbox_test.go
  • pkg/operator/encryption/controllers/state_controller.go

Comment on lines +85 to +87
for _, action := range fakeKubeClient.Actions() {
if action.GetVerb() == "create" || action.GetVerb() == "update" {
t.Fatalf("live client must not be mutated, got action %#v", action)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Do not serialize the full Secret action into CI logs.

%#v can recursively print the action’s Secret object and its Data, including generated key material or the Vault secret-id. Log only verb, resource, and namespace.

Proposed fix
-			t.Fatalf("live client must not be mutated, got action %#v", action)
+			t.Fatalf("live client must not be mutated, got %s on %s in namespace %q",
+				action.GetVerb(), action.GetResource().Resource, action.GetNamespace())

As per coding guidelines, “Flag test logging that may expose passwords, tokens, API keys, PII …”.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
for _, action := range fakeKubeClient.Actions() {
if action.GetVerb() == "create" || action.GetVerb() == "update" {
t.Fatalf("live client must not be mutated, got action %#v", action)
for _, action := range fakeKubeClient.Actions() {
if action.GetVerb() == "create" || action.GetVerb() == "update" {
t.Fatalf("live client must not be mutated, got %s on %s in namespace %q",
action.GetVerb(), action.GetResource().Resource, action.GetNamespace())
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/operator/encryption/controllers/kms_preflight_sandbox_test.go` around
lines 85 - 87, Update the failure message in the fakeKubeClient action loop to
avoid formatting the entire action object. Log only the action verb, resource,
and namespace, using the corresponding action accessors, so Secret data and
generated credentials cannot appear in CI output.

Source: Coding guidelines

@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 28, 2026
@openshift-ci

openshift-ci Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants