Skip to content

CORENET-7356: Add TLS profile support for network-check-source metrics#3050

Closed
tpantelis wants to merge 1 commit into
openshift:masterfrom
tpantelis:network_check_source_tls
Closed

CORENET-7356: Add TLS profile support for network-check-source metrics#3050
tpantelis wants to merge 1 commit into
openshift:masterfrom
tpantelis:network_check_source_tls

Conversation

@tpantelis

Copy link
Copy Markdown
Contributor

Use the OCP API server TLS profile to configure TLS options for the metrics server and exit the process on TLS profile changes.

library-go's controllercmd framework does not have support for TLS profiles and does not easily allow TLS options to be customized. Therefore the code was migrated to use the controller-runtime Manager API which allows direct customization.

The code respects the TLS profile adherence setting and uses the library-go strong defaults when adherence is optional so as not to fall back to weaker Go default ciphers.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 6929077c-6328-40f5-89d5-314a8b3b10ba

📥 Commits

Reviewing files that changed from the base of the PR and between fcdb6b9 and f05b3c5.

📒 Files selected for processing (3)
  • bindata/network-diagnostics/network-check-source.yaml
  • pkg/cmd/checkendpoints/cmd.go
  • pkg/cmd/checkendpoints/controller/metrics.go
💤 Files with no reviewable changes (1)
  • bindata/network-diagnostics/network-check-source.yaml
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/cmd/checkendpoints/cmd.go

Summary by CodeRabbit

  • New Features

    • Added a metrics listen-address option to the connectivity check command.
    • Metrics can now be served over TLS using the cluster’s TLS configuration.
    • Improved controller startup, cancellation handling, and lifecycle orchestration.
  • Bug Fixes

    • Metrics are now registered via the controller-runtime metrics system for better compatibility.
    • The controller waits for the required custom resource definition before starting.
  • Chores

    • Simplified the network diagnostics container arguments by removing the redundant namespace argument.

Walkthrough

The check-endpoints CLI now uses controller-runtime for command setup, TLS-protected metrics serving, cancellation, client and informer construction, controller startup, and TLS profile change handling. Its metrics are registered with the controller-runtime registry.

Changes

Check Endpoints Runtime

Layer / File(s) Summary
Command and TLS runtime
pkg/cmd/checkendpoints/cmd.go
The standalone Cobra command adds logging, signal cancellation, pod environment handling, the --listen flag, TLS profile selection, TLS metrics serving, and cancellation on TLS profile or adherence changes.
Controller and informer startup
pkg/cmd/checkendpoints/cmd.go
Clients and informers are constructed, the pod’s node is resolved for event recording, CRD availability is awaited, and controllers run until cancellation.
Metrics registry and deployment wiring
pkg/cmd/checkendpoints/controller/metrics.go, bindata/network-diagnostics/network-check-source.yaml
Connectivity-check metric vectors use Prometheus types and the controller-runtime registry; the deployment removes the namespace command-line argument while retaining the namespace environment variable.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CobraCommand
  participant CheckEndpointsRuntime
  participant TLSProfileAPI
  participant MetricsServer
  participant Controllers
  CobraCommand->>CheckEndpointsRuntime: execute with listen address and pod identity
  CheckEndpointsRuntime->>TLSProfileAPI: fetch cluster TLS profile
  TLSProfileAPI-->>CheckEndpointsRuntime: return profile
  CheckEndpointsRuntime->>MetricsServer: start TLS-protected metrics server
  CheckEndpointsRuntime->>Controllers: build clients and start controllers
  TLSProfileAPI-->>CheckEndpointsRuntime: report profile or adherence change
  CheckEndpointsRuntime->>MetricsServer: cancel shared context
  CheckEndpointsRuntime->>Controllers: stop controllers
Loading

Suggested reviewers: danwinship, taanyas, jcaamano


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (5 errors, 1 warning)

Check name Status Explanation Resolution
Pr Quality ❌ Error PR text has no explicit Why/What/Testing sections; it’s just 3 paragraphs, so the non-trivial change doesn’t meet the required description criteria. Diff size is fine (~310 lines). Rewrite the description with clear Why, What, and CI Testing sections (including lanes/platforms), plus the bug/issue link and any user-impact/rollback notes.
Unit Tests For Go Changes ❌ Error pkg/cmd and bindata files changed, but no *_test.go files were added or modified in the PR. Add or update tests alongside the Go/bindata changes, or document why tests are unnecessary in the PR description under "How to verify it".
E2e Tests For Feature Changes ❌ Error cmd.go adds new metrics-server TLS/profile behavior; no test/e2e files changed and the PR description lacks a Testing/How to verify it section. Add e2e coverage for the new behavior and a Testing/How to verify it section listing CI lanes/jobs, covered platforms, and results; if infeasible, justify that there.
Docs For Feature And Behavior Changes ❌ Error PR changes check-endpoints behavior/TLS/flags, but no docs/ files were added or modified. Add docs under docs/ for the new TLS-profile metrics server behavior, listen flag, env vars, and shutdown-on-profile-change; or justify why docs aren't needed in the PR.
Ai-Generated Code Smell ❌ Error cmd.go adds tautological comments that just restate their functions (runControllers, fetchClusterTLSProfile, buildTLSOptions), a clear comment-slop smell. Remove the restating comments or rewrite them to explain non-obvious behavior only; keep comments only where they add real context.
Go And Test Code Quality ⚠️ Warning runControllers returns bare err twice (node lookup retry and timeToStart.Ready), so meaningful startup errors lose context. Wrap both returns with fmt.Errorf to add operation context, e.g. pod/node lookup and CRD readiness, using %w.
✅ Passed checks (18 passed)
Check name Status Explanation
Title check ✅ Passed The title is imperative, concise, and accurately summarizes the TLS profile metrics change.
Description check ✅ Passed The description is clearly related to the TLS profile and controller-runtime migration changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Commit Message Quality ✅ Passed Single non-merge commit has a concise descriptive subject and a body that explains why the TLS-profile migration was needed.
Rbac Least Privilege ✅ Passed PASS: The only YAML diff removes a container arg; the Role stanza is unchanged and no RBAC rule edits appear.
Stale Project Docs And Config ✅ Passed No docs/config files reference the removed namespace arg or controllercmd wiring; the only docs mention of network-check-source remains accurate.
Stable And Deterministic Test Names ✅ Passed No changed files add Ginkgo specs; the touched package only has standard Test... functions, with no dynamic titles introduced.
Test Structure And Quality ✅ Passed No Ginkgo/test files were changed in this PR; only cmd, metrics, and bindata were modified, so the test-structure check is not applicable.
Microshift Test Compatibility ✅ Passed No new Ginkgo/e2e tests were added; the PR only changes cmd, metrics, and a deployment manifest, with no It/Describe/Context or unsupported MicroShift API use.
Single Node Openshift (Sno) Test Compatibility ✅ Passed No new Ginkgo/e2e test files or It/Describe/Context/When additions were changed, so no SNO assumptions to review.
Topology-Aware Scheduling Compatibility ✅ Passed No new topology-sensitive scheduling constraints were added; the deployment stays single-replica/Recreate with no anti-affinity or topology spread changes.
Ote Binary Stdout Contract ✅ Passed No process-level stdout writes were added; cmd.go only uses klog/logs.InitLogs, and the binary main sends errors to stderr.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The patch changes only CLI, metrics, and YAML; no _test.go files or Ginkgo e2e definitions were added, and no IPv4/external-connectivity assumptions appear in the diff.
No-Weak-Crypto ✅ Passed Touched files only use crypto/tls and library-go TLS profile helpers; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB or secret compares found.
Container-Privileges ✅ Passed The touched deployment keeps a restricted securityContext: runAsNonRoot, allowPrivilegeEscalation:false, drop ALL; no privileged/hostNetwork/hostPID/hostIPC/SYS_ADMIN found.
No-Sensitive-Data-In-Logs ✅ Passed No added logs expose secrets/PII/hostnames; new logging only reports TLS profile metadata and generic startup/status messages.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="Running error: context loading failed: failed to load packages: failed to load packages: failed to load with go/packages: err: exit status 1: stderr: go: inconsistent vendoring in :\n\tgithub.com/Masterminds/semver@v1.5.0: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/Masterminds/sprig/v3@v3.2.3: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/containernetworking/cni@v1.3.0: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/ghodss/yaml@v1.0.1-0.20190212211648-25d852aebe32: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/go-bindata/go-bindata@v3.1.2+incompatible: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/onsi/gomega@v1.39.1: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tgithub.com/ope

... [truncated 17377 characters] ...

red in go.mod, but not marked as explicit in vendor/modules.txt\n\tk8s.io/gengo/v2@v2.0.0-20251215205346-5ee0d033ba5b: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tk8s.io/kms@v0.35.2: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tk8s.io/kube-aggregator@v0.35.1: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tsigs.k8s.io/randfill@v1.0.0: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\tsigs.k8s.io/structured-merge-diff/v6@v6.3.2: is explicitly required in go.mod, but not marked as explicit in vendor/modules.txt\n\n\tTo ignore the vendor directory, use -mod=readonly or -mod=mod.\n\tTo sync the vendor directory, run:\n\t\tgo mod vendor\n"


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from jcaamano and taanyas July 13, 2026 21:28
@tpantelis tpantelis changed the title CORENET-7356Add TLS profile support for network-check-source metrics CORENET-7356: Add TLS profile support for network-check-source metrics Jul 13, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 13, 2026
@openshift-ci-robot

openshift-ci-robot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

@tpantelis: This pull request references CORENET-7356 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Use the OCP API server TLS profile to configure TLS options for the metrics server and exit the process on TLS profile changes.

library-go's controllercmd framework does not have support for TLS profiles and does not easily allow TLS options to be customized. Therefore the code was migrated to use the controller-runtime Manager API which allows direct customization.

The code respects the TLS profile adherence setting and uses the library-go strong defaults when adherence is optional so as not to fall back to weaker Go default ciphers.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@tpantelis

Copy link
Copy Markdown
Contributor Author

/cc @danwinship

@openshift-ci
openshift-ci Bot requested a review from danwinship July 13, 2026 21:32

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/cmd/checkendpoints/cmd.go`:
- Line 169: Update the setup around controller.RegisterMetrics() so its
collectors are registered with controller-runtime’s metrics.Registry, or
explicitly bridge the legacy registry into that registry before serving metrics.
Preserve the existing collector registration behavior while ensuring the
collectors appear on the controller-runtime /metrics endpoint.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: b60db286-01ee-41d5-9bad-36733b28ea2d

📥 Commits

Reviewing files that changed from the base of the PR and between e7f4de6 and a3f09cf.

📒 Files selected for processing (1)
  • pkg/cmd/checkendpoints/cmd.go

Comment thread pkg/cmd/checkendpoints/cmd.go
@tpantelis
tpantelis force-pushed the network_check_source_tls branch from a3f09cf to 0aed9d8 Compare July 13, 2026 21:42
@tpantelis

Copy link
Copy Markdown
Contributor Author

@coderabbitai approve

@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown
✅ Action performed

Comments resolved and changes approved.

@tpantelis
tpantelis force-pushed the network_check_source_tls branch from 0aed9d8 to fcdb6b9 Compare July 14, 2026 12:34
Use the OCP API server TLS profile to configure TLS options for the
metrics server and exit the process on TLS profile changes.

library-go's controllercmd framework does not have support for TLS
profiles and does not easily allow TLS options to be customized.
Therefore the code was migrated to use the controller-runtime Manager
API which allows direct customization.

The code respects the TLS profile adherence setting and uses the
library-go strong defaults when adherence is optional so as not to
fall back to weaker Go default ciphers.

Signed-off-by: Tom Pantelis <tompantelis@gmail.com>
@tpantelis
tpantelis force-pushed the network_check_source_tls branch from fcdb6b9 to f05b3c5 Compare July 15, 2026 13:40
@openshift-ci

openshift-ci Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: tpantelis
Once this PR has been reviewed and has the lgtm label, please assign kyrtapz for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@danwinship

Copy link
Copy Markdown
Contributor

/pipeline required

I don't really understand all of this, but it should fail pretty obviously if something is wrong

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-ovn-upgrade-ipsec
/test e2e-gcp-ovn-techpreview

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aws-ovn-fdp-qe
/test e2e-aws-ovn-hypershift-conformance
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-aws-ovn-upgrade
/test e2e-aws-ovn-windows
/test e2e-azure-ovn-upgrade
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-dualstack-bgp
/test e2e-metal-ipi-ovn-dualstack-bgp-local-gw
/test e2e-metal-ipi-ovn-ipv6
/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-ovn-ipsec-step-registry
/test hypershift-e2e-aks

@danwinship

Copy link
Copy Markdown
Contributor

/assign

@tpantelis

Copy link
Copy Markdown
Contributor Author

I don't really understand all of this, but it should fail pretty obviously if something is wrong

We'll see how many succeed. But the failures I've looked at (with claude's help) are infrastructure-related or caused by other issues not related to this PR. Also this PR changes the network-check-source which is for diagnostics so not sure what CI job, if any, actually tests it. I deployed and tested it manually locally. @weliang1 intends to do pre-merge testing with this PR.

@openshift-ci

openshift-ci Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

@tpantelis: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-gcp-ovn-techpreview f05b3c5 link true /test e2e-gcp-ovn-techpreview
ci/prow/e2e-metal-ipi-ovn-ipv6-ipsec f05b3c5 link true /test e2e-metal-ipi-ovn-ipv6-ipsec
ci/prow/e2e-gcp-ovn f05b3c5 link true /test e2e-gcp-ovn
ci/prow/5.0-upgrade-from-stable-4.22-e2e-azure-ovn-upgrade f05b3c5 link false /test 5.0-upgrade-from-stable-4.22-e2e-azure-ovn-upgrade
ci/prow/e2e-ovn-ipsec-step-registry f05b3c5 link true /test e2e-ovn-ipsec-step-registry
ci/prow/e2e-gcp-ovn-upgrade f05b3c5 link true /test e2e-gcp-ovn-upgrade
ci/prow/hypershift-e2e-aks f05b3c5 link true /test hypershift-e2e-aks
ci/prow/e2e-metal-ipi-ovn-dualstack-bgp-local-gw f05b3c5 link true /test e2e-metal-ipi-ovn-dualstack-bgp-local-gw
ci/prow/e2e-azure-ovn-upgrade f05b3c5 link true /test e2e-azure-ovn-upgrade
ci/prow/e2e-aws-ovn-serial-2of2 f05b3c5 link true /test e2e-aws-ovn-serial-2of2
ci/prow/e2e-aws-ovn-serial-1of2 f05b3c5 link true /test e2e-aws-ovn-serial-1of2
ci/prow/e2e-metal-ipi-ovn-ipv6 f05b3c5 link true /test e2e-metal-ipi-ovn-ipv6
ci/prow/e2e-aws-ovn-hypershift-conformance f05b3c5 link true /test e2e-aws-ovn-hypershift-conformance
ci/prow/e2e-aws-ovn-upgrade f05b3c5 link true /test e2e-aws-ovn-upgrade
ci/prow/5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade f05b3c5 link false /test 5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade
ci/prow/e2e-metal-ipi-ovn-dualstack-bgp f05b3c5 link true /test e2e-metal-ipi-ovn-dualstack-bgp
ci/prow/e2e-aws-ovn-upgrade-ipsec f05b3c5 link true /test e2e-aws-ovn-upgrade-ipsec
ci/prow/5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-upgrade f05b3c5 link false /test 5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-upgrade

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@danwinship

Copy link
Copy Markdown
Contributor

@tpantelis

F0715 17:29:50.341379 1 cmd.go:53] failed to fetch TLS profile: failed to get TLS profile: failed to fetch apiserver.config.openshift.io/cluster: apiservers.config.openshift.io "cluster" is forbidden: User "system:serviceaccount:openshift-network-diagnostics:network-diagnostics" cannot get resource "apiservers" in API group "config.openshift.io" at the cluster scope

It might be better to just have CNO deploy the pods with --tls-cipher-suites args or whatever, rather than having them try to figure it out themselves...

@tpantelis

Copy link
Copy Markdown
Contributor Author

It might be better to just have CNO deploy the pods with --tls-cipher-suites args or whatever, rather than having them try to figure it out themselves...

yeah that's a good idea. We'll still need the controller-runtime approach to apply the TLS args, unless we keep controllercmd and do similar ugliness that I did in CNO main.go. I'd rather use controller-runtime, also b/c I don't think controllercmd will be updated to support TLS curve/group preferences coming in 5.1. Also changes in #3043 will be needed so I might as well add another commit to that PR. Sound good?

@danwinship

Copy link
Copy Markdown
Contributor

sounds good

@tpantelis

Copy link
Copy Markdown
Contributor Author

sounds good

Submitted 1117c8e and cbf9d4e to #3043

@tpantelis tpantelis closed this Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants