Skip to content

fix(clusterchecksrunner): preserve packaged conf.d assets (e.g. SNMP … - #3374

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 5 commits into
mainfrom
fspano/CONS-8516-clc-preserve-check-assets
Aug 24, 2026
Merged

fix(clusterchecksrunner): preserve packaged conf.d assets (e.g. SNMP …#3374
gh-worker-dd-mergequeue-cf854d[bot] merged 5 commits into
mainfrom
fspano/CONS-8516-clc-preserve-check-assets

Conversation

@frank-spano

@frank-spano frank-spano commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

…profiles) on CLC runners (CONS-8516)

What does this PR do?

Adds an init container to the cluster-checks-runner that seeds the conf.d overlay with the agent image's packaged check assets (e.g. SNMP default_profiles), then removes only *.yaml.default so default core checks still don't run. This preserves out-of-the-box profiles on Operator CLC runners—aligning with Helm #1594—so SNMP/NDM cluster checks can autodetect profiles without a manual init-container workaround.

Motivation

https://datadoghq.atlassian.net/browse/CONS-8516

Additional Notes

Manually tested -

Ran the patched operator locally (go run ./cmd/main.go) against a minikube cluster and applied a DatadogAgent with features.clusterChecks.useClusterChecksRunners: true plus an SNMP cluster check via clusterAgent.extraConfd. On the resulting cluster-checks-runner pod I confirmed the new init-copy-check-assets init container ran and that /etc/datadog-agent/conf.d/snmp.d/default_profiles is now populated (e.g. _arista.yaml, _aruba-base.yaml, …), while find /etc/datadog-agent/conf.d -name '*.yaml.default' returns 0 — so packaged profiles are preserved and default core checks are still pruned. In agent status, the previous profile errors (failed to read profile dir … no such file or directory / no profiles found for sysObjectID) are gone; the SNMP check now proceeds past profile autodetection and only fails on device reachability (request timeout), which is expected since there's no real SNMP device at the test IP. This confirms the fix restores OOTB SNMP profiles on Operator CLC runners. Unit tests and the render golden files were also updated and pass.

Minimum Agent Versions

Are there minimum versions of the Datadog Agent and/or Cluster Agent required?

  • Agent: vX.Y.Z
  • Cluster Agent: vX.Y.Z

Describe your test plan

Write there any instructions and details you may have to test your PR.

Checklist

  • PR has at least one valid label: bug, enhancement, refactoring, documentation, tooling, and/or dependencies
  • PR has a milestone or the qa/skip-qa label
  • All commits are signed (see: signing commits)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0301eb3f53

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

// cluster checks such as the SNMP check autodetect profiles.
copyCheckAssetsCmd = `set -euo pipefail
if [ -d /etc/datadog-agent/conf.d ]; then
cp -a /etc/datadog-agent/conf.d/. ` + common.RmCorechecksConfdInitPath + `/

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid preserving root-owned attributes in the asset copy

When the Cluster Checks Runner is forced to run as a non-root UID—such as under an OpenShift restricted SCC or a pod-level securityContext.runAsUser override—this init container exits here and blocks the Deployment. cp --help defines -a as -dR --preserve=all, so copying the root-owned image directory attempts to preserve ownership and directory timestamps; a non-root reproduction returns Operation not permitted, and set -e prevents the runner from starting. Copy the assets recursively without preserving privileged attributes.

Useful? React with 👍 / 👎.

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Aug 20, 2026

Copy link
Copy Markdown

Code Coverage

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 50.18% (+0.05%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ba28f38 | Docs | View more details | Give us feedback!

The patch-coverage gate computes per-package coverage (go test ./...
without -coverpkg), so GetVolumeMountForRmCorechecksInit and the edited
remove-corechecks volume helpers in the common package registered as 0%
even though a clusterchecksrunner test exercises them. Add a common-package
test that covers these helpers directly.
cp -a (= --preserve=all) tries to preserve the root ownership of the
image's packaged conf.d. When the runner is forced to run as a non-root
UID (OpenShift restricted SCC, or a securityContext.runAsUser override),
that fails with "Operation not permitted" and, under set -e, blocks the
pod from starting. Use cp -RL to copy recursively without preserving
privileged attributes so the assets are owned by the runtime user.
@frank-spano frank-spano added the bug Something isn't working label Aug 21, 2026
@frank-spano frank-spano added this to the v1.30.0 milestone Aug 21, 2026
@levan-m levan-m modified the milestones: v1.30.0, v1.31.0 Aug 21, 2026
name: checksd
readOnly: true
- args:
- |-

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will lead to errors when simply upgrading on GKE Autopilot so first allowlist needs to be updated.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@levan-m The init-copy-check-assets init container only lands on the cluster-checks-runner Deployment, not the node agent DaemonSet. That Deployment sets no host namespaces, so it doesn't match the datadog-datadog-daemonset-exemption allowlist's hostPID: true criteria, and there's no CLC-runner WorkloadAllowlist. The CLC runner already runs a bash init-config init container on Autopilot today; this new one is the same shape (agent image, readOnlyRootFilesystem, allowPrivilegeEscalation: false, emptyDir only), so it should run under Autopilot's default policy without an exemption. Did you have a specific rule in mind that it would trip, or a cluster where you saw the rejection? Happy to add it to the allowlist + bump the synchronizer version if you can confirm it's actually required.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, somehow assumed with was daemonset. Deployments aren't affected by allowlist.

@frank-spano
frank-spano requested a review from levan-m August 24, 2026 13:39
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot merged commit 34e8a62 into main Aug 24, 2026
76 checks passed
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot deleted the fspano/CONS-8516-clc-preserve-check-assets branch August 24, 2026 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants