Skip to content

Bump ovn-controller CPU request to 100m to prevent CPU starvation#3051

Open
shreyasbe wants to merge 1 commit into
openshift:masterfrom
shreyasbe:UDN_CORENET_7217_adjust_cpu_priority
Open

Bump ovn-controller CPU request to 100m to prevent CPU starvation#3051
shreyasbe wants to merge 1 commit into
openshift:masterfrom
shreyasbe:UDN_CORENET_7217_adjust_cpu_priority

Conversation

@shreyasbe

@shreyasbe shreyasbe commented Jul 14, 2026

Copy link
Copy Markdown

Why this change is needed:
The default 10m CPU request for ovn-controller causes CPU starvation during bulk User Defined Network (UDN) creation, resulting in "Unreasonably long poll interval" warnings (1000ms-4000ms+) and delayed network provisioning.

What this change does:
Increases the ovn-controller CPU request from 10m to 100m in the ovnkube-node DaemonSet to ensure sufficient CPU resources during heavy control-plane reconciliation operations.

Testing:

  • Manual testing: Created 50 UDNs across multiple namespaces on a 3-node cluster
  • Before: Multiple "Unreasonably long poll interval" warnings (1154ms-4401ms) with 0ms user time indicating CPU starvation
  • After: Zero warnings during the same bulk UDN creation test

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci
openshift-ci Bot requested review from kyrtapz and marty-power July 14, 2026 07:36
@shreyasbe shreyasbe changed the title Increase ovn-controller CPU request from 10m to 100m to prevent CPU s… Increase ovn-controller CPU request from 10m to 100m to prevent CPU starvation Jul 14, 2026
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Walkthrough

The managed and self-hosted OVN Kubernetes node DaemonSet manifests increase the ovn-controller container CPU request from 10m to 100m.

Changes

OVN controller resources

Layer / File(s) Summary
Update OVN controller CPU request
bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml, bindata/network/ovn-kubernetes/self-hosted/ovnkube-node.yaml
Changes the ovn-controller CPU request from 10m to 100m in both manifests.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Suggested reviewers: marty-power


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (4 errors)

Check name Status Explanation Resolution
Pr Quality ❌ Error The PR text is only 2 sentences: it lacks explicit Why/What/CI verification sections and any issue link; testing is manual only. Add clear Why, What, and automated CI verification sections, plus an issue link and any rollback/impact notes for this behavioral change.
Commit Message Quality ❌ Error The commit is logical and has a body, but the subject isn’t prefixed with the affected component as requested. Rewrite the subject with a component prefix, e.g. 'ovn-kubernetes: Bump ovn-controller CPU request to 100m', and keep the explanatory body.
Unit Tests For Go Changes ❌ Error Bindata ovnkube-node YAML templates changed, but the PR adds/modifies no *_test.go files. Add or update tests alongside the manifest change, or document why tests are unnecessary in the PR description under "How to verify it" and use @coderabbitai ignore pre-merge checks if appropriate.
Container-Privileges ❌ Error Both changed ovnkube-node manifests still set hostNetwork:true, hostPID:true, and multiple privileged:true containers. Remove or justify these privilege settings, or exempt these node-daemonset manifests if the elevated privileges are intentional.
✅ Passed checks (20 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
E2e Tests For Feature Changes ✅ Passed Only two bindata YAML manifests changed; no pkg/cmd Go files or test/e2e files were modified, so the E2E rule doesn't apply.
Rbac Least Privilege ✅ Passed Only ovn-controller CPU requests changed in two ovnkube-node manifests; no Role/ClusterRole rules or RBAC fields were modified.
Docs For Feature And Behavior Changes ✅ Passed Only two ovnkube-node manifest CPU-request edits; no docs/ files changed, and the change is a bug-fix resource tweak allowed by policy.
Stale Project Docs And Config ✅ Passed No docs or config reference the changed ovnkube-node CPU request, and no referenced file paths or entities were renamed or removed.
Go And Test Code Quality ✅ Passed Only two YAML manifests changed; no Go or test code was modified, so none of the listed Go/test quality issues apply.
Ai-Generated Code Smell ✅ Passed The patch only bumps ovn-controller CPU requests from 10m to 100m in two manifests; no AI-style comments, test bloat, or verbose logic were added.
Stable And Deterministic Test Names ✅ Passed Only ovnkube-node YAML manifests are in scope; no test files or Ginkgo titles were changed, so the check is not applicable.
Test Structure And Quality ✅ Passed PR only changes two YAML manifests; no Ginkgo or test files are touched, so the test-structure review is not applicable.
Microshift Test Compatibility ✅ Passed PR only changes two ovnkube-node YAML manifests; no Ginkgo/e2e tests were added or modified, so MicroShift test compatibility is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed Only two YAML manifests changed; no Ginkgo/e2e test files or test logic were added, so the SNO compatibility check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed Only the ovn-controller CPU request changed from 10m to 100m in two DaemonSets; no topology/scheduling constraints were added or modified.
Ote Binary Stdout Contract ✅ Passed PR only changes ovn-kubernetes DaemonSet CPU requests in YAML; no process-level code or stdout writes were added.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed Only two ovnkube-node YAML manifests changed; no Ginkgo/e2e tests or network-connecting code were added, so the IPv6/disconnected check is not applicable.
No-Weak-Crypto ✅ Passed Only CPU requests in two YAML manifests changed; no MD5/SHA1/DES/RC4/3DES/Blowfish, ECB, custom crypto, or secret-comparison code is present.
No-Sensitive-Data-In-Logs ✅ Passed Only CPU requests changed in two YAML manifests; no logging statements or sensitive data were introduced.
Title check ✅ Passed The title clearly summarizes the CPU request increase and the motivation.
Description check ✅ Passed The description is directly related to the ovn-controller CPU request increase and explains the rationale and testing.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@tssurya

tssurya commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

@coderabbitai: why aren't you asking for "changes requested" via GUI?

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

@tssurya CodeRabbit intentionally does not submit formal GitHub "Request Changes" review states by default. Here's why:

  • CodeRabbit posts its review as a set of comments (using GitHub's "Comment" review event) rather than "Approve" or "Request Changes," so it doesn't block merges via branch protection rules that require review approval/resolution.
  • This design choice avoids CodeRabbit becoming a hard gate on PRs — teams often want AI feedback to be advisory, with humans deciding whether findings are blocking.
  • The only exception is explicit approval: when a maintainer or PR author comments @coderabbitai approve, CodeRabbit will submit an "Approve" review via the GUI. There isn't a corresponding @coderabbitai request changes command — CodeRabbit is designed to leave the "Request Changes" decision to human reviewers.

If you'd like CodeRabbit to behave differently (e.g., more strongly flag certain issues, or you want stricter gating), that would need to be handled through your repository's branch protection rules or required status checks rather than CodeRabbit's review state, since CodeRabbit's core behavior here isn't currently configurable via YAML settings.

@tssurya

tssurya commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

@coderabbitai: I want it to be hard gating, how can I achieve that?

@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Tip

For best results, initiate chat on the files or code changes.

@tssurya: Let me pull the current configuration schema to give you the precise setting name and behavior.

resources:
requests:
cpu: 10m
cpu: 100m

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coderabbitai why did you miss

- When changing a file in one, ALWAYS check if the same change is needed in
the other. This is the #1 source of HyperShift drift bugs. Examples:
masquerade subnet config, interconnect flags, IPv6 URL formatting.
Some settings are HyperShift-only (SOCKS proxy, different cert paths,
`{{.OVNControlPlaneImage}}`), but most feature flags must appear in both.

@shreyasbe you are missing: https://github.com/openshift/cluster-network-operator/blob/master/bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
cat -n .coderabbit.yaml | sed -n '95,125p'
echo "---managed ovnkube-node.yaml ovn-controller resources---"
rg -n "ovn-controller" -A 40 bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml | head -80

Length of output: 2084


🏁 Script executed:

#!/bin/bash
fd -i coderabbit.yaml
find / -maxdepth 2 -iname ".coderabbit.yaml" 2>/dev/null

Length of output: 229


🏁 Script executed:

#!/bin/bash
ls -la | grep -i coderabbit
git show HEAD:.coderabbit.yaml 2>&1 | sed -n '95,125p'

Length of output: 233


🏁 Script executed:

#!/bin/bash
pwd
ls -la
git log --oneline -3
git show e7f4de6204c26e2f8784fc4c7e2b508b08324fb5:.coderabbit.yaml | sed -n '95,125p'

Length of output: 4099


@kyrtapz You're right, and I apologize for the miss — the managed/self-hosted alignment check in .coderabbit.yaml (lines 109-113) should have flagged this. I confirmed the drift directly:

bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml (HyperShift control-plane) still has:

      - name: ovn-controller
        ...
        resources:
          requests:
            cpu: 10m
            memory: 300Mi

while bindata/network/ovn-kubernetes/self-hosted/ovnkube-node.yaml was bumped to cpu: 100m in this PR. Since the same CPU-starvation issue (long poll intervals under bulk UDN creation) would equally affect the HyperShift-managed ovn-controller, the managed/ovnkube-node.yaml file should get the identical change for consistency and to avoid drift between the two topologies.


✏️ Learnings added
Learnt from: kyrtapz
Repo: openshift/cluster-network-operator PR: 3051
File: bindata/network/ovn-kubernetes/self-hosted/ovnkube-node.yaml:189-189
Timestamp: 2026-07-14T16:57:34.955Z
Learning: In openshift/cluster-network-operator, when a resource request/limit (e.g. CPU/memory) is changed for a container in `bindata/network/ovn-kubernetes/self-hosted/ovnkube-node.yaml` (e.g. the `ovn-controller` container), the parallel container in `bindata/network/ovn-kubernetes/managed/ovnkube-node.yaml` (HyperShift control-plane) must be checked and updated to match, per the managed/self-hosted alignment rule in `.coderabbit.yaml`. Missing this is the `#1` source of HyperShift drift bugs.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @kyrtapz for the pointer. I have addressed the code changes.

@shreyasbe
shreyasbe force-pushed the UDN_CORENET_7217_adjust_cpu_priority branch from 4f30235 to 3388762 Compare July 15, 2026 04:18
The default 10m CPU request causes 'Unreasonably long poll interval' warnings
during bulk UDN creation due to CPU starvation. Testing shows increasing to
100m eliminates these warnings and prevents network provisioning delays.

Tested: 50 UDNs created with zero warnings after change.

Signed-off-by: Shreyas Be <52690686+shreyasbe@users.noreply.github.com>
@shreyasbe
shreyasbe force-pushed the UDN_CORENET_7217_adjust_cpu_priority branch from 3388762 to bfa0556 Compare July 15, 2026 04:45
@shreyasbe shreyasbe changed the title Increase ovn-controller CPU request from 10m to 100m to prevent CPU starvation Bump ovn-controller CPU request to 100m to prevent CPU starvation Jul 15, 2026
@shreyasbe

Copy link
Copy Markdown
Author

/test 5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-upgrade

@shreyasbe

Copy link
Copy Markdown
Author

/test 5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade

@shreyasbe

Copy link
Copy Markdown
Author

/retest-required

@kyrtapz

kyrtapz commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 16, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling required tests:
/test e2e-aws-ovn-upgrade-ipsec
/test e2e-gcp-ovn-techpreview

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aws-ovn-fdp-qe
/test e2e-aws-ovn-hypershift-conformance
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-aws-ovn-upgrade
/test e2e-aws-ovn-windows
/test e2e-azure-ovn-upgrade
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-dualstack-bgp
/test e2e-metal-ipi-ovn-dualstack-bgp-local-gw
/test e2e-metal-ipi-ovn-ipv6
/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-ovn-ipsec-step-registry
/test hypershift-e2e-aks

@openshift-ci

openshift-ci Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kyrtapz, shreyasbe

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 16, 2026
@shreyasbe

Copy link
Copy Markdown
Author

/retest-required

@shreyasbe

Copy link
Copy Markdown
Author

/retest

@shreyasbe

Copy link
Copy Markdown
Author

/test e2e-aws-ovn-upgrade-ipsec
/test e2e-gcp-ovn-techpreview
/test e2e-aws-ovn-fdp-qe
/test e2e-aws-ovn-hypershift-conformance
/test e2e-aws-ovn-serial-1of2
/test e2e-aws-ovn-serial-2of2
/test e2e-aws-ovn-upgrade
/test e2e-aws-ovn-windows
/test e2e-azure-ovn-upgrade
/test e2e-gcp-ovn
/test e2e-gcp-ovn-upgrade
/test e2e-metal-ipi-ovn-dualstack-bgp
/test e2e-metal-ipi-ovn-dualstack-bgp-local-gw
/test e2e-metal-ipi-ovn-ipv6
/test e2e-metal-ipi-ovn-ipv6-ipsec
/test e2e-ovn-ipsec-step-registry
/test hypershift-e2e-aks

@openshift-ci

openshift-ci Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

@shreyasbe: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-hypershift-conformance bfa0556 link true /test e2e-aws-ovn-hypershift-conformance
ci/prow/e2e-ovn-ipsec-step-registry bfa0556 link true /test e2e-ovn-ipsec-step-registry
ci/prow/e2e-metal-ipi-ovn-ipv6 bfa0556 link true /test e2e-metal-ipi-ovn-ipv6
ci/prow/5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade bfa0556 link false /test 5.0-upgrade-from-stable-4.22-e2e-aws-ovn-upgrade
ci/prow/e2e-gcp-ovn-techpreview bfa0556 link true /test e2e-gcp-ovn-techpreview
ci/prow/e2e-metal-ipi-ovn-dualstack-bgp-local-gw bfa0556 link true /test e2e-metal-ipi-ovn-dualstack-bgp-local-gw
ci/prow/e2e-metal-ipi-ovn-ipv6-ipsec bfa0556 link true /test e2e-metal-ipi-ovn-ipv6-ipsec
ci/prow/e2e-aws-ovn-serial-2of2 bfa0556 link true /test e2e-aws-ovn-serial-2of2
ci/prow/e2e-aws-ovn-windows bfa0556 link true /test e2e-aws-ovn-windows
ci/prow/hypershift-e2e-aks bfa0556 link true /test hypershift-e2e-aks
ci/prow/e2e-aws-ovn-upgrade-ipsec bfa0556 link true /test e2e-aws-ovn-upgrade-ipsec
ci/prow/e2e-aws-ovn-serial-1of2 bfa0556 link true /test e2e-aws-ovn-serial-1of2
ci/prow/5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-upgrade bfa0556 link false /test 5.0-upgrade-from-stable-4.22-e2e-gcp-ovn-upgrade
ci/prow/e2e-aws-ovn-upgrade bfa0556 link true /test e2e-aws-ovn-upgrade
ci/prow/e2e-metal-ipi-ovn-dualstack-bgp bfa0556 link true /test e2e-metal-ipi-ovn-dualstack-bgp
ci/prow/e2e-azure-ovn-upgrade bfa0556 link true /test e2e-azure-ovn-upgrade
ci/prow/e2e-gcp-ovn bfa0556 link true /test e2e-gcp-ovn

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants