Skip to content

CNTRLPLANE-2654: added the library functions for workloads - #2336

Open
sandeepknd wants to merge 1 commit into
openshift:masterfrom
sandeepknd:ote-kdso-lib
Open

CNTRLPLANE-2654: added the library functions for workloads#2336
sandeepknd wants to merge 1 commit into
openshift:masterfrom
sandeepknd:ote-kdso-lib

Conversation

@sandeepknd

@sandeepknd sandeepknd commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Added the library helper functions for workloads.
Kindly refer this openshift/cluster-kube-descheduler-operator#2030 (comment) for the actual context.

Summary by CodeRabbit

  • New Features
    • Added helper support for cordoning/uncordoning Kubernetes nodes by toggling schedulability.
    • Added OLM workload helpers to create/delete OperatorGroups and Subscriptions, including automatic install plan and optional starting CSV.
    • Added utilities to verify CatalogSource availability and to fetch CSV details (name, related images, and phase).
    • Added helpers to read package metadata and select the current CSV from the default channel.
  • Bug Fixes
    • Avoid redundant cordon/uncordon operations when nodes are already in the requested state.
    • Improved CSV polling to treat “Succeeded” as success and “Failed” as an immediate error.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 26, 2026
@openshift-ci-robot

openshift-ci-robot commented Jun 26, 2026

Copy link
Copy Markdown

@sandeepknd: This pull request references CNTRLPLANE-2654 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the epic to target the "5.0.0" version, but no target version was set.

Details

In response to this:

Added the library helper functions for workloads.
Kindly refer this openshift/cluster-kube-descheduler-operator#2030 (comment) for the actual context.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Adds node cordon/uncordon helpers and OLM helpers for resource lifecycle, CSV lookup, and package-manifest mapping.

Changes

Node workload helpers

Layer / File(s) Summary
Cordon and uncordon flow
test/library/workloads/node.go
Fetches Nodes, checks spec.unschedulable, and patches the field when cordoning or uncordoning.

OLM resource helpers

Layer / File(s) Summary
OLM types and GVRs
test/library/workloads/olm.go
Adds OLM helper types, adds SourceNamespace to Subscription, and exposes GVR accessors for the supported resource kinds.
OperatorGroup lifecycle
test/library/workloads/olm.go
Creates and deletes OperatorGroups through the dynamic client with polling around the unstructured resource operations.
Subscription and catalog source flow
test/library/workloads/olm.go
Creates and deletes Subscriptions, sets source and CSV fields, and verifies the referenced CatalogSource.
CSV and package manifest helpers
test/library/workloads/olm.go
Adds CSV lookup, related-image extraction, CSV phase polling, and package-manifest lookup that selects the default channel’s currentCSV.

🎯 3 (Moderate) | ⏱️ ~30 minutes

🚥 Pre-merge checks | ✅ 14 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (14 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed No Ginkgo test titles were added in the changed files; both are helper libraries, not test specs.
Test Structure And Quality ✅ Passed PR only adds workload helper libraries; no Ginkgo test blocks or test setup/cleanup patterns are present to review.
Microshift Test Compatibility ✅ Passed PASS: The PR only adds helper functions in test/library/workloads; no new Ginkgo test blocks or MicroShift-incompatible test code were added.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The PR only adds workload helper libraries; changed files contain no Ginkgo specs or SNO-sensitive multi-node assumptions.
Topology-Aware Scheduling Compatibility ✅ Passed PASS: The PR only adds test helper functions for cordoning nodes and OLM resource CRUD; no manifests/controllers, replicas, affinities, selectors, or topology-aware scheduling logic.
Ote Binary Stdout Contract ✅ Passed The PR only adds helper functions in test/library/workloads; no main/init/TestMain/suite setup or top-level stdout writes were added.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PASS: The PR only adds helper-library code in test/library/workloads; no new Ginkgo tests, IPv4 literals, or external connectivity requirements were found.
No-Weak-Crypto ✅ Passed Changed files only add Kubernetes/OLM helper logic; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret/token timing compares found.
Container-Privileges ✅ Passed Changed workload helper Go files add node/OLM helpers only; no container specs or privileged/host* security settings appear in the PR files.
No-Sensitive-Data-In-Logs ✅ Passed No added logs print secrets/PII; messages only include resource names, namespaces, CSV metadata, and generic retry errors.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately reflects the main change: adding workload-related library helper functions.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@sandeepknd

Copy link
Copy Markdown
Contributor Author

/assign @ingvagabund
/assign @YamunadeviShanmugam
/assign @ropatil010
/assign @gangwgr

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/library/workloads/node.go`:
- Around line 15-29: The Node cordon helper currently does a direct
get/mutate/update flow and can fail on normal resource-version conflicts. Update
the logic in the Node helper(s) around the get/set Unschedulable/Update sequence
to retry on conflicts using retry.RetryOnConflict, or replace the Update with a
targeted patch to spec.unschedulable. Keep the existing error handling in the
cordon function so transient conflicts are retried instead of surfacing as flaky
failures.

In `@test/library/workloads/olm.go`:
- Around line 343-353: The code in the OLM workload helpers is ignoring error
returns from unstructured.NestedString, which can hide a wrongly typed
currentCSV and leave startingCSV empty without any signal. Update the logic in
the channel-selection flow and the similar NestedString usage elsewhere to
capture and handle the returned error instead of discarding it; if parsing
currentCSV fails, propagate or report the error so the caller can detect the bad
channel data. Use the existing startingCSV, channels, and defaultChannel path to
locate the affected calls and apply the same fix consistently.
- Around line 100-115: The retry helpers for OperatorGroup and Subscription
creation/deletion are losing the real API error and are not idempotent. Update
the polling logic in the create/delete helpers (including CreateSubscription and
the delete counterparts) to preserve and return the last error from the closure
instead of returning false,nil on failures. Use apierrors.IsAlreadyExists to
treat pre-existing creates as success and apierrors.IsNotFound to treat missing
deletes as success, so PollUntilContextTimeout does not mask the underlying
failure and the helpers become idempotent.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 21bb9ee6-e14a-4d0b-a2f9-756db3e75852

📥 Commits

Reviewing files that changed from the base of the PR and between dd144d2 and 2cc8ed2.

📒 Files selected for processing (2)
  • test/library/workloads/node.go
  • test/library/workloads/olm.go

Comment thread test/library/workloads/node.go Outdated
Comment thread test/library/olm/olm.go Outdated
Comment on lines +100 to +115
err := wait.PollUntilContextTimeout(ctx, 5*time.Second, 20*time.Second, true, func(ctx context.Context) (bool, error) {
_, err := dynamicClient.Resource(GetOperatorGroupGVR()).Namespace(og.Namespace).Create(ctx, operatorGroup, metav1.CreateOptions{})
if err != nil {
klog.Warningf("Failed to create OperatorGroup, retrying: %v", err)
return false, nil
}
return true, nil
})

if err != nil {
return fmt.Errorf("failed to create OperatorGroup %s: %w", og.Name, err)
}

klog.Infof("Successfully created OperatorGroup %s", og.Name)
return nil
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Create retry loses the real API error and is not idempotent.

The poll closure returns false, nil on every Create error, so when the 20s window expires PollUntilContextTimeout returns a generic timeout error — the actual API failure cause is discarded and only logged at warning level, making failures hard to debug. Additionally, if the OperatorGroup already exists, Create returns AlreadyExists and this helper retries until timeout and then fails, instead of treating a pre-existing resource as success. The same pattern applies to CreateSubscription (Lines 167-178), and the delete counterparts (Lines 121-128 and 188-195) similarly retry on NotFound instead of treating it as already-deleted.

Consider capturing the last error and short-circuiting on AlreadyExists/NotFound via apierrors.IsAlreadyExists/apierrors.IsNotFound.

🛠️ Sketch for create idempotency + error preservation
-	err := wait.PollUntilContextTimeout(ctx, 5*time.Second, 20*time.Second, true, func(ctx context.Context) (bool, error) {
-		_, err := dynamicClient.Resource(GetOperatorGroupGVR()).Namespace(og.Namespace).Create(ctx, operatorGroup, metav1.CreateOptions{})
-		if err != nil {
-			klog.Warningf("Failed to create OperatorGroup, retrying: %v", err)
-			return false, nil
-		}
-		return true, nil
-	})
+	var lastErr error
+	err := wait.PollUntilContextTimeout(ctx, 5*time.Second, 20*time.Second, true, func(ctx context.Context) (bool, error) {
+		_, createErr := dynamicClient.Resource(GetOperatorGroupGVR()).Namespace(og.Namespace).Create(ctx, operatorGroup, metav1.CreateOptions{})
+		if createErr != nil && !apierrors.IsAlreadyExists(createErr) {
+			lastErr = createErr
+			klog.Warningf("Failed to create OperatorGroup, retrying: %v", createErr)
+			return false, nil
+		}
+		return true, nil
+	})
+	if err != nil && lastErr != nil {
+		err = lastErr
+	}

(requires apierrors "k8s.io/apimachinery/pkg/api/errors")

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
err := wait.PollUntilContextTimeout(ctx, 5*time.Second, 20*time.Second, true, func(ctx context.Context) (bool, error) {
_, err := dynamicClient.Resource(GetOperatorGroupGVR()).Namespace(og.Namespace).Create(ctx, operatorGroup, metav1.CreateOptions{})
if err != nil {
klog.Warningf("Failed to create OperatorGroup, retrying: %v", err)
return false, nil
}
return true, nil
})
if err != nil {
return fmt.Errorf("failed to create OperatorGroup %s: %w", og.Name, err)
}
klog.Infof("Successfully created OperatorGroup %s", og.Name)
return nil
}
var lastErr error
err := wait.PollUntilContextTimeout(ctx, 5*time.Second, 20*time.Second, true, func(ctx context.Context) (bool, error) {
_, createErr := dynamicClient.Resource(GetOperatorGroupGVR()).Namespace(og.Namespace).Create(ctx, operatorGroup, metav1.CreateOptions{})
if createErr != nil && !apierrors.IsAlreadyExists(createErr) {
lastErr = createErr
klog.Warningf("Failed to create OperatorGroup, retrying: %v", createErr)
return false, nil
}
return true, nil
})
if err != nil && lastErr != nil {
err = lastErr
}
if err != nil {
return fmt.Errorf("failed to create OperatorGroup %s: %w", og.Name, err)
}
klog.Infof("Successfully created OperatorGroup %s", og.Name)
return nil
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/library/workloads/olm.go` around lines 100 - 115, The retry helpers for
OperatorGroup and Subscription creation/deletion are losing the real API error
and are not idempotent. Update the polling logic in the create/delete helpers
(including CreateSubscription and the delete counterparts) to preserve and
return the last error from the closure instead of returning false,nil on
failures. Use apierrors.IsAlreadyExists to treat pre-existing creates as success
and apierrors.IsNotFound to treat missing deletes as success, so
PollUntilContextTimeout does not mask the underlying failure and the helpers
become idempotent.

Comment thread test/library/workloads/olm.go Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/library/workloads/olm.go`:
- Around line 285-288: The CSV phase polling logic currently treats all
NestedString errors from status.phase the same as “not ready,” which causes
malformed CSV objects to keep retrying until timeout. Update the polling
callback around unstructured.NestedString in the CSV phase check to distinguish
a type/schema error from a missing field: return the error immediately when err
is non-nil, but keep returning false,nil only when phase is simply not found.
Use the existing csvName logging and the surrounding polling function to keep
the behavior consistent while making malformed CSV phase failures fail fast.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b22242e3-a2f1-4d5f-b57b-d84fcc2df547

📥 Commits

Reviewing files that changed from the base of the PR and between 2cc8ed2 and 4b084c4.

📒 Files selected for processing (2)
  • test/library/workloads/node.go
  • test/library/workloads/olm.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/library/workloads/node.go

Comment thread test/library/olm/olm.go Outdated
Comment on lines +285 to +288
phase, found, err := unstructured.NestedString(csv.Object, "status", "phase")
if err != nil || !found {
klog.Warningf("CSV %s has no phase yet", csvName)
return false, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Return malformed CSV phase errors instead of polling until timeout.

Line 285 can return an error when status.phase exists with the wrong type, but Lines 286-288 treat it like a missing phase and retry until the 3-minute timeout. Split the cases so schema/type errors fail immediately while absent phase keeps polling.

Proposed fix
 		// Get the phase from status
 		phase, found, err := unstructured.NestedString(csv.Object, "status", "phase")
-		if err != nil || !found {
+		if err != nil {
+			return false, fmt.Errorf("failed to read CSV %s phase: %w", csvName, err)
+		}
+		if !found {
 			klog.Warningf("CSV %s has no phase yet", csvName)
 			return false, nil
 		}

As per path instructions, “Never ignore error returns”.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
phase, found, err := unstructured.NestedString(csv.Object, "status", "phase")
if err != nil || !found {
klog.Warningf("CSV %s has no phase yet", csvName)
return false, nil
phase, found, err := unstructured.NestedString(csv.Object, "status", "phase")
if err != nil {
return false, fmt.Errorf("failed to read CSV %s phase: %w", csvName, err)
}
if !found {
klog.Warningf("CSV %s has no phase yet", csvName)
return false, nil
}
🧰 Tools
🪛 golangci-lint (2.12.2)

[error] 288-288: error is not nil (line 285) but it returns nil

(nilerr)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/library/workloads/olm.go` around lines 285 - 288, The CSV phase polling
logic currently treats all NestedString errors from status.phase the same as
“not ready,” which causes malformed CSV objects to keep retrying until timeout.
Update the polling callback around unstructured.NestedString in the CSV phase
check to distinguish a type/schema error from a missing field: return the error
immediately when err is non-nil, but keep returning false,nil only when phase is
simply not found. Use the existing csvName logging and the surrounding polling
function to keep the behavior consistent while making malformed CSV phase
failures fail fast.

Sources: Path instructions, Linters/SAST tools

Comment thread test/library/workloads/node.go Outdated
@@ -0,0 +1,63 @@
package workloads

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

workloads is too generic. Better to have each set of helpers under its own name. E.g. test/library/olm or similar.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it like
test/library/olm/olm.go
test/library/node/node.go ?

Comment thread test/library/workloads/olm.go Outdated
}

// GetOperatorGroupGVR returns the GroupVersionResource for OperatorGroup
func GetOperatorGroupGVR() schema.GroupVersionResource {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetOperatorGroupGVR -> OperatorGroupGVR as there's really no "Get" operation performed. The same for other GetXXX functions.

Comment thread test/library/workloads/olm.go Outdated
}

// SkipMissingCatalogSources checks if required catalog sources are available
func (sub *Subscription) SkipMissingCatalogSources(ctx context.Context, dynamicClient dynamic.Interface) error {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where the "Skip" keyword comes from? The method checks for an existence, it does not skip.

Comment thread test/library/olm/olm.go
Comment thread test/library/workloads/olm.go Outdated
continue
}
// Use k8s built-in helpers instead of manual map access
name, _, _ := unstructured.NestedString(imgMap, "name")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if the field does not exists or an error is returned? The same for the other field.

Comment thread test/library/workloads/node.go Outdated
}

node.Spec.Unschedulable = true
_, err = client.CoreV1().Nodes().Update(ctx, node, metav1.UpdateOptions{})

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about Patch instead of Update?

Comment thread test/library/workloads/node.go Outdated
}

node.Spec.Unschedulable = false
_, err = client.CoreV1().Nodes().Update(ctx, node, metav1.UpdateOptions{})

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Comment thread test/library/workloads/node.go Outdated
}

if node.Spec.Unschedulable {
klog.Infof("Node %s is already cordoned", nodeName)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is logging this information useful?

Comment thread test/library/workloads/node.go Outdated
return err
}

klog.Infof("Successfully cordoned node %s", nodeName)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is logging this information useful? Instead, the invoker might check for err == nil and log this if needed.

Comment thread test/library/workloads/olm.go Outdated
)

// OperatorGroup represents an OLM OperatorGroup resource configuration
type OperatorGroup struct {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: Using github.com/operator-framework/api/pkg/operators/v1 OperatorGroup

This dependency is currently unavailable in the project. Tests confirm it's not present and would require adding a new dependency:

  # Check if dependency exists
  $ grep "operator-framework/api" go.mod go.sum
  # (no output)

  $ go mod why github.com/operator-framework/api
  (main module does not need package github.com/operator-framework/api)

  $ go list -m github.com/operator-framework/api
  go: module github.com/operator-framework/api: not a known dependency

  $ find vendor -type d -name "operator-framework"
  # (no output)

Using the official v1.OperatorGroup type would require:

  • go get github.com/operator-framework/api@
  • go mod tidy && go mod vendor

Are we okay to import dependencies in test directory ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is ok. The dependency will be vendored only when test/library/olm is imported.

Comment thread test/library/olm/olm.go Outdated
func (og *OperatorGroup) CreateOperatorGroup(ctx context.Context, dynamicClient dynamic.Interface) error {
klog.Infof("Creating OperatorGroup %s in namespace %s", og.Name, og.Namespace)

operatorGroup := &unstructured.Unstructured{

@ingvagabund ingvagabund Jun 28, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not import https://github.com/operator-framework/api/blob/492d6ba7263f792b5a3dcf24569fcd4f7d1c498d/pkg/operators/v1/operatorgroup_types.go#L118 type directly and populate it? There might even be the corresponding clients available? The same for other places.

There's a chance most of the code here can be replaced by the github.com/operator-framework/*** code.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as here

Comment thread test/library/workloads/node.go Outdated
// CordonNode marks a node as unschedulable, preventing new pods from being scheduled on it.
// This is commonly used during maintenance operations or when testing pod eviction scenarios.
// Uses retry.RetryOnConflict to handle concurrent node updates by kubelet/controllers.
func CordonNode(ctx context.Context, client kubernetes.Interface, nodeName string) error {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would need adding new dependencies.
k8s.io/kubectl/pkg/drain

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, you should be able to use CordonHelper directly in the KDO repository. Without creating any helpers here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Available k8s.io packages in kdo vendor:

  • k8s.io/api
  • k8s.io/client-go
  • k8s.io/kubernetes
    Unavailable:
  • k8s.io/kubectl
    So, CordonHelper usage will cause new dependency to be downloaded in kdo repo.
$ go list -m k8s.io/kubectl
go: module k8s.io/kubectl: can't resolve module using the vendor directory
	(Use -mod=mod or -mod=readonly to bypass.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ingvagabund ,
Currently in kdo repo, k8s.io/kubectl is not used anywhere.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is your concern?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is my current concern.
#2336 (comment)
If I need to use CordonHelper directly in kdo, we need import the dependency k8s.io/kubectl. Are we good to go with the import of this dep ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's ok to vendor the extra dependency.

@sandeepknd
sandeepknd force-pushed the ote-kdso-lib branch 2 times, most recently from 1ff25c4 to db795b4 Compare June 29, 2026 05:47

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/library/workloads/olm.go`:
- Around line 303-307: WaitForCSVSucceeded currently retries on every CSV Get
error and hides non-transient failures behind a timeout. In the callback inside
WaitForCSVSucceeded, keep retrying only for expected NotFound cases from
dynamicClient.Resource(CSVGVR()).Namespace(namespace).Get, but return the actual
error immediately for RBAC, GVR, transport, or other non-NotFound failures so
PollUntilContextTimeout can fail fast and preserve the real cause.
- Around line 343-390: The PackageManifest fallback logic in the
subscription-building path is too permissive and can synthesize mismatched
source/channel/CSV values. In the code that reads catalogSource, defaultChannel,
and channels/currentCSV, remove the hardcoded defaults and first-channel
fallback; instead, require a valid default channel and its currentCSV to be
present and return an error from this workflow when they cannot be resolved.
Keep the validation and extraction localized around the
unstructured.NestedString/NestedSlice handling so the resulting Subscription is
only created from a coherent manifest tuple.
- Around line 23-29: The Subscription helper contract is missing the catalog
source namespace, causing CreateSubscription and VerifyCatalogSourceExists to
assume openshift-marketplace for every package. Add a SourceNamespace field to
Subscription, populate it in GetPackageManifest from
status.catalogSourceNamespace, and update CreateSubscription and
VerifyCatalogSourceExists to use Subscription.SourceNamespace instead of the
hard-coded namespace so packages from other catalog namespaces are handled
correctly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cb883a7c-2016-43d8-b274-afd65adf6047

📥 Commits

Reviewing files that changed from the base of the PR and between 4b084c4 and 1ff25c4.

📒 Files selected for processing (2)
  • test/library/workloads/node.go
  • test/library/workloads/olm.go

Comment thread test/library/workloads/olm.go Outdated
Comment thread test/library/workloads/olm.go Outdated
Comment thread test/library/workloads/olm.go Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (4)
test/library/workloads/olm.go (4)

387-389: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Don’t swallow malformed channel entries.

If status.channels[*].name has the wrong type, this continue degrades into a misleading “default channel not found” later. Return the parsing error instead of treating it as a non-match.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/library/workloads/olm.go` around lines 387 - 389, In the channel parsing
logic around unstructured.NestedString in the workload test helper, do not treat
a type error on status.channels[*].name as a simple non-match. Update the code
path in the channel iteration to distinguish an error from a missing field and
return the parsing error immediately instead of continuing, so malformed channel
entries are surfaced instead of later appearing as “default channel not found.”

Source: Path instructions


316-319: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Fail fast on malformed status.phase.

A NestedString type/schema error is still treated like “phase not ready”, so bad CSV objects poll for three minutes and lose the real cause.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/library/workloads/olm.go` around lines 316 - 319, The CSV phase check in
the workload polling logic is swallowing schema/type errors from NestedString
and treating them like a missing phase, which delays the real failure. Update
the CSV status handling in the phase-check path to distinguish a malformed
status.phase from an unset phase: when unstructured.NestedString returns an
error, fail fast and return that error instead of logging “no phase yet” and
continuing. Keep the existing “not found yet” retry behavior only for the
found=false case in the CSV polling code.

Sources: Path instructions, Linters/SAST tools


234-242: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Don’t pick an arbitrary CSV when the selector matches multiple objects.

Using csvList.Items[0] makes downstream image/phase helpers depend on list ordering rather than a stable rule. Fail here or apply an explicit selection criterion.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/library/workloads/olm.go` around lines 234 - 242, The CSV selection in
the OLM workload helper is currently arbitrary because `csvList.Items[0]` is
used after `csvList.Items` may contain multiple matches. Update the `csvName`
selection logic in `test/library/workloads/olm.go` to avoid depending on list
order: either fail fast when more than one CSV matches, or add a deterministic
selection rule before `klog.Infof("Using CSV: %s", csvName)`. Keep the behavior
localized around the `csvList.Items` handling and the
`klog.Warningf`/`klog.Infof` logging block.

102-109: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Preserve API failures and make the helpers idempotent.

These poll callbacks still convert every create/delete error into false, nil, so AlreadyExists/NotFound cases retry until timeout and real RBAC/GVR/transport failures get masked as a generic poll timeout.

Also applies to: 123-130, 169-176, 190-197

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/library/workloads/olm.go` around lines 102 - 109, The polling helpers in
OLM workload setup are swallowing create/delete errors by returning false, nil
from the callback, which hides real failures and makes AlreadyExists/NotFound
cases retry until timeout. Update the callbacks in the OperatorGroup and related
helper logic to treat idempotent API responses as success while propagating
unexpected errors from dynamicClient.Resource(...).Create/Delete instead of
converting them to generic poll retries. Use the existing helper functions
around the create/delete loops in olm.go to keep the behavior consistent across
the affected blocks.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/library/workloads/olm.go`:
- Around line 123-130: The deletion polling in the OperatorGroup cleanup logic
returns success as soon as Delete is accepted, which can race later
read/recreate steps. Update the wait.PollUntilContextTimeout loop in the
OperatorGroup removal flow to first issue the Delete and then poll using Get on
the same OperatorGroupGVR()/Namespace object until apierrors.IsNotFound is
returned. Apply the same change to the other matching cleanup block referenced
by the comment, keeping the retry logging around transient delete/Get errors.

---

Duplicate comments:
In `@test/library/workloads/olm.go`:
- Around line 387-389: In the channel parsing logic around
unstructured.NestedString in the workload test helper, do not treat a type error
on status.channels[*].name as a simple non-match. Update the code path in the
channel iteration to distinguish an error from a missing field and return the
parsing error immediately instead of continuing, so malformed channel entries
are surfaced instead of later appearing as “default channel not found.”
- Around line 316-319: The CSV phase check in the workload polling logic is
swallowing schema/type errors from NestedString and treating them like a missing
phase, which delays the real failure. Update the CSV status handling in the
phase-check path to distinguish a malformed status.phase from an unset phase:
when unstructured.NestedString returns an error, fail fast and return that error
instead of logging “no phase yet” and continuing. Keep the existing “not found
yet” retry behavior only for the found=false case in the CSV polling code.
- Around line 234-242: The CSV selection in the OLM workload helper is currently
arbitrary because `csvList.Items[0]` is used after `csvList.Items` may contain
multiple matches. Update the `csvName` selection logic in
`test/library/workloads/olm.go` to avoid depending on list order: either fail
fast when more than one CSV matches, or add a deterministic selection rule
before `klog.Infof("Using CSV: %s", csvName)`. Keep the behavior localized
around the `csvList.Items` handling and the `klog.Warningf`/`klog.Infof` logging
block.
- Around line 102-109: The polling helpers in OLM workload setup are swallowing
create/delete errors by returning false, nil from the callback, which hides real
failures and makes AlreadyExists/NotFound cases retry until timeout. Update the
callbacks in the OperatorGroup and related helper logic to treat idempotent API
responses as success while propagating unexpected errors from
dynamicClient.Resource(...).Create/Delete instead of converting them to generic
poll retries. Use the existing helper functions around the create/delete loops
in olm.go to keep the behavior consistent across the affected blocks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: eca8c9f5-1964-4c34-b117-cfc6783a0758

📥 Commits

Reviewing files that changed from the base of the PR and between db795b4 and a774a36.

📒 Files selected for processing (2)
  • test/library/workloads/node.go
  • test/library/workloads/olm.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/library/workloads/node.go

Comment thread test/library/olm/olm.go Outdated
@sandeepknd sandeepknd changed the title CNTRLPLANE-2654: added the library functions for workloads [WIP]CNTRLPLANE-2654: added the library functions for workloads Jul 3, 2026
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 3, 2026
@sandeepknd
sandeepknd force-pushed the ote-kdso-lib branch 4 times, most recently from 52ec140 to 89c24b3 Compare July 3, 2026 13:09
@sandeepknd sandeepknd changed the title [WIP]CNTRLPLANE-2654: added the library functions for workloads CNTRLPLANE-2654: added the library functions for workloads Jul 3, 2026
@sandeepknd sandeepknd changed the title CNTRLPLANE-2654: added the library functions for workloads [WIP] CNTRLPLANE-2654: added the library functions for workloads Jul 9, 2026
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 9, 2026
@sandeepknd
sandeepknd requested a review from ingvagabund July 9, 2026 08:09
@sandeepknd sandeepknd changed the title [WIP] CNTRLPLANE-2654: added the library functions for workloads CNTRLPLANE-2654: added the library functions for workloads Jul 9, 2026
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 9, 2026
Comment thread test/library/olm/olm.go
// PackageManifestGVR returns the GroupVersionResource for PackageManifest
func PackageManifestGVR() schema.GroupVersionResource {
return schema.GroupVersionResource{
Group: "packages.operators.coreos.com",

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason behind the hard-coded Group is that PackageManifest belongs to a different API group (packages.operators.coreos.com) that doesn't have constants defined in the operator-framework/api package we're using.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It introduces dependency conflicts and once again updates the go version to Go 1.26.x that creates the same issue which we encountered in "make verify:.

Comment thread test/library/olm/olm.go Outdated

// GetCSVName gets the CSV name for the operator using label selector.
// When multiple CSVs are found, it returns the one with the highest semver version.
func GetCSVName(ctx context.Context, dynamicClient dynamic.Interface, namespace, labelSelector string) (string, error) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the function could be invoked to get the latest CSV it's better to return the whole CSV object.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/GetCSVName/GetTheLatestCSVName

Comment thread test/library/olm/olm.go Outdated

// BuildSubscriptionFromPackageManifest fetches a packagemanifest for a given package
// and builds a Subscription object populated with the default channel, catalog source, and starting CSV information.
func BuildSubscriptionFromPackageManifest(ctx context.Context, dynamicClient dynamic.Interface, packageName, namespace string) (*operatorsv1alpha1.Subscription, error) {

@ingvagabund ingvagabund Jul 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function will get more usable if you pass the whole package manifest object. The type is defined at https://github.com/operator-framework/operator-lifecycle-manager/blob/master/pkg/package-server/apis/operators/v1/packagemanifest_types.go#L22.

@sandeepknd sandeepknd Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same issue is encountered here on importing the types from
packagesv1 "github.com/operator-framework/operator-lifecycle-manager/pkg/package-server/apis/operators/v1"

go mod tidy 2>&1 | head -50)
  ⎿  go: finding module for package github.com/operator-framework/operator-lifecycle-manager/pkg/package-server/apis/operators/v1
     go: toolchain upgrade needed to resolve github.com/operator-framework/operator-lifecycle-manager/pkg/package-server/apis/operators/v1
     go: github.com/operator-framework/operator-lifecycle-manager@v0.45.0 requires go >= 1.26.3; switching to go1.26.5

which will cause

make verify

to fail

Comment thread test/library/olm/olm.go Outdated
}

// GetCSVRelatedImages gets the relatedImages from a CSV
func GetCSVRelatedImages(ctx context.Context, dynamicClient dynamic.Interface, namespace, csvName string) ([]RelatedImage, error) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto. More useful to pass the whole csv object

Comment thread test/library/olm/olm.go Outdated
}

// DeleteSubscription deletes the Subscription
func DeleteSubscription(ctx context.Context, dynamicClient dynamic.Interface, sub *operatorsv1alpha1.Subscription) error {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is deleting a subscription waiting for something? Or, how often does it happen the deletion errors?

Comment thread test/library/olm/olm.go Outdated
}

// DeleteOperatorGroup deletes the OperatorGroup
func DeleteOperatorGroup(ctx context.Context, dynamicClient dynamic.Interface, og *operatorsv1.OperatorGroup) error {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is deleting a group waiting for something? Or, how often does it happen the deletion errors?

@ingvagabund ingvagabund left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the helpers receives either a name or an object and then convert it into unstructured object via runtime.DefaultUnstructuredConverter.ToUnstructured it's worth considering to pass only unstructured objects and perform the conversion in individual operator repositories. With a detailed comment about the intention. This way the OLM specific types vendoring can be moved to the operator repositories.

The current suggestion:

  • updating the helper signatures to accept and return unstructured objects (with proper comment)
  • vendoring the code changes into the KDO repository
  • (de)converting objects

Comment thread test/library/olm/olm.go
// PackageManifestGVR returns the GroupVersionResource for PackageManifest
func PackageManifestGVR() schema.GroupVersionResource {
return schema.GroupVersionResource{
Group: "packages.operators.coreos.com",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sandeepknd

Copy link
Copy Markdown
Contributor Author

Given the helpers receives either a name or an object and then convert it into unstructured object via runtime.DefaultUnstructuredConverter.ToUnstructured it's worth considering to pass only unstructured objects and perform the conversion in individual operator repositories. With a detailed comment about the intention. This way the OLM specific types vendoring can be moved to the operator repositories.

The current suggestion:

  • updating the helper signatures to accept and return unstructured objects (with proper comment)
  • vendoring the code changes into the KDO repository
  • (de)converting objects

Does it mean reverting to the original changes without any of these imports

 -  "github.com/operator-framework/api/pkg/operators"                                                                                                                          
     operatorsv1 "github.com/operator-framework/api/pkg/operators/v1"                                                                                                           
     operatorsv1alpha1 "github.com/operator-framework/api/pkg/operators/v1alpha1" 

?

@sandeepknd
sandeepknd requested a review from ingvagabund July 9, 2026 12:09
@sandeepknd
sandeepknd force-pushed the ote-kdso-lib branch 5 times, most recently from 3bd0318 to 8cbfad7 Compare July 10, 2026 18:38
@sandeepknd

sandeepknd commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

Given the helpers receives either a name or an object and then convert it into unstructured object via runtime.DefaultUnstructuredConverter.ToUnstructured it's worth considering to pass only unstructured objects and perform the conversion in individual operator repositories. With a detailed comment about the intention. This way the OLM specific types vendoring can be moved to the operator repositories.

The current suggestion:

  • updating the helper signatures to accept and return unstructured objects (with proper comment)
  • vendoring the code changes into the KDO repository
  • (de)converting objects

addressed.
No vendor dependencies needed now, which is the major advantage of leveraging the unstructured k8s client object. I think this is best approach here as it avoids unnecessary dep vendor-ing in library-go which is used in many other core repos.

@openshift-ci

openshift-ci Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

@sandeepknd: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@sandeepknd

Copy link
Copy Markdown
Contributor Author

Hi @ingvagabund ,
Could you PTAL.

1 similar comment
@sandeepknd

Copy link
Copy Markdown
Contributor Author

Hi @ingvagabund ,
Could you PTAL.

@mrda

mrda commented Jul 29, 2026

Copy link
Copy Markdown

Hi @ingvagabund, can you take a look at this PR?

@ropatil010

Copy link
Copy Markdown
Contributor

/lgtm

@ropatil010

Copy link
Copy Markdown
Contributor

Relatable PR: https://github.com/openshift/cluster-kube-descheduler-operator/blob/main/go.mod#L71 got merged.
/assign @p0lyn0mial
PTAL look on this PR and help to merge

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Aug 5, 2026
@openshift-ci

openshift-ci Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ropatil010, sandeepknd
Once this PR has been reviewed and has the lgtm label, please assign p0lyn0mial for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants