chore(e2e): add Cloud SQL runtime E2E with sidecar

[zdrapela]

Description

Adds Helm and Operator runtime E2E coverage for Google Cloud SQL via the Cloud SQL Auth Proxy (native sidecar), matching the approach already used for RDS/Azure external DB suites.

Closes RHIDP-9140 and RHIDP-9141.

Why Auth Proxy (not direct type: cloudsql)

Backstage documents direct Cloud SQL via @google-cloud/cloud-sql-connector, but that package is only an optional peer of @backstage/backend-defaults and is not installed in shipped RHDH (unlike e.g. better-sqlite3). Helm/Operator docs today cover host/port TLS for RDS/Azure-style Postgres, not Cloud SQL connector config.

Product support for a direct Cloud SQL connection is tracked as RHDHPLAN-155 (“Support direct connection to Google Cloud SQL”). Until that lands (and related plugin gaps such as RBAC/Casbin adapters are addressed), automating the verified Auth Proxy path is the feasible E2E approach.

What this PR does

• New Playwright suite: verify-tls-config-with-external-cloudsql.spec.ts
• Cloud SQL helpers + RuntimeHarness APIs to inject the proxy sidecar (Helm JSON patch / Operator CR deployment.patch)
• Load CLOUDSQL_* + cloudsql-service-account.json from Vault in .ci/pipelines/env_variables.sh (no DB CA PEM)
• Unit coverage for Cloud SQL config helpers

Stacked on #5083 (fix/e2e-readiness-and-localization-gates).

Which issue(s) does this PR fix

• Closes RHIDP-9140 / RHIDP-9141 (Jira)
• Related product gap: RHDHPLAN-155

PR acceptance criteria

• GitHub Actions are completed and successful
• Unit Tests are updated and passing (yarn test:unit in e2e-tests)
• E2E Tests are updated and passing
• Documentation is updated if necessary (docs/e2e-tests/CI-medic-guide.md)
• Add a screenshot if the change is UX/UI related

How to test changes / Special notes to the reviewer

Vault keys (populated in CI): CLOUDSQL_USER, CLOUDSQL_PASSWORD, CLOUDSQL_INSTANCE_1..4 (project:region:instance), CLOUDSQL_1_HOST..4, cloudsql-service-account.json. DB cleanup uses public host TLS with rejectUnauthorized: false — do not use cloudsql-db-certificates.pem for now.

Local checks already run: yarn fmt, yarn lint, yarn test:unit under e2e-tests.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[github-actions]

The container image build workflow finished with status: cancelled.

[zdrapela]

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly

[openshift-ci]

@zdrapela: No presubmit jobs available for redhat-developer/rhdh@fix/e2e-readiness-and-localization-gates

Details

In response to this:

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[zdrapela]

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly

[zdrapela]

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.96%. Comparing base (d090bd1) to head (4ba9967).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5143      +/-   ##
==========================================
- Coverage   63.69%   59.96%   -3.74%     
==========================================
  Files         123      111      -12     
  Lines        2424     2198     -226     
  Branches      572      547      -25     
==========================================
- Hits         1544     1318     -226     
- Misses        878      879       +1     
+ Partials        2        1       -1     

Flag	Coverage Δ
rhdh	59.96% <ø> (-3.74%)	⬇️

Components	Coverage Δ
Backend plugins	∅ <ø> (∅)
Backend app	66.66% <ø> (ø)
Frontend app	58.89% <ø> (ø)
Plugin utils	∅ <ø> (∅)
Dynamic plugins utils	∅ <ø> (∅)

---

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d090bd1...4ba9967. Read the comment docs.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[zdrapela]

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly

[rhdh-qodo-merge]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Context used

⚠️ Tickets: not configured — ticket URL found in PR but could not be fetched — check ticket provider credentials

✅ Compliance rules (platform): 46 rules

✅ Skills: e2e-verify-fix, e2e-diagnose-and-fix

✅ Cross-repo context

  Explored: repo: redhat-developer/rhdh-chart (sha: e476f298)

  Explored: repo: redhat-developer/rhdh-operator (sha: cde4968e)

  Not relevant to this PR: redhat-developer/rhdh-plugins

  Not relevant to this PR: redhat-developer/rhdh-local

1. Chart Postgres secret mismatch ✓ Resolved 🔗 Cross-repo conflict ≡ Correctness

Description

The PR’s Helm Cloud SQL overlay sets upstream.postgresql.enabled=false but leaves the
chart-default POSTGRESQL_ADMIN_PASSWORD env var and default app-config DB password placeholder
intact, which still reference the <release>-postgresql Secret from rhdh-chart and can cause the
Helm-deployed pod to fail (missing Secret) when the subchart is disabled.

Code

e2e-tests/playwright/utils/cloudsql-config.ts[R167-173]

+  backstage.extraVolumes = upsertNamedRecord(
+    asNamedRecordList(backstage.extraVolumes),
+    buildCloudSqlProxyVolume(),
+  );
+  backstage.extraEnvVarsSecrets = ["postgres-cred"];
+  upstream.postgresql = { enabled: false };
+

Relevance

⭐⭐⭐ High

Team has accepted fixes preventing missing/incorrect Postgres secret refs in runtime env/config;
likely to avoid Helm pod failures.

PR-#5027
PR-#4288

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR code disables the chart’s Postgres subchart while inheriting chart-default extraEnvVars; the
pinned rhdh-chart defaults still render POSTGRESQL_ADMIN_PASSWORD via a SecretKeyRef whose name
resolves to <release>-postgresql by default, and the upstream chart template renders
extraEnvVars unconditionally, so the missing Secret can break pod startup.

e2e-tests/playwright/utils/cloudsql-config.ts[153-174]
e2e-tests/playwright/utils/runtime-config.ts[114-125]
External repo: redhat-developer/rhdh-chart, charts/backstage/values.yaml [207-227]
External repo: redhat-developer/rhdh-chart, charts/backstage/values.yaml [283-294]
External repo: redhat-developer/rhdh-chart, charts/backstage/templates/_helpers.tpl [25-36]
External repo: redhat-developer/rhdh-chart, charts/backstage/vendor/backstage/charts/backstage/templates/backstage-deployment.yaml [206-235]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`generateCloudSqlHelmValuesOverlay()` disables `upstream.postgresql.enabled`, but the pinned `rhdh-chart` still renders `upstream.backstage.extraEnvVars` (including `POSTGRESQL_ADMIN_PASSWORD` from a SecretKeyRef) and its default `upstream.backstage.appConfig.backend.database.connection.password: ${POSTGRESQL_ADMIN_PASSWORD}`. When the Postgres subchart is disabled, the referenced `<release>-postgresql` Secret is typically not created, so the workload can fail with missing Secret / CreateContainerConfigError.

## Issue Context
- The runtime Helm values generator intentionally *inherits* the chart defaults for `upstream.backstage.extraEnvVars`.
- The Cloud SQL Helm overlay currently only disables the Postgres subchart and adds `postgres-cred` via `extraEnvVarsSecrets`, but it does **not** override the chart’s default `POSTGRESQL_ADMIN_PASSWORD` env var entry (nor the default app-config DB password placeholder).

## Fix Focus Areas
- e2e-tests/playwright/utils/cloudsql-config.ts[153-175]
- e2e-tests/playwright/utils/runtime-config.ts[114-125]

### Suggested fix approach
In `generateCloudSqlHelmValuesOverlay()` explicitly override the relevant chart defaults when you set `upstream.postgresql.enabled=false`:
- Override `upstream.backstage.extraEnvVars` to remove the `POSTGRESQL_ADMIN_PASSWORD` entry (keep `BACKEND_SECRET` if needed).
- Override `upstream.backstage.appConfig.backend.database.connection` to use `${POSTGRES_HOST}`, `${POSTGRES_PORT}`, `${POSTGRES_USER}`, `${POSTGRES_PASSWORD}` (matching what `prepareForExternalDatabase()` patches to), so the Helm-managed ConfigMap stays consistent across `helm upgrade`.

This keeps Helm deployments compatible with the pinned `rhdh-chart` behavior while disabling the subchart.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Env-derived CloudSQL targets ✓ Resolved 🐞 Bug ≡ Correctness

Description

injectCloudSqlSidecar/configureCloudSqlProxyInstance accept namespace and releaseName, but derive
the Deployment name (and Helm upgrade config) from process.env via
getRhdhDeploymentName()/resolveConfig, so callers overriding releaseName can update one Backstage CR
while patching/restarting a different Deployment or Helm release.

Code

e2e-tests/playwright/utils/cloudsql-config.ts[R282-306]

+  const installMethod = resolveInstallMethod();
+  const deploymentName = getRhdhDeploymentName();
+
+  if (installMethod === "helm") {
+    const routerBase = process.env.K8S_CLUSTER_ROUTER_BASE ?? (await discoverRouterBase());
+    const config = resolveConfig(routerBase);
+    await upgradeRuntimeHelmRelease(config, generateCloudSqlHelmValuesOverlay());
+    await upsertCloudSqlProxyOnDeployment(
+      kubeClient,
+      namespace,
+      deploymentName,
+      instanceConnectionName,
+    );
+  } else {
+    const cr = await getRuntimeBackstageCr(kubeClient, namespace, releaseName);
+    applyCloudSqlToBackstageCr(cr, instanceConnectionName);
+    await replaceRuntimeBackstageCr(kubeClient, namespace, releaseName, cr);
+    await waitForProxyOnDeployment(kubeClient, namespace, deploymentName, instanceConnectionName);
+  }
+
+  // Rollout: native sidecar startupProbe must pass before backstage-backend is Ready.
+  await kubeClient.restartDeploymentWithRetry(deploymentName, namespace);
+}
+
+/**

Relevance

⭐⭐⭐ High

Correctness issue: accepted precedents prefer deriving K8s/Helm targets from explicit inputs vs
hardcoded/env-only names.

PR-#2939
PR-#4288

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

RuntimeHarness passes an explicit releaseName into Cloud SQL injection, but the injection code
ignores it (and the namespace for Helm upgrade selection) by re-deriving targets from environment
variables; those env-derived helpers explicitly read RELEASE_NAME/NAME_SPACE_RUNTIME.

e2e-tests/playwright/utils/cloudsql-config.ts[276-304]
e2e-tests/playwright/support/harnesses/runtime-harness.ts[100-125]
e2e-tests/playwright/utils/runtime-config.ts[83-95]
e2e-tests/playwright/utils/kube-client/helpers.ts[138-145]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`injectCloudSqlSidecar` (and similarly `configureCloudSqlProxyInstance`) accepts `namespace` and `releaseName`, but it still targets Helm upgrades and Deployment restarts using env-derived helpers (`resolveConfig`, `getRhdhDeploymentName`). If a caller passes a non-default `releaseName`/`namespace` (the `RuntimeHarness` constructor supports this), the code can mutate the CR for one release while restarting (or upgrading) another.

### Issue Context
The operator branch uses the `releaseName` argument for the Backstage CR read/replace, but the subsequent `deploymentName` used for waiting/restarting is computed from `process.env.RELEASE_NAME`. The helm branch uses `resolveConfig(routerBase)` (also env-derived) to select which release/namespace to upgrade.

### Fix Focus Areas
- e2e-tests/playwright/utils/cloudsql-config.ts[276-335]
- e2e-tests/playwright/utils/runtime-config.ts[83-109]
- e2e-tests/playwright/utils/kube-client/helpers.ts[138-145]

### Implementation notes
- Compute `deploymentName` from the *passed* `releaseName` + resolved install method (e.g., via `deploymentName(installMethod, releaseName)` from `instance-route-identity`) instead of `getRhdhDeploymentName()`.
- For Helm upgrades, avoid `resolveConfig(routerBase)` as the sole source of truth; instead construct a config using the passed `releaseName`/`namespace` (e.g., `const config = { ...resolveConfig(routerBase), releaseName, namespace }`) so the upgrade targets the same release the caller intends.
- Apply the same consistency in `configureCloudSqlProxyInstance` (restart/wait should target the same release that was patched).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[github-actions]

The container image build workflow finished with status: cancelled.

[zdrapela]

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly

[zdrapela]

/test e2e-ocp-helm-nightly

[zdrapela]

/test e2e-ocp-helm-nightly

[github-actions]

Image was built and published successfully. It is available at:

• quay.io/rhdh-community/rhdh:pr-5143
• quay.io/rhdh-community/rhdh:pr-5143-76f3ef96

[zdrapela]

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly

[github-actions]

Image was built and published successfully. It is available at:

• quay.io/rhdh-community/rhdh:pr-5143
• quay.io/rhdh-community/rhdh:pr-5143-53616492

[zdrapela]

/test e2e-ocp-helm-nightly

[github-actions]

Image was built and published successfully. It is available at:

• quay.io/rhdh-community/rhdh:pr-5143
• quay.io/rhdh-community/rhdh:pr-5143-d6cdc3fa

[zdrapela]

/test e2e-ocp-helm-nightly

[github-actions]

The container image build workflow finished with status: failure.

[zdrapela]

/test e2e-ocp-operator-nightly

[zdrapela]

/test e2e-ocp-helm-nightly
/test e2e-ocp-operator-nightly

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[github-actions]

Image was built and published successfully. It is available at:

• quay.io/rhdh-community/rhdh:pr-5143
• quay.io/rhdh-community/rhdh:pr-5143-4ba99675

[openshift-ci]

@zdrapela: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-ocp-helm-nightly	4ba9967	link	false	/test e2e-ocp-helm-nightly
ci/prow/e2e-ocp-operator-nightly	4ba9967	link	false	/test e2e-ocp-operator-nightly

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-ci]

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.