Skip to content

fix(eks): repair the EKS create flow — pinned modules, working CNI/CSI, honest errors and status - #289

Merged
oleg-flamingocx merged 9 commits into
mainfrom
fix/eks-report-findings
Aug 6, 2026
Merged

fix(eks): repair the EKS create flow — pinned modules, working CNI/CSI, honest errors and status#289
oleg-flamingocx merged 9 commits into
mainfrom
fix/eks-report-findings

Conversation

@oleg-flamingocx

@oleg-flamingocx oleg-flamingocx commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary by CodeRabbit

  • New Features

    • Cluster creation now verifies node readiness and default storage availability, clearly reporting healthy, incomplete, or unverifiable results.
    • Installations validate Git branches and tags before deployment, with clearer guidance when references are unavailable.
    • EKS clusters now include required core add-ons, default EBS-backed storage, pinned infrastructure versions, and updated defaults.
  • Bug Fixes

    • Status, access, and uninstall commands now require cluster targeting through --context.
    • Resumed cluster operations correctly refresh status and timestamps.
  • Documentation

    • Updated installation examples, release references, and cloud-cluster guidance.

…surface real terraform errors

The root cause of three broken-create failure modes was one floating module
range: '~> 21.0' resolved to v21.24.1, whose changed defaults broke create
in three independent ways.

Template:
- pin terraform-aws-modules/eks to 21.24.1 and /vpc to 6.6.1 exactly; guard
  test rejects ranges
- default kubernetes_version to a concrete 1.33 — module v21 cannot plan with
  a null version (Invalid count argument on the documented no-flag create)
- declare vpc-cni (before_compute), kube-proxy, coredns: module v21 sets
  bootstrap_self_managed_addons=false, so an undeclared CNI meant permanently
  NotReady nodes
- set IMDS hop limit 2 on the node group: the v21 default of 1 cut the
  (non-hostNetwork) EBS CSI controller off from the node role
- enable the CSI addon's defaultStorageClass: since EKS 1.30 AWS ships no
  default class, so every PVC stayed Pending
- default instance type m7i-flex.large (Free-Tier-eligible drop-in for
  m6i.large): new AWS accounts refuse non-eligible types after a ~30-minute
  opaque ASG wait

Error surfacing:
- splitCause kept only the deepest error ('exit status 1') and discarded
  the terraform stderr appended after it — the actual failure reason now
  reaches the default, non-verbose panel
- verbose mode no longer tees terraform's machine-readable output (the
  641 KB 'show -json' line) into the terminal; init/plan human output still
  streams via a selective wrapper

Commands and docs:
- app status/access/uninstall now reject a stray positional cluster name
  instead of silently running against the current kube-context
- --ref examples match the chart repo's real tag scheme; machine-type
  defaults and the EKS out-of-the-box description updated
A typo'd --ref used to surface only at clone time — AFTER certificates were
refreshed, the argo-cd helm release deployed and the API port awaited —
leaving the cluster mutated with ArgoCD and no applications.

- git.Repository.ValidateRef: one ls-remote round-trip (go-git remote list,
  same transport and auth as the clone) checks the ref exists as a branch or
  tag before anything touches the cluster
- Installer runs the preflight first when app-of-apps is configured; the
  production validator is the git repository provider, injected through a
  ChartService seam so orchestration tests stay off the network
- BranchNotFoundError now carries the refs the repository DOES offer; the
  error panel lists branches in full and the version-sorted top tags
  (1.0.9 < 1.0.48 — plain string sort would invert them), capped so hundreds
  of release tags cannot bury the branches
… list drifting from reality

The creation box printed a hardcoded green 'Ready' and the configured node
count on the provisioner's exit code alone; it would render identically over
NotReady nodes or a cluster that cannot bind a single PVC. The box now
verifies via client-go (brief poll, 45s cap): every node Ready and a default
StorageClass present render 'Ready' with the observed fraction; anything else
renders 'Provisioned — n/m nodes Ready' (title downgraded to ⚠️) with a
warning naming the specific gap; an unreachable API says 'health not
verified' instead of guessing.

The list's CONTEXT column was fabricated from the cluster name even when no
kubeconfig entry existed (plan-stage failure). It is now derived from the
kubeconfig via tfengine.KubeconfigHasContext. A resumed create also flips
the record to 'creating' so list stops reporting the previous attempt's
'Failed' while an apply is running.

On the transition to Ready the record's CreatedAt is stamped fresh, so a
resumed create no longer reports the first failed attempt's timestamp
forever.
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@oleg-flamingocx, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 47 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d8de1a41-14e0-4e7a-8c94-902196015ac0

📥 Commits

Reviewing files that changed from the base of the PR and between 170dfab and a4300fb.

📒 Files selected for processing (2)
  • internal/cluster/providers/gke/provider.go
  • internal/cluster/providers/gke/provider_test.go
📝 Walkthrough

Walkthrough

Changes

Cluster provisioning and CLI behavior

Layer / File(s) Summary
CLI argument and release reference contracts
cmd/app/*, docs/reference/architecture/overview.md
Status, access, and uninstall reject positional arguments. Install examples use release reference 1.0.48.
Git reference validation before installation
internal/chart/providers/git/*, internal/chart/services/*, internal/chart/utils/types/interfaces.go, internal/shared/errors/*
Installation validates branches and tags before cluster changes and reports available references for missing refs.
Cluster lifecycle and health verification
internal/cluster/service.go, internal/cluster/health_test.go, internal/cluster/providers/{eks,gke}/*, internal/cluster/providers/terraform/kubecontext*
Creation summaries verify node readiness and StorageClass state. Resumed creates reset status and refresh timestamps.
EKS template defaults and addon configuration
internal/cluster/providers/eks/templates/main.tf, internal/cluster/providers/eks/template*, internal/cluster/models/flags.go, internal/cluster/ui/wizard.go, docs/getting-started/cloud-clusters.md
EKS uses pinned Terraform modules, concrete defaults, IMDSv2, and explicit core and storage addon configuration.
Terraform output and kubeconfig context handling
internal/cluster/providers/terraform/engine.go, internal/cluster/providers/terraform/kubecontext.go, go.mod
Verbose Terraform mode scopes stdout to Init and Plan. Kubeconfig helpers detect existing contexts deterministically. The Go toolchain and two indirect dependencies are updated.

Estimated code review effort: 5 (Critical) | ~90 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant Installer
  participant GitRepository
  participant Cluster
  User->>Installer: Start app installation
  Installer->>GitRepository: Validate configured branch or tag
  GitRepository-->>Installer: Return validation result
  Installer->>Cluster: Install only when the reference is valid
Loading
sequenceDiagram
  participant ClusterService
  participant KubernetesAPI
  participant ClusterSummary
  ClusterService->>KubernetesAPI: Check nodes and default StorageClass
  KubernetesAPI-->>ClusterService: Return observed cluster health
  ClusterService->>ClusterSummary: Render verified status and warnings
Loading

Possibly related PRs

Suggested reviewers: ivan-flamingo

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 76.36% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main EKS provisioning fixes, including pinned modules, addon configuration, error reporting, and status handling.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/eks-report-findings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cmd/app/app.go`:
- Around line 15-20: Update the app help example near noPositionalArgs to
replace the invalid positional-cluster form of the install command with the
--context option, while preserving the example’s intended targeting behavior.

In `@internal/cluster/providers/eks/template_guard_test.go`:
- Around line 55-60: Update the guard test around the add-on assertions to match
each add-on map assignment rather than merely searching the full template
source, so comments cannot satisfy the check. Require before_compute = true
specifically within the vpc-cni entry, while preserving validation that vpc-cni,
kube-proxy, and coredns are declared.

In `@internal/cluster/providers/terraform/engine_test.go`:
- Around line 241-244: Update the fixture setup around FindTerraform in the test
to detect runtime.GOOS == "windows" and create an additional executable
terraform.exe in binDir, while preserving the existing terraform fixture and
permissions for all platforms.

In `@internal/cluster/providers/terraform/kubecontext.go`:
- Around line 21-22: Update infoFor to check kubeconfig contexts using the same
candidates as EKS discovery: try rec.Name first, then the cluster ARN. Set
Context to the first matching candidate instead of only returning whether
rec.Name exists.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ee445a6f-dd71-487c-8de3-16585aaf02d0

📥 Commits

Reviewing files that changed from the base of the PR and between 4b91628 and 0d2bce6.

📒 Files selected for processing (31)
  • cmd/app/.install.md
  • cmd/app/access.go
  • cmd/app/app.go
  • cmd/app/install.go
  • cmd/app/status.go
  • cmd/app/status_access_test.go
  • cmd/app/uninstall.go
  • docs/getting-started/cloud-clusters.md
  • docs/reference/architecture/overview.md
  • internal/chart/providers/git/repository.go
  • internal/chart/providers/git/repository_test.go
  • internal/chart/services/chart_service.go
  • internal/chart/services/install_orchestration_test.go
  • internal/chart/services/installer.go
  • internal/chart/utils/types/interfaces.go
  • internal/cluster/health_test.go
  • internal/cluster/models/flags.go
  • internal/cluster/providers/eks/provider.go
  • internal/cluster/providers/eks/provider_test.go
  • internal/cluster/providers/eks/template.go
  • internal/cluster/providers/eks/template_guard_test.go
  • internal/cluster/providers/eks/templates/main.tf
  • internal/cluster/providers/gke/provider.go
  • internal/cluster/providers/terraform/engine.go
  • internal/cluster/providers/terraform/engine_test.go
  • internal/cluster/providers/terraform/kubecontext.go
  • internal/cluster/providers/terraform/kubecontext_test.go
  • internal/cluster/service.go
  • internal/cluster/ui/wizard.go
  • internal/shared/errors/errors.go
  • internal/shared/errors/errors_test.go

Comment thread cmd/app/app.go
Comment thread internal/cluster/providers/eks/template_guard_test.go Outdated
Comment thread internal/cluster/providers/terraform/engine_test.go
Comment thread internal/cluster/providers/terraform/kubecontext.go Outdated
…, context matching

- engine.go: drop redundant .Terraform selectors on promoted methods
  (staticcheck QF1008); the Init/Plan overrides keep the explicit selector,
  which is what prevents self-recursion
- template_guard_test: bind the addon guard to the actual map entries via
  regex — the addon names also appear in comments, which could satisfy a
  plain Contains; before_compute is now required inside the vpc-cni block
  specifically
- engine_test: create a terraform.exe fixture on Windows — LookPath there
  resolves only PATHEXT extensions, so the bare stub is invisible
- cluster list context: beyond the plain cluster name, recognize the
  conventional context names (EKS: the ARN 'aws eks update-kubeconfig'
  writes, matched by prefix/suffix since the record has no account id;
  GKE: gcloud's gke_<project>_<location>_<name>) — the same candidate
  shapes discovery matches
ivan-flamingo
ivan-flamingo previously approved these changes Aug 6, 2026
The Windows unit-test leg dies at random with 'Exception 0xc0000005 /
unknown pc' — a Go 1.26.0/1.26.1 runtime bug corrupting return addresses on
the stack during GC (suspected Green Tea GC; golang/go#77975,
release-blocking, fix targeted at a patch release). The argocd test binary
happened to be the victim this run; any package can be next. Our code is not
implicated: the suite is race-clean under -race and the crash is inside the
runtime during test cleanup.

Disable the new GC via GOEXPERIMENT=nogreenteagc on the Windows leg only;
drop this once a fixed Go patch release is picked up.
…d in 1.26.2

golang/go#77975 (return addresses corrupted during GC stack scanning,
'Exception 0xc0000005 / unknown pc' at random on windows-amd64) was fixed in
Go 1.26.2 (backport golang/go#78041, closed). CI and goreleaser both resolve
the toolchain from go.mod via go-version-file, so the toolchain directive is
the single switch that moves unit tests, e2e AND the shipped Windows release
binaries off the buggy 1.26.0 runtime.

Drop the GOEXPERIMENT=nogreenteagc workaround from the Windows unit-test
leg — with a fixed toolchain it only kept a non-default GC configuration
alive in CI.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/cluster/providers/gke/provider.go`:
- Around line 504-506: Update the GKE context predicate passed to
tfengine.KubeconfigContextMatching so it also requires the location segment to
equal rec.Region or begin with rec.Region followed by a hyphen, while preserving
the existing project/name prefix and suffix checks. Add a regression case
covering same-project, same-name contexts in different regions and verify only
the recorded region matches.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a459a59c-b30b-4625-9dee-0e450449f1b9

📥 Commits

Reviewing files that changed from the base of the PR and between 0d2bce6 and 170dfab.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (10)
  • go.mod
  • internal/cluster/models/flags.go
  • internal/cluster/providers/eks/provider.go
  • internal/cluster/providers/eks/provider_test.go
  • internal/cluster/providers/eks/template_guard_test.go
  • internal/cluster/providers/gke/provider.go
  • internal/cluster/providers/gke/provider_test.go
  • internal/cluster/providers/terraform/engine.go
  • internal/cluster/providers/terraform/engine_test.go
  • internal/cluster/providers/terraform/kubecontext.go
🚧 Files skipped from review as they are similar to previous changes (5)
  • internal/cluster/models/flags.go
  • internal/cluster/providers/eks/provider.go
  • internal/cluster/providers/terraform/engine_test.go
  • internal/cluster/providers/eks/template_guard_test.go
  • internal/cluster/providers/terraform/engine.go

Comment thread internal/cluster/providers/gke/provider.go
The gke_<project>_<location>_<name> match checked only project and name: a
same-name, same-project cluster in ANOTHER region satisfied the prefix/suffix
too, and the lexically first kubeconfig context won regardless of the record.
The location segment must now be the recorded region itself (regional
cluster) or a zone inside it (region + "-" prefix, e.g. us-central1-a);
us-east10 does not false-match us-east1 since its next rune is not a hyphen.
@oleg-flamingocx
oleg-flamingocx enabled auto-merge (squash) August 6, 2026 09:02
@oleg-flamingocx
oleg-flamingocx merged commit 08dc432 into main Aug 6, 2026
11 of 13 checks passed
@oleg-flamingocx
oleg-flamingocx deleted the fix/eks-report-findings branch August 6, 2026 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants