docs: file two k3s-cluster bugs found during worker recovery#114
Merged
Conversation
Recovering a k3s worker whose VM was deleted out-of-band surfaced two real reconciliation gaps: 1. Cluster apply does not detect an out-of-band child-VM deletion: it reports Ready from its persisted children list without verifying the VMs exist, and the verifying-trace cache short-circuits re-applies (input+refs hashes unchanged -> cache hit -> no reconcile). Recovery needed a manual applied_manifests cache clear. 2. Plan-based count-up breaks on legacy (pre-K3sNode) clusters: existing- cluster converge now goes solely through the Plan path, which emits worker K3sNode children with $ref joins to K3sNode/<cp>. A cluster created via the old inline path has no K3sNode resources for its CPs, so "resolve refs: K3sNode dev-cp-0 not found" -- the worker VM is created but never joins. Both filed as ROADMAP follow-ups with suggested fixes. Docs-only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recovering a k3s worker whose VM had been deleted out-of-band surfaced two genuine reconciliation gaps. Filed as ROADMAP follow-ups with suggested fixes.
1. Cluster apply does not detect an out-of-band child-VM deletion
Cluster
Get/apply reportsReadystraight from its persistedchildrenlist without verifying the child VMs actually exist on the provider — so a deleted worker is not seen as drift. The verifying-trace cache compounds it: with the manifest unchanged,input+refshashes match and the apply is a cache hit that returnsReadywithout reconciling (recovery required manually clearing theapplied_manifestsrow).2. Plan-based count-up breaks on legacy (pre-
K3sNode) clustersExisting-cluster converge now goes solely through the Plan/dispatcher path, which emits worker
K3sNodechildren with$refjoins toK3sNode/<cp>. A cluster created via the old inline path has noK3sNoderesources for its CPs, so ref resolution fails (K3sNode "dev-cp-0" not found) — the worker VM is created but never joins.Docs-only.