fix: write credentials atomically so concurrent refreshes cannot corrupt them - #11
Open
raykuo998 wants to merge 1 commit into
Open
fix: write credentials atomically so concurrent refreshes cannot corrupt them#11raykuo998 wants to merge 1 commit into
raykuo998 wants to merge 1 commit into
Conversation
…upt them SaveCredentials wrote credentials.json in place with os.WriteFile (O_TRUNC then write). With no cross-process lock, several ghealth invocations that refresh an expired token at the same time interleave: A opens and truncates, B opens and truncates, A writes its payload, B writes a shorter one over the front. The tail of A's write survives past the end of B's JSON and the file is permanently unparseable, after which every invocation reports authentication failed: not authenticated: stored credentials: no stored credentials: invalid character '}' after top-level value even though a valid refresh token is still in the file. Recovery needs hand-editing or a fresh `ghealth auth login`. Write to a temp file in the same directory, fsync, then rename over the target so a writer is all-or-nothing and a reader never sees a partial file. The rename needs retrying on Windows: MoveFileEx returns ERROR_ACCESS_DENIED both when a reader holds the destination open and when two replaces of the same path overlap (8 goroutines x 40 renames onto one path: 67/320 failed, no reader present). Retrying is safe because a losing attempt leaves the previous file intact, so the worst case is the already-handled "could not persist" warning rather than a corrupt file. SavePendingAuth gets the same treatment; it has the same shape.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
Author
|
I've signed the CLA (individual, same address as the commit author). Could the check be re-run? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #10.
Problem
SaveCredentialswrotecredentials.jsonin place withos.WriteFile(O_TRUNC, then write). With no cross-process lock, severalghealthinvocations that refresh an expired token at the same time interleave:B only overwrites the front, and the tail of A's payload survives past the end of B's JSON. The file is then permanently unparseable and every later invocation reports "not authenticated" even though a valid refresh token is still sitting in it.
Change
Write to a temp file in the same directory,
fsync, then rename over the target, so a writer is all-or-nothing and a reader never observes a partial file.SavePendingAuthgets the same treatment — same shape, same exposure.The Windows part
Plain temp+rename is not sufficient on Windows on its own.
MoveFileExreturnsERROR_ACCESS_DENIEDboth when a reader holds the destination open and when two replaces of the same destination overlap. Measured on windows/arm64, go1.26, with no reader present at all:So the rename retries (50 attempts, 10ms apart). Retrying is safe because each attempt is atomic: a losing attempt leaves the previous file intact rather than a partial one. The worst case degrades to "could not persist the refreshed token" — which
KeyringTokenSource.Tokenalready warns about — instead of a corrupt file that needs manual repair.Happy to swap the retry for a proper lock file if you would rather serialise writers outright; retry seemed like the smaller change for the failure mode actually reported.
Tests
pkg/auth/credentials_test.go(new — the package had no test file):TestSaveCredentialsConcurrentStaysParseable— 6 writers x 8 rounds with deliberately differing payload lengths, plus a concurrent reader. The reader tolerates a failed open (retryable, and on Windows expected while a replace is in flight) but fails the test if it ever opens the file and finds content that does not parse. Verified it fails against the current code (unexpected end of JSON input (0 bytes)) and passes with this change,-count=5.TestSaveCredentialsLeavesNoTempFiles— no.tmp-*residue left in the config dir.go build ./... && go test ./...is green.Not included
The error text in #10's last section ("no stored credentials" when the file exists but is corrupt) is untouched here — happy to send it separately if you want it.