Skip to content

ci: stabilize TMPDIR for Go test cache#21277

Open
davdhacs wants to merge 1 commit into
masterfrom
davdhacs/tmpdir-fix
Open

ci: stabilize TMPDIR for Go test cache#21277
davdhacs wants to merge 1 commit into
masterfrom
davdhacs/tmpdir-fix

Conversation

@davdhacs

@davdhacs davdhacs commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Description

Set TMPDIR=/tmp in the unit-tests workflow env block.

Go's test cache (computeTestInputsID) hashes every environment variable that the test binary reads via os.Getenv(). These reads are recorded in the test binary's internal log (via testing/internal/testdeps) and replayed by computeTestInputsID on subsequent runs to verify that inputs haven't changed.

Approximately half of our test packages call os.TempDir() (directly or through imported libraries), which reads TMPDIR. On GHA, each job gets a unique TMPDIR — so testInputsID differs between the master push (cache save) and PR run (cache restore), causing Phase 2 cache misses for every affected package.

Config Cache hits
Master baseline (no fix) 571/762 (75%)
With TMPDIR=/tmp (from master cache) 693/762 (91%)
With TMPDIR=/tmp (own cache, stable) 712/762 (93%)

GODEBUG=gocachehash=1 on CI (PR #21276) captured 128,527 testInputs hash lines across 762 packages. 817 of those reference TMPDIR.

User-facing documentation

  • CHANGELOG.md is updated OR update is not needed
  • documentation PR is created and is linked above OR is not needed

Testing and quality

  • the change is production ready: the change is GA, or otherwise the functionality is gated by a feature flag
  • CI results are inspected

Automated testing

No test changes. This fixes the CI test caching infrastructure.

How I validated my change

@openshift-ci

openshift-ci Bot commented Jun 18, 2026

Copy link
Copy Markdown

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@coderabbitai

coderabbitai Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: a7248273-a048-4b76-9fee-f2bb12634712

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch davdhacs/tmpdir-fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

🚀 Build Images Ready

Images are ready for commit 3f08c9a. To use with deploy scripts:

export MAIN_IMAGE_TAG=4.12.x-239-g3f08c9a93c

Set TMPDIR=/tmp in the unit-tests workflow env block.

Go's test cache (computeTestInputsID in cmd/go/internal/test/test.go)
hashes the current value of every environment variable the test binary
reads via os.Getenv(). Many packages call os.TempDir() which reads
TMPDIR. On GHA, TMPDIR is unique per job, causing testInputsID to
differ between cache-save (master push) and cache-restore (PR) runs.

This caused ~25% of test packages to miss Phase 2 of the cache lookup
on every PR run (75% cache hit rate instead of 93%+).

Evidence:
- GODEBUG=gocachehash=1 on CI showed 817 testInputs lines referencing
  TMPDIR out of 128,527 total
- With TMPDIR=/tmp: 693/762 (91%) on first run from master cache,
  stabilizing at 712/762 (93%) across subsequent commits
- Without fix (master baseline): 571/762 (75%)

Partially generated by AI.
@davdhacs davdhacs force-pushed the davdhacs/tmpdir-fix branch from 25c0a84 to 3f08c9a Compare June 19, 2026 04:36
@davdhacs davdhacs changed the title ci: fix test cache by stabilizing TMPDIR across CI runs ci: stabilize TMPDIR for Go test cache Jun 19, 2026
@davdhacs davdhacs requested a review from a team June 19, 2026 04:47
@davdhacs davdhacs marked this pull request as ready for review June 19, 2026 04:47
@davdhacs davdhacs requested a review from a team as a code owner June 19, 2026 04:47
@openshift-ci

openshift-ci Bot commented Jun 19, 2026

Copy link
Copy Markdown

@davdhacs: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/gke-qa-e2e-tests 3f08c9a link false /test gke-qa-e2e-tests

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@tommartensen

tommartensen commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

What is a "Phase 2 cache" in this context?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants