Skip to content

fix(e2e): ground-truth pool verdict + earned Tari lag tolerance (#746)#747

Merged
VijitSingh97 merged 1 commit into
developfrom
fix/746-e2e-flaky-verdicts
Jul 22, 2026
Merged

fix(e2e): ground-truth pool verdict + earned Tari lag tolerance (#746)#747
VijitSingh97 merged 1 commit into
developfrom
fix/746-e2e-flaky-verdicts

Conversation

@VijitSingh97

Copy link
Copy Markdown
Collaborator

Closes #746. The two Tor-timing flakes that hit 2 of 4 e2e runs during the v1.12.0 cut (neither a code defect — the same ref passed 374/0 and 380/0 on clean runs).

Pool type — check the ground truth, not just the classifier

After a sidechain switch, p2pool runs the NEW --mini/--nano flags while the dashboard's peer-port classifier can still report the pre-switch pool — determinate, wrong, and transient. Both assert_scenario and assert_pool_switched hard-failed that as a "render bug". New shared assert_pool_type adds the check the harness never made: the rendered P2POOL_FLAGS.

got vs want P2POOL_FLAGS verdict
match pass
Unknown/empty warn (peer timing, #454)
wrong correct warn (classifier lag, #746)
wrong wrong fail (real render bug)

The mismatch path now verifies actual flags, so this is a stronger check than the old hard-fail.

Tari — earned lag tolerance

Every per-scenario restart sends Tari back through "discovering the target height" (loading), which can outlast wait_tari_synced over Tor; the bare assert_eq then fired on an in-progress state (the cold-read anti-pattern the harness's own #687 comment describes — it passed in 3 of 4 scenarios of the failing run). New assert_tari_synced_required: done passes and records TARI_SEEN_DONE; loading/syncing after that proof warns as re-discovery lag; a Tari that never syncs still fails the gate — the tolerance is earned, not free.

Validation

  • Harness selftest: 148/0, covering every new branch (flags ground truth per pool, all four pool verdicts, the earned-tolerance matrix incl. first-look-loading → FAIL, and the TARI_SEEN_DONE side effect).
  • make lint-sh clean.
  • Live targeted e2e with this branch (gouda + borrowed miner-1) running now; result will be posted here before merge.

Two Tor-timing flakes hit 2 of 4 runs during the v1.12.0 cut; neither was a
code defect (same ref passed 374/0 and 380/0 on the clean runs):

- pool type: after a sidechain switch the dashboard's peer-port classifier can
  keep reporting the PRE-switch pool — determinate, wrong, and transient — and
  both assert_scenario and assert_pool_switched hard-failed it as a "render
  bug". New shared assert_pool_type consults the ground truth the harness never
  checked: the rendered P2POOL_FLAGS (--mini/--nano; main carries neither).
  Correct flags -> classifier-lag WARN (same class as Unknown, #454/#687);
  wrong flags -> FAIL. The mismatch path now verifies actual flags, so the
  check is stronger than before, not looser.

- tari synced (required): every per-scenario restart sends Tari back through
  "discovering the target height" ('loading'), which can outlast
  wait_tari_synced over Tor; the bare assert_eq then fired on an in-progress
  state — the same cold-read anti-pattern the #687 comment describes. New
  assert_tari_synced_required earns its tolerance: "done" passes and records
  TARI_SEEN_DONE; loading/syncing AFTER that proof warns as re-discovery lag;
  a Tari that never syncs still fails the gate.

Both helpers live in lib.sh so the selftest covers every branch (stubbed
emitters; 148/0): flags ground truth per pool, all four pool verdicts, and the
earned-tolerance matrix including first-look-loading -> FAIL.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@VijitSingh97

Copy link
Copy Markdown
Collaborator Author

Live validation complete: targeted e2e with this branch on gouda + borrowed miner-1 — 380 passed / 0 failed / 3 skipped, ✓ E2E PASSED. The shared assert_pool_type warn path fired once live (Unknown → peer-timing WARN, #454), exercising the refactored verdict on the real bench; the classifier-lag and Tari-tolerance branches are covered by the selftest (148/0). Clean restore (miner repointed, baseline healthy).

@VijitSingh97
VijitSingh97 merged commit 10c2c2f into develop Jul 22, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

e2e: pool-type and tari-sync assertions flake on Tor timing after per-scenario restarts

1 participant