test: standalone it_minmax bin + aws-gate smoke_minio (vbench follow-ups) - #1486
test: standalone it_minmax bin + aws-gate smoke_minio (vbench follow-ups)#1486aaj3f wants to merge 4 commits into
Conversation
…feature it_minmax_fast_path_fired documents that its thread-local set_default tracing capture requires being the only test in its binary, but the grp_* consolidation bundled it into grp_misc with 241 siblings — parallel interleaving flips its event capture (surfaced during the PR-2 gates). A process-global subscriber is NOT a sound fix here: the test asserts the ABSENCE of 'fast path declined; running fallback', which siblings also emit from the shared aggregate fast path, so global capture would false-fail nondeterministically. Restore the standalone [[test]] bin (matches the existing it_cyclic_bgp_probe convention). Full api suite green; standalone bin passes; grp_misc passes in parallel. smoke_minio needs the aws-gated S3IcebergStorage: mark the example required-features=["aws"] so bare cargo test -p fluree-db-iceberg is green. Also closes the PR-2-era q009/q010 native perf flags as environmental: idle-machine 5-rep recheck = 1.10x / 1.03x vs blessed (was ~2.3x under concurrent build load).
…to chore/vbench-followups
…e note (review follow-up) The #1486 review's one optional code item: the header documented BOTH interference mechanisms but not their scope, leaving the rationale open to a false disproof ("nextest-run bundled siblings don't interfere"). Both mechanisms are process-scoped — callsite-interest cache poisoning and sibling decline-event capture bite under thread-parallel cargo test (one shared process), not under nextest's process-per-test model. The standalone bin keeps the test sound under both runners, and cargo test is what contributors reach for first. Also from the review: the surfaced binary-index reload flake is now filed as #1489 (was "issue-filing decision with AJ" in the PR body; body updated with the number). Gates: standalone bin 1/1; cargo fmt --all --check green.
bplatz
left a comment
There was a problem hiding this comment.
Approving — leaving notes below rather than blocking on any of them. Please decide which are worth folding in here versus deferring.
What I checked locally:
cargo test -p fluree-db-iceberg --no-runon main fails withcould not compile fluree-db-iceberg (example "smoke_minio"), 5 errors. On this branch it compiles. The break is real and the gate fixes it.cargo check --workspace --all-targetsunder default features is clean, sosmoke_miniowas the only instance of this class.cargo test -p fluree-db-api --test it_minmax_fast_path_firedpasses standalone.cargo clippy -p fluree-db-api -p fluree-db-iceberg --all-features --all-targets -- -D warningsis clean.
The branch is two commits behind main; the perf/r2rml-pr2-decode-levers merge is already in main, so it is history noise rather than extra payload.
1. The iceberg gate closes the instance, not the class
required-features = ["aws"] matches the two examples directly above it and is clearly correct. But nothing in CI would have caught this and nothing guards the next one: both CI jobs run --all-features, which turns aws on and compiles smoke_minio fine. The break was only ever visible to someone typing a bare cargo test -p fluree-db-iceberg.
A cargo check --workspace --all-targets step under default features would close the class. Given the shared cache the marginal cost is close to zero. This is the one substantive addition I would suggest for this PR.
2. docs/contributing/tests.md needs the third standalone category
This is the item I would most like to see land here.
tests.md:85-91 lists exactly two reasons a test stays out of a grp_* binary: feature-gated suites, and process-global env mutation. This PR adds a third — process-global tracing state — and documents it only in a source comment, in a file that the same doc four lines earlier tells contributors to add to a group by default.
Without that bullet the next person doing a grouping pass re-bundles it and we repeat this. One line in tests.md makes the invariant discoverable from where people actually look.
3. The new header comment's reason (2) does not hold
the
!has_event("fast path declined")ABSENCE assertion would capture a sibling's decline
I do not think this is possible. span_capture::init_test_tracing uses tracing::subscriber::set_default() — thread-local — and its own header says so explicitly ("not set_global_default, for test isolation"). libtest runs each test on its own thread, so a sibling's events go to that thread's NoSubscriber and never reach this SpanStore. Under --test-threads=1 the DefaultGuard has already dropped before the next test starts. There is no path for cross-test capture.
Separately, I could not reproduce the flake: I restored the bundled arrangement (it_minmax_fast_path_fired back inside grp_misc) and ran cargo test -p fluree-db-api --test grp_misc six times — 251 passed, 0 failed, with multilang_min_served_by_fast_path ... ok on every run. Reason (1), the process-global callsite-interest cache, is a genuine tracing mechanism, but Dispatch::new rebuilds the interest cache and tracing-core re-evaluates callsites once a scoped default has been set, which is presumably why it does not bite here.
None of that makes the change wrong — a dedicated binary is cheap insurance for a test whose assertions are about instrumentation, and this arguably just restores what the file header always claimed. I would only trim the comment to what is demonstrable (the callsite-interest cache is process-global; the capture is thread-local, so any work pushed off-thread is invisible; isolation is cheap) and drop the sibling-decline claim, so nobody reasons from a mechanism that is not there.
Mechanics of the move itself all look right: mod support; plus #![cfg(feature = "native")] with no required-features matches the it_cyclic_bgp_probe convention exactly, and the [[test]] entry is in alphabetical position.
Test-infra follow-ups from the perf-series gate batteries (no engine code). Originally stacked on the perf series (#1482); that stack was consolidated and has fully landed on main, so this now targets
maindirectly — clean merge-up oforigin/main, no conflicts, and the effective diff is unchanged.it_minmax_fast_path_fired→ standalone test bin. Its thread-localset_defaultcapture requires sole ownership of the process — its own doc header says so, and main currently bundles it intogrp_misc(grp_misc.rs:34-35) while the file still opens with "Kept as the only test in this binary". A process-global subscriber is unsound here: the test asserts the absence of the shared fast-path decline event, which siblings also emit → nondeterministic false-fail under parallelcargo test. CI's nextest process-per-test model masks this, which is exactly why it's still latent — barecargo testis what contributors reach for first. Main has since adopted this same standalone-bin convention forit_count_datatype_pin, so this just brings the minmax test in line with it.smoke_miniogated onrequired-features=["aws"]— the example unconditionally imports theaws-gatedS3IcebergStorage(lib.rs:98), so barecargo test -p fluree-db-icebergfails to compile under default features today; with the[[example]]gate it's green.The binary-index reload flake that surfaced during the original gate batteries (
it_values_type_deleted_repro::values_bound_type_after_delete_incremental_index_timeline) is tracked in #1489 and not addressed here.