Skip to content

test: standalone it_minmax bin + aws-gate smoke_minio (vbench follow-ups) - #1486

Open
aaj3f wants to merge 4 commits into
mainfrom
chore/vbench-followups
Open

test: standalone it_minmax bin + aws-gate smoke_minio (vbench follow-ups)#1486
aaj3f wants to merge 4 commits into
mainfrom
chore/vbench-followups

Conversation

@aaj3f

@aaj3f aaj3f commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Test-infra follow-ups from the perf-series gate batteries (no engine code). Originally stacked on the perf series (#1482); that stack was consolidated and has fully landed on main, so this now targets main directly — clean merge-up of origin/main, no conflicts, and the effective diff is unchanged.

  1. it_minmax_fast_path_fired → standalone test bin. Its thread-local set_default capture requires sole ownership of the process — its own doc header says so, and main currently bundles it into grp_misc (grp_misc.rs:34-35) while the file still opens with "Kept as the only test in this binary". A process-global subscriber is unsound here: the test asserts the absence of the shared fast-path decline event, which siblings also emit → nondeterministic false-fail under parallel cargo test. CI's nextest process-per-test model masks this, which is exactly why it's still latent — bare cargo test is what contributors reach for first. Main has since adopted this same standalone-bin convention for it_count_datatype_pin, so this just brings the minmax test in line with it.
  2. smoke_minio gated on required-features=["aws"] — the example unconditionally imports the aws-gated S3IcebergStorage (lib.rs:98), so bare cargo test -p fluree-db-iceberg fails to compile under default features today; with the [[example]] gate it's green.

The binary-index reload flake that surfaced during the original gate batteries (it_values_type_deleted_repro::values_bound_type_after_delete_incremental_index_timeline) is tracked in #1489 and not addressed here.

aaj3f added 2 commits July 13, 2026 16:18
…feature

it_minmax_fast_path_fired documents that its thread-local set_default
tracing capture requires being the only test in its binary, but the grp_*
consolidation bundled it into grp_misc with 241 siblings — parallel
interleaving flips its event capture (surfaced during the PR-2 gates).
A process-global subscriber is NOT a sound fix here: the test asserts the
ABSENCE of 'fast path declined; running fallback', which siblings also
emit from the shared aggregate fast path, so global capture would
false-fail nondeterministically. Restore the standalone [[test]] bin
(matches the existing it_cyclic_bgp_probe convention). Full api suite
green; standalone bin passes; grp_misc passes in parallel.

smoke_minio needs the aws-gated S3IcebergStorage: mark the example
required-features=["aws"] so bare cargo test -p fluree-db-iceberg is green.

Also closes the PR-2-era q009/q010 native perf flags as environmental:
idle-machine 5-rep recheck = 1.10x / 1.03x vs blessed (was ~2.3x under
concurrent build load).
…e note (review follow-up)

The #1486 review's one optional code item: the header documented BOTH
interference mechanisms but not their scope, leaving the rationale open to
a false disproof ("nextest-run bundled siblings don't interfere"). Both
mechanisms are process-scoped — callsite-interest cache poisoning and
sibling decline-event capture bite under thread-parallel cargo test (one
shared process), not under nextest's process-per-test model. The
standalone bin keeps the test sound under both runners, and cargo test is
what contributors reach for first.

Also from the review: the surfaced binary-index reload flake is now filed
as #1489 (was "issue-filing decision with AJ" in the PR body; body
updated with the number).

Gates: standalone bin 1/1; cargo fmt --all --check green.
@aaj3f
aaj3f changed the base branch from perf/r2rml-pr2-decode-levers to main July 30, 2026 22:10
@aaj3f
aaj3f requested a review from bplatz July 30, 2026 22:39

@bplatz bplatz left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving — leaving notes below rather than blocking on any of them. Please decide which are worth folding in here versus deferring.

What I checked locally:

  • cargo test -p fluree-db-iceberg --no-run on main fails with could not compile fluree-db-iceberg (example "smoke_minio"), 5 errors. On this branch it compiles. The break is real and the gate fixes it.
  • cargo check --workspace --all-targets under default features is clean, so smoke_minio was the only instance of this class.
  • cargo test -p fluree-db-api --test it_minmax_fast_path_fired passes standalone.
  • cargo clippy -p fluree-db-api -p fluree-db-iceberg --all-features --all-targets -- -D warnings is clean.

The branch is two commits behind main; the perf/r2rml-pr2-decode-levers merge is already in main, so it is history noise rather than extra payload.


1. The iceberg gate closes the instance, not the class

required-features = ["aws"] matches the two examples directly above it and is clearly correct. But nothing in CI would have caught this and nothing guards the next one: both CI jobs run --all-features, which turns aws on and compiles smoke_minio fine. The break was only ever visible to someone typing a bare cargo test -p fluree-db-iceberg.

A cargo check --workspace --all-targets step under default features would close the class. Given the shared cache the marginal cost is close to zero. This is the one substantive addition I would suggest for this PR.

2. docs/contributing/tests.md needs the third standalone category

This is the item I would most like to see land here.

tests.md:85-91 lists exactly two reasons a test stays out of a grp_* binary: feature-gated suites, and process-global env mutation. This PR adds a third — process-global tracing state — and documents it only in a source comment, in a file that the same doc four lines earlier tells contributors to add to a group by default.

Without that bullet the next person doing a grouping pass re-bundles it and we repeat this. One line in tests.md makes the invariant discoverable from where people actually look.

3. The new header comment's reason (2) does not hold

the !has_event("fast path declined") ABSENCE assertion would capture a sibling's decline

I do not think this is possible. span_capture::init_test_tracing uses tracing::subscriber::set_default() — thread-local — and its own header says so explicitly ("not set_global_default, for test isolation"). libtest runs each test on its own thread, so a sibling's events go to that thread's NoSubscriber and never reach this SpanStore. Under --test-threads=1 the DefaultGuard has already dropped before the next test starts. There is no path for cross-test capture.

Separately, I could not reproduce the flake: I restored the bundled arrangement (it_minmax_fast_path_fired back inside grp_misc) and ran cargo test -p fluree-db-api --test grp_misc six times — 251 passed, 0 failed, with multilang_min_served_by_fast_path ... ok on every run. Reason (1), the process-global callsite-interest cache, is a genuine tracing mechanism, but Dispatch::new rebuilds the interest cache and tracing-core re-evaluates callsites once a scoped default has been set, which is presumably why it does not bite here.

None of that makes the change wrong — a dedicated binary is cheap insurance for a test whose assertions are about instrumentation, and this arguably just restores what the file header always claimed. I would only trim the comment to what is demonstrable (the callsite-interest cache is process-global; the capture is thread-local, so any work pushed off-thread is invisible; isolation is cheap) and drop the sibling-decline claim, so nobody reasons from a mechanism that is not there.


Mechanics of the move itself all look right: mod support; plus #![cfg(feature = "native")] with no required-features matches the it_cyclic_bgp_probe convention exactly, and the [[test]] entry is in alphabetical position.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants