feat(quarantine): send test collection id and record resolution mode by max-trunk · Pull Request #1147 · trunk-io/analytics-cli

max-trunk · 2026-07-24T16:11:48Z

What

Wires the CLI to the test-collection additions in the public API's POST /metrics/getQuarantineConfig endpoint (server side: trunk-io/trunk#33132), across both the traditional and rspec invocation paths:

Sends --test-collection-id as testCollectionShortId on the request.
Records the response's new quarantineResolutionMode (repo | test_collection) in telemetry and logs the resolved source.

The --no-repo flag / noRepo request param are intentionally out of scope for this PR.

Changes

api/src/message.rs
- Add test_collection_short_id to GetQuarantineConfigRequest (testCollectionShortId, omitted when None).
- Add a typed QuarantineResolutionMode enum (Repo | TestCollection | Unspecified) as the quarantine_resolution_mode field on GetQuarantineConfigResponse (#[serde(default)]). Its Deserialize is tolerant: any unknown, absent, or non-string value degrades to Unspecified rather than failing the whole response parse, so a future server value never breaks older clients. A single From<QuarantineResolutionMode> for the proto enum centralizes the mapping.
Request sourcing (both paths) — the id comes from the --test-collection-id flag / TRUNK_TEST_COLLECTION_ID env var, threaded into the request on the traditional path (cli/src/upload_command.rs → cli/src/context.rs) and read directly on the rspec path (test_report/src/report.rs). Empty values are filtered so an empty env var doesn't send "".
Telemetry — new proto QuarantineResolutionMode enum + UploadMetrics.quarantine_resolution_mode = 9 in proto/proto/upload_metrics.proto; carried through QuarantineContext (traditional) and a new RunUploadOptions.quarantine_resolution_mode_override (rspec).
Logging — on a successful lookup a human-readable info log names the resolved source (Resolved quarantine status for test collection <id> / ... for repo <name>), emitted at most once per report (not per test); disk-cache hits are tagged (cached).
rspec disk cache — the resolution mode is stored in the cache entry and restored on a cache hit (so telemetry is correct without a re-fetch), and the collection id is part of the cache key so runs with different ids in the same repo can't collide.

Compatibility

Response field is safe with older CLIs — GetQuarantineConfigResponse has no deny_unknown_fields, so the extra key is ignored; the server PR can land first.
Unknown / non-string modes are safe — the tolerant deserializer degrades anything unrecognized to Unspecified.
Telemetry field is safe with an un-updated server — telemetry is protobuf-encoded, and a new field tag (= 9) is skipped by servers that don't know it.

Testing

✅ cargo check and the full test suite pass across api, trunk-analytics-cli, and test_report.
cli/tests/upload.rs asserts that --test-collection-id reaches the getQuarantineConfig request (guards the traditional-path wiring).

Follow-up

Confirm the rspec Ruby gem forwards --test-collection-id / sets TRUNK_TEST_COLLECTION_ID (no Rust-side change needed).

🤖 Generated with Claude Code

Pipe the existing --test-collection-id value into the getQuarantineConfig request (testCollectionShortId) and record the response's new quarantineResolutionMode field in telemetry, across both the traditional and rspec invocation paths. - api: add test_collection_short_id to GetQuarantineConfigRequest and quarantine_resolution_mode (Option, serde default) to the response so older servers that omit it still deserialize. - populate test_collection_short_id from meta (traditional) and from TRUNK_TEST_COLLECTION_ID (rspec). - proto: add QuarantineResolutionMode enum and UploadMetrics field 9; carry the mode through QuarantineContext and a RunUploadOptions override. - emit a Sentry-only tracing event (hidden_in_console) with the resolved mode in both paths for debugging. Server side: trunk-io/trunk#33132 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

trunk-io · 2026-07-24T16:12:27Z

😎 Merged directly without going through the merge queue, as the queue was empty and the PR was up to date with the target branch - details.

TylerJang27 · 2026-07-24T16:17:29Z

-            anyhow::Result::Ok(response) => (Some(response), QuarantineFetchStatus::FetchSucceeded),
+            anyhow::Result::Ok(response) => {
+                // Surface which source the server resolved quarantine status from. Sent to Sentry
+                // only (hidden from the console) to aid debugging without adding CLI noise.


Imo would be preferable to include a visible log for it if test collection id is set

TylerJang27 · 2026-07-24T16:21:19Z

                    quarantined_tests.insert(quarantined_test_id.clone(), true);
                }
+                // Surface which source the server resolved quarantine status from. Sent to Sentry
+                // only (hidden from the console) to aid debugging without adding CLI noise.


codecov-commenter · 2026-07-24T16:24:26Z

Codecov Report

❌ Patch coverage is 88.46154% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.17%. Comparing base (6bed25e) to head (509680e).

Files with missing lines	Patch %	Lines
api/src/message.rs	66.66%	10 Missing ⚠️
test_report/src/report.rs	95.31%	3 Missing ⚠️
cli/src/context_quarantine.rs	89.47%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1147      +/-   ##
==========================================
+ Coverage   82.90%   83.17%   +0.27%     
==========================================
  Files          70       71       +1     
  Lines       15919    16040     +121     
==========================================
+ Hits        13197    13342     +145     
+ Misses       2722     2698      -24

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

trunk-staging-io · 2026-07-24T16:28:38Z

Failed Test	Failure Summary	Logs
`pending_quarantine_test should be quarantined when run with variant`	A test marked as pending was expected to fail but unexpectedly passed.	Logs ↗︎
`variant_quarantine_test should be quarantined when run with variant`	A test expected the sum of 2 + 2 to be 5, but it was actually 4, indicating a failing assertion.	Logs ↗︎

_{View Full Report ↗︎ ⋅ Docs}

Introduce a typed api::message::QuarantineResolutionMode enum with a tolerant deserializer (unknown or absent values -> Unspecified) so a new server-side mode never breaks deserialization on older clients. It replaces the Option<String> previously threaded through the response, QuarantineContext, TestReport, and the quarantine disk cache, and collapses the two duplicated string-match sites into a single From impl. Also: - Cache the resolution mode in the disk-cache entry and restore it on a cache hit, so telemetry reports the true mode without a re-fetch. - Rework resolution-mode logging into human-readable messages, emitted at most once per report. - Drop the redundant standalone TestReport field (telemetry now reads from quarantine_config). - Populate quarantine_resolution_mode in the QuarantineContext test constructors the vendored-openssl build never compiled. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

trunk-io · 2026-07-24T21:04:54Z

_{View Full Report ↗︎ ⋅ Docs}

… parse, cache key - BLOCKING: the traditional CLI path built getQuarantineConfig with test_collection_short_id always None. meta.base_props.test_collection isn't populated until upload_bundle (after the request is built), and only when the upload-intent response returns collection metadata, so the CLI value was silently dropped. Thread upload_args.test_collection_short_id (empty-filtered) into gather_exit_code_and_quarantined_tests_context and use it directly, matching the rspec path. - Include the collection id in the rspec disk-cache key so two runs in the same repo with different TRUNK_TEST_COLLECTION_ID values can't share an entry (and the wrong quarantine list). - Make QuarantineResolutionMode's deserializer tolerant of non-string JSON types (number/bool/object/null), not just unknown strings. - Filter empty TRUNK_TEST_COLLECTION_ID so an empty env var doesn't send "". - Collapse the duplicated resolution-mode log message into a QuarantineResolutionMode::resolution_log_line helper; mark cached lookups. - Inline the trivial quarantine_resolution_mode() mapping fn at its one call site; use the imported enum name consistently. - Tests: assert the collection id reaches the getQuarantineConfig request on the traditional path (cli/tests/upload.rs), plus a non-string deserialize case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…ests - Shorten the QuarantineResolutionMode doc comment; drop the deserializer and resolution_log_line comments in api/src/message.rs. - Remove the api/src/message.rs serde unit tests (type-only, low value). - Prune the per-variant and trailing comments on the proto QuarantineResolutionMode enum, keeping only the top-level line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

TylerJang27 approved these changes Jul 24, 2026

View reviewed changes

max-trunk marked this pull request as ready for review July 24, 2026 20:42

max-trunk and others added 3 commits July 24, 2026 21:12

chore(quarantine): drop test_collection_short_id field comment

509680e

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

max-trunk requested review from acatxnamedvirtue and dfrankland July 27, 2026 14:24

dfrankland approved these changes Jul 27, 2026

View reviewed changes

trunk-io Bot merged commit 846c623 into main Jul 27, 2026
27 checks passed

max-trunk mentioned this pull request Jul 27, 2026

feat(quarantine): persist quarantine resolution mode in meta.json #1148

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(quarantine): send test collection id and record resolution mode - #1147

feat(quarantine): send test collection id and record resolution mode#1147
trunk-io[bot] merged 5 commits into
mainfrom
max/add-tc-id-to-get-quarantine-config

max-trunk commented Jul 24, 2026 •

edited

Loading

Uh oh!

trunk-io Bot commented Jul 24, 2026 •

edited

Loading

Uh oh!

Uh oh!

TylerJang27 Jul 24, 2026

Uh oh!

TylerJang27 Jul 24, 2026

Uh oh!

codecov-commenter commented Jul 24, 2026 •

edited

Loading

Uh oh!

trunk-staging-io Bot commented Jul 24, 2026 •

edited

Loading

Uh oh!

trunk-io Bot commented Jul 24, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

max-trunk commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Changes

Compatibility

Testing

Follow-up

Uh oh!

trunk-io Bot commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

TylerJang27 Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

TylerJang27 Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

trunk-staging-io Bot commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trunk-io Bot commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

4 participants

max-trunk commented Jul 24, 2026 •

edited

Loading

trunk-io Bot commented Jul 24, 2026 •

edited

Loading

codecov-commenter commented Jul 24, 2026 •

edited

Loading

trunk-staging-io Bot commented Jul 24, 2026 •

edited

Loading

trunk-io Bot commented Jul 24, 2026 •

edited

Loading