Skip to content

docs(webhooks): move Flaky Tests webhook content off the Merge Queue page #285

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
samgutentag wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs(webhooks): move Flaky Tests webhook content off the Merge Queue page #285

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion flaky-tests/webhooks/index.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Trunk lets you create custom workflows with **event-triggered webhooks**. Flaky
www.svix.com
</Card>

Trunk publishes three Flaky Tests event types to Svix. Each event includes a full JSON schema with field descriptions visible in the Svix app portal.
Trunk publishes the following Flaky Tests event types to Svix. Each event includes a full JSON schema with field descriptions visible in the Svix app portal.

### `test_case.monitor_status_changed`

Expand Down Expand Up @@ -65,6 +65,16 @@ Emitted when a test case changes status (e.g., becomes flaky or is resolved), as
| `repository` | object | See `repository` fields above |
| `test_case` | object | See `test_case` fields above |

<Note>
The original `test_case.status_changed` event is being phased out in favor of `v2.test_case.status_changed`. v2 intentionally omits aggregate metrics like `failure_rate_last_7d`. To enrich a status-change webhook with current metrics, call [`flaky-tests/get-test-details`](https://api.trunk.io/docs#tag/flaky-tests) from your webhook handler using the `test_case.id` from the payload.
</Note>

### `test_case.quarantining_setting_changed`

Emitted only when a quarantining setting is manually overridden to `ALWAYS_QUARANTINE` or `NEVER_QUARANTINE`. It does **not** fire when Trunk auto-quarantines a test based on flakiness — to react to auto-quarantine, listen to `v2.test_case.status_changed` and check `new_status == "FLAKY"`.

The full payload schema is available in the Svix event catalog linked above.

### `test_case.investigation_completed`

Emitted when an AI-powered flaky test analysis finishes for a test case.
Expand All @@ -87,6 +97,8 @@ Emitted when an AI-powered flaky test analysis finishes for a test case.

Citations in `markdown_summary` and `facts[].content` are delivered as fully rendered links to the supporting evidence, so you can consume these fields directly without resolving placeholder tags.

`test_case.investigation_completed` fires only when an AI investigation finishes — it is not a general status-change signal. For "this test just became flaky" reactions (Linear ticket creation, Slack alerts, and so on), subscribe to `v2.test_case.status_changed` instead.

**Delivery reliability.** If the webhook delivery provider rate-limits a request, Trunk automatically retries with exponential backoff, so a transient spike rarely drops an event. This applies to every Flaky Tests webhook event and needs no configuration on your side.

You can also find guides for specific examples here:
Expand Down
24 changes: 2 additions & 22 deletions merge-queue/webhooks.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,29 +60,9 @@ All webhook payloads include an ISO 8601 `timestamp` field marking when the stat

***

### `test_case.status_changed` v1 vs v2
### Flaky Tests webhook events

The original `test_case.status_changed` event is being phased out in favor of `v2.test_case.status_changed`. The v2 event intentionally **omits aggregate metrics** like `failure_rate_last_7d` — those fields will be `0` or missing on the v1 event today and are not present in v2 at all.

To enrich a status-change webhook with current metrics, call the [`flaky-tests/get-test-details`](https://api.trunk.io/docs#tag/flaky-tests) API from your webhook handler using the `test_case.id` from the payload.

See the [Flaky Tests webhooks reference](/flaky-tests/webhooks) for the full `v2.test_case.status_changed` payload schema and other flaky-tests event types.

***

### Quarantining events

`test_case.quarantining_setting_changed` fires only when a quarantining setting is manually overridden to `ALWAYS_QUARANTINE` or `NEVER_QUARANTINE`. It does **not** fire when Trunk auto-quarantines a test based on flakiness.

To react to auto-quarantine, listen to `v2.test_case.status_changed` and check `new_status == "FLAKY"` in the payload.

***

### AI investigation events

`test_case.investigation_completed` fires only when an AI/autofix flaky test analysis finishes — this is a beta feature scoped to AI investigations, not general status changes.

For "this test just became flaky" reactions (Linear ticket creation, Slack alerts, and so on), subscribe to `v2.test_case.status_changed` instead.
`test_case.*` events — status changes (`v2.test_case.status_changed`), quarantining setting overrides, and AI investigation results — are part of Flaky Tests, not Merge Queue. See the [Flaky Tests webhooks reference](/flaky-tests/webhooks) for their payload schemas and usage.

***

Expand Down