gazette: surface fragment store diagnostic on FRAGMENT_STORE_UNHEALTHY#3159
Merged
Merged
Conversation
The broker's AppendResponse and FragmentStoreHealthResponse both carry a `store_health_error` string with the failing fragment store's own diagnostic, populated when status is FRAGMENT_STORE_UNHEALTHY. The gazette crate collapsed every non-OK status into Error::BrokerStatus(status), which holds only the enum value, so the diagnostic was dropped and callers saw only the generic "unexpected broker status: FragmentStoreUnhealthy". Add a dedicated Error::FragmentStoreUnhealthy(String) variant and populate it from `store_health_error` in the append and fragment_store_health paths. A new variant (rather than a field on BrokerStatus) avoids churning the many call sites that pattern-match BrokerStatus as a tuple. This also fixes the storage-mappings GraphQL health check, whose existing error arm now surfaces the store's real diagnostic instead of the generic status string.
jgraettinger
requested review from
GregorShear
and removed request for
GregorShear
July 13, 2026 20:04
GregorShear
approved these changes
Jul 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The broker's
AppendResponseandFragmentStoreHealthResponseboth carry astore_health_errorstring — the actual diagnostic from the failing fragment store — populated when the status isFRAGMENT_STORE_UNHEALTHY. But thegazettecrate collapsed every non-OK status intoError::BrokerStatus(status), which holds only the enum value. The response (and itsstore_health_error) was discarded, so upstack callers saw only the generic"unexpected broker status: FragmentStoreUnhealthy".Fix
Error::FragmentStoreUnhealthy(String)variant (Display:"fragment store is unhealthy: {0}"). A new variant, rather than a field onBrokerStatus, avoids churning the ~15 call sites that pattern-matchBrokerStatus(broker::Status::X)as a tuple.is_transient(), matching prior behavior (the status means the store has failed its health check for an extended period).store_health_error(append andfragment_store_health) now special-caseFRAGMENT_STORE_UNHEALTHYand move that string into the new error before the response is dropped.Bonus
This also fixes the
storage_mappings.rsGraphQL health-check consumer. Its existingOk(Err(err)) => Some(err.to_string())arm previously produced the useless generic status string; it now surfaces the store's real diagnostic.Retry semantics are unchanged: the append
Streamstill retries on its own loop regardless ofis_transient().cargo checkpasses forgazetteand dependent crates (publisher,shuffle,dekaf,control-plane-api,migrate);cargo fmtapplied.