Introduce SourceRuntime to own the source connection (GTID failover 1/3)#451
Open
driv3r wants to merge 1 commit into
Open
Introduce SourceRuntime to own the source connection (GTID failover 1/3)#451driv3r wants to merge 1 commit into
driv3r wants to merge 1 commit into
Conversation
driv3r
changed the title
Ghostferry's source connection and its DatabaseConfig are currently copied by value into every consumer (data iterator, cursors, verifiers, ad-hoc SHOW CREATE queries). That is fine while the source never changes, but a source master failover needs to repoint every consumer at a newly promoted writer atomically, which today would mean hand-editing a field on each one. Introduce SourceRuntime as the future single owner of "the current source": it holds the source *sql.DB and *DatabaseConfig behind concurrency-safe accessors and offers Replace to swap in a new source (opening the new connection and retaining the old handle rather than closing it, since in-flight cursors and cached statements may still reference it) plus CloseRetired to release retired handles at teardown. This change only introduces the type and has the Ferry construct/hold one alongside SourceDB in Initialize; no consumer is migrated yet, so behavior is unchanged. Follow-up changes migrate consumers to the runtime and then use Replace to implement master failover.
driv3r
force-pushed
the
gtid-stage6-master-failover
branch
from
July 23, 2026 14:29
964f52d to
b465681
Compare
driv3r
changed the title
This was referenced Jul 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Introduces
SourceRuntime, a single owner of Ghostferry's current source connection and itsDatabaseConfig.This is PR 1 of 3 reworking master failover recovery into a clean abstraction:
SourceRuntime(no consumers migrated, behavior-neutral).SourceRuntime.Supersedes the earlier single-PR spike (streamer recovery + hand-picked Ferry-wide field swap), which proved what needs to move but wired every source field by hand. The spike is preserved on the
backup/gtid-failover-spike-964f52dbranch /failover-spike-backuptag for reference.Why
Ghostferry's source
*sql.DBand*DatabaseConfigare copied by value into every consumer: the data iterator, each cursor, the inline/iterative/checksum verifiers, and ad-hocSHOW CREATEqueries. That is fine while the source never changes, but a source master failover must repoint every consumer at a promoted writer atomically. Doing that by editing a field on each consumer is brittle — every future source consumer would have to remember to join the swap.Centralising ownership behind one type inverts that: consumers ask the runtime for the current source instead of holding stale copies.
How
SourceRuntime(source_runtime.go):*sql.DB+*DatabaseConfigbehind concurrency-safe accessors (DB(),Config()).Replace(config, logger)opens a new source connection, installs it, and retires (retains, does not close) the previous handle, since in-flight cursors and cached prepared statements may still reference it.CloseRetired()releases retired handles at a safe point (teardown).Ferry wiring:
Initializeconstructs aSourceRuntimealongsideSourceDB.Ferry.SourceRuntime()exposes it.SourceDB/Config.Sourceremain authoritative, so this PR changes no runtime behavior.Testing
Replace(install + retire + multi-swap retention),CloseRetireddrain, and nil-initial handling;-raceclean.test/gointegration suite passes in bothfile_positionandgtidmodes (confirming behavior neutrality).go build+gofmtclean.