From 2b11b9ebf590e2e972e4237ca7c6f1122578c0a6 Mon Sep 17 00:00:00 2001 From: iteye Date: Mon, 13 Jul 2026 11:01:50 +0800 Subject: [PATCH 1/5] Add "QuarkChain Slave Migration Design" --- L1/qkc-slave-migration-design.md | 146 +++++++++++++++++++++++++++++++ 1 file changed, 146 insertions(+) create mode 100644 L1/qkc-slave-migration-design.md diff --git a/L1/qkc-slave-migration-design.md b/L1/qkc-slave-migration-design.md new file mode 100644 index 0000000..f58bb94 --- /dev/null +++ b/L1/qkc-slave-migration-design.md @@ -0,0 +1,146 @@ +# QuarkChain Slave Migration Design Document + +## 1. Migration Goal + +Migrate the Slave component in QuarkChain cluster communication from Python to Go while maintaining byte-level protocol +compatibility with the Python Master. + +The Python implementation is the only protocol specification. + +## 2. Boundary + +This phase implements: + +- Frame encoding and decoding +- Metadata / Opcode / Message definitions +- RpcConn (basic connection framework) +- MasterConn (Master-Slave communication) +- XshardConn (Slave-Slave communication) +- PeerConn (Peer virtual connection) +- Dispatcher (message routing) +- Slave Runtime (lifecycle management) +- Handler Registration (registration mechanism) +- Handler Dispatch (dispatch scheduling) + +This phase does not implement business logic, such as: + +- Shard Runtime +- StateDB +- TxPool +- Miner +- State Transition + +## 3. Current Status + +| PR | Content | Status | +|-----|--------------------------------------------|-------------| +| PR1 | Wire Frame encoding/decoding | ✅ Completed | +| PR2 | Metadata and Opcode definitions | ✅ Completed | +| PR3 | Message definitions and protocol constants | Review | +| PR4 | RpcConn + XshardConn foundation | Review | + +## 4. Migration Plan + +``` +PR1-PR3 (Protocol foundation layer) +↓ +PR4 (RpcConn / XshardConn / Xshard Handler registration) +↓ +PR5 (MasterConn / Master Handler registration and dispatch) +↓ +PR6 (PeerConn / Dispatcher / Peer Handler registration and dispatch) +↓ +PR7 (SlaveServer Runtime) +``` + +### PR5: MasterConn + +Corresponds to Python: `MasterConnection` + +Main contents: + +- Master-Slave TCP connection +- ClusterMetadata handling (12B) +- MasterConn lifecycle +- Master Handler registration and dispatch framework +- Stub implementations for Handlers depending on business components + +### PR6: PeerConn + Dispatcher + +Corresponds to Python: `PeerShardConnection`, `VirtualConnection` + +Main contents: + +- Peer virtual connection + forwarding mechanism +- Dispatcher message routing +- Peer Handler registration and dispatch framework +- Stub implementations for Handlers depending on business components + +### PR7: SlaveServer Runtime + +Corresponds to Python: `SlaveServer` + +Main contents: + +- Slave lifecycle management + TCP listener +- Unified connection management (MasterConn / PeerConn / XshardConn) +- Dispatcher orchestration +- Wiring of all components + +## 5. Handler Implementation Strategy + +Handlers are categorized by connection type: + +- Xshard Handler → XshardConn +- Master Handler → MasterConn +- Peer Handler → PeerConn + +This phase completes: + +- Handler registration (Registration) +- Handler dispatch (Dispatch) +- RPC request/response flow + +For Handlers depending on other components, such as: + +- Shard Runtime +- StateDB +- TxPool +- Miner + +Only Stub implementations are provided, without implementing actual business logic. + +The goal of the Stubs is to ensure: + +- Opcode can be recognized +- Messages can be parsed +- Handlers can be correctly dispatched +- RPC request/response flow is complete +- Protocol-compatible placeholder responses are returned + +## 6. Testing + +The Python implementation is the protocol authority. + +Compatibility coverage: + +- Frame encoding/decoding +- Metadata encoding +- Opcode mapping +- Message serialization +- RPC request/response flow +- Connection lifecycle + +Test vectors generated by the Python implementation should be used whenever possible. Do not rely solely on Go self +encode/decode tests. + +## 7. Expected Result + +After completing PR1-PR7: + +- Complete the migration of the Slave communication framework +- Master-Slave, Slave-Slave, and Peer virtual connections are all ready +- All RPC Opcodes have completed registration and dispatch +- Business Handlers provide compatible Stub implementations + +This phase only completes the communication layer infrastructure. Business logic will be implemented in later phases. \ No newline at end of file From 273baa41eafd95091b80e8833461cc63eb70d95e Mon Sep 17 00:00:00 2001 From: iteye Date: Mon, 13 Jul 2026 19:16:39 +0800 Subject: [PATCH 2/5] fix comment --- L1/qkc-slave-migration-design.md | 113 +++++++++++++++++++++++++------ 1 file changed, 93 insertions(+), 20 deletions(-) diff --git a/L1/qkc-slave-migration-design.md b/L1/qkc-slave-migration-design.md index f58bb94..79286df 100644 --- a/L1/qkc-slave-migration-design.md +++ b/L1/qkc-slave-migration-design.md @@ -5,9 +5,12 @@ Migrate the Slave component in QuarkChain cluster communication from Python to Go while maintaining byte-level protocol compatibility with the Python Master. -The Python implementation is the only protocol specification. +The Python implementation remains the sole protocol specification. -## 2. Boundary +The goal of this phase is to complete the migration of the communication infrastructure while keeping the existing +Python Master unchanged, providing a foundation for future business logic migration. + +## 2. Scope This phase implements: @@ -38,21 +41,87 @@ This phase does not implement business logic, such as: | PR2 | Metadata and Opcode definitions | ✅ Completed | | PR3 | Message definitions and protocol constants | Review | | PR4 | RpcConn + XshardConn foundation | Review | +| PR5 | MasterConn, handler registration | Review | +| PR6 | PeerConn, Dispatcher, and peer routing | Planned | +| PR7 | Slave runtime and lifecycle management | Planned | + +## 4. Communication Flow + +QuarkChain cluster communication consists of three communication paths: + +- Master ↔ Slave +- Peer ↔ Slave (forwarded through Master) +- Slave ↔ Slave + +### 4.1 Master → Slave RPC + +``` + Master Slave (MasterConn) + │ │ + │──[TCP Frame]──────────────────►│ + │ 12B ClusterMetadata + │ + │ op + rpc_id + payload │ + │ │── cluster_peer_id == 0 + │ │── lookup MASTER_OP_RPC_MAP + │ │── dispatch handler + │◄──[TCP Frame]──────────────────│── return response + │ 12B ClusterMetadata + │ + │ op + rpc_id + payload │ +``` + +### 4.2 Peer Virtual Connection Forwarding + +``` + External P2P Peer Master Slave (MasterConn) + │ │ │ + │──[P2P Frame]─────►│ │ + │ │── convert to ClusterMetadata + │ │──[Cluster Frame]─────►│ + │ │ │── cluster_peer_id ≠ 0 + │ │ │── route to PeerConn + │ │◄──[Cluster Frame]─────│── return response + │◄──[P2P Frame]─────│ │ +``` -## 4. Migration Plan +### 4.3 Slave → Slave RPC + +``` + Slave A (XshardConn) Slave B (XshardConn) + │ │ + │──[TCP Frame]──────────────────►│ + │ 0B Metadata + │ + │ op + rpc_id + payload │ + │ │── lookup XSHARD_OP_RPC_MAP + │ │── dispatch handler + │◄──[TCP Frame]──────────────────│── return response + │ 0B Metadata + │ + │ op + rpc_id + payload │ +``` + +## 5. Migration Plan ``` PR1-PR3 (Protocol foundation layer) ↓ -PR4 (RpcConn / XshardConn / Xshard Handler registration) +PR4 (RpcConn / XshardConn) ↓ -PR5 (MasterConn / Master Handler registration and dispatch) +PR5 (MasterConn) ↓ -PR6 (PeerConn / Dispatcher / Peer Handler registration and dispatch) +PR6 (PeerConn / Dispatcher) ↓ -PR7 (SlaveServer Runtime) +PR7 (Slave Runtime) ``` +### PR4: RpcConn + XshardConn + +Corresponding Python components: `Connection`, `SlaveConnection` + +Main contents: + +- RpcConn: connection framework (TCP read/write loops, RPC ID management, request/response matching) +- XshardConn: Slave-to-Slave TCP connections (0B Metadata) +- Xshard handler registration and Ping/Pong handshake + ### PR5: MasterConn Corresponds to Python: `MasterConnection` @@ -76,7 +145,7 @@ Main contents: - Peer Handler registration and dispatch framework - Stub implementations for Handlers depending on business components -### PR7: SlaveServer Runtime +### PR7: Slave Runtime Corresponds to Python: `SlaveServer` @@ -87,7 +156,7 @@ Main contents: - Dispatcher orchestration - Wiring of all components -## 5. Handler Implementation Strategy +## 6. Handler Implementation Strategy Handlers are categorized by connection type: @@ -118,23 +187,27 @@ The goal of the Stubs is to ensure: - RPC request/response flow is complete - Protocol-compatible placeholder responses are returned -## 6. Testing +## 7. Testing The Python implementation is the protocol authority. -Compatibility coverage: +Testing is performed in two stages: -- Frame encoding/decoding -- Metadata encoding -- Opcode mapping -- Message serialization -- RPC request/response flow -- Connection lifecycle +- PR1–PR6: Use Python-generated test vectors and lightweight Python protocol peers to validate serialization, RPC flows, + and connection lifecycles. These protocol peers are used solely for protocol compatibility testing and do not require + a full Python Master or business components. +- PR7: Run end-to-end interoperability tests using a real Python Master and the Go Slave implementation. + +Testing focuses on: -Test vectors generated by the Python implementation should be used whenever possible. Do not rely solely on Go self -encode/decode tests. +- Frame and Metadata compatibility +- Message serialization compatibility +- RPC request/response flow +- Master-Slave communication +- Slave-Slave communication +- Peer communication -## 7. Expected Result +## 8. Expected Result After completing PR1-PR7: From 727b37f43227f7421a78a011fdbb9f605ba9e8b7 Mon Sep 17 00:00:00 2001 From: iteye Date: Tue, 14 Jul 2026 11:47:28 +0800 Subject: [PATCH 3/5] fix comment --- L1/qkc-slave-migration-design.md | 43 +++++++++++++++++++++++++++----- 1 file changed, 37 insertions(+), 6 deletions(-) diff --git a/L1/qkc-slave-migration-design.md b/L1/qkc-slave-migration-design.md index 79286df..fea606a 100644 --- a/L1/qkc-slave-migration-design.md +++ b/L1/qkc-slave-migration-design.md @@ -98,6 +98,34 @@ QuarkChain cluster communication consists of three communication paths: │ op + rpc_id + payload │ ``` +### 4.4 Connection Model Rules + +**Connection Identity** + +| Connection Type | Description | +|-----------------|-----------------------------------------------------------------------------------------| +| MasterConn | Connection used for Master-Slave communication | +| XshardConn | Connection used for Slave-Slave communication | +| PeerConn | Virtual connection representing forwarded peer traffic, identified by `cluster_peer_id` | + +**RPC Isolation** + +- Each logical connection is an independent RPC channel. +- RPC IDs are scoped per connection and are not required to be globally unique. +- Multiple PeerConn instances sharing the same MasterConn transport may use overlapping RPC IDs. + +**Lifecycle** + +- PeerConn lifecycle is driven by Master commands, never autonomously by the Slave +- Closing MasterConn closes all associated PeerConn instances. + +**Message Routing** + +MasterConn routes frames based on `cluster_peer_id`: + +- `cluster_peer_id == 0`: Master RPC, handled locally by MasterConn +- `cluster_peer_id != 0`: Peer RPC, forwarded by the Dispatcher to the corresponding PeerConn + ## 5. Migration Plan ``` @@ -119,7 +147,7 @@ Corresponding Python components: `Connection`, `SlaveConnection` Main contents: - RpcConn: connection framework (TCP read/write loops, RPC ID management, request/response matching) -- XshardConn: Slave-to-Slave TCP connections (0B Metadata) +- XshardConn: Slave-to-Slave communication - Xshard handler registration and Ping/Pong handshake ### PR5: MasterConn @@ -128,9 +156,8 @@ Corresponds to Python: `MasterConnection` Main contents: -- Master-Slave TCP connection -- ClusterMetadata handling (12B) -- MasterConn lifecycle +- Master-Slave communication +- MasterConn serves as the transport channel for forwarded peer traffic - Master Handler registration and dispatch framework - Stub implementations for Handlers depending on business components @@ -140,8 +167,12 @@ Corresponds to Python: `PeerShardConnection`, `VirtualConnection` Main contents: -- Peer virtual connection + forwarding mechanism -- Dispatcher message routing +- PeerConn: virtual connection for forwarded external peer traffic. +- PeerConn does not own a TCP socket and uses MasterConn as its transport. +- PeerConn is an independent RPC channel with its own RPC ID namespace. +- PeerConn lifecycle is controlled by Master commands. +- Dispatcher: routes frames to the correct PeerConn based on `cluster_peer_id` (`cluster_peer_id == 0` routes to MasterConn + itself) - Peer Handler registration and dispatch framework - Stub implementations for Handlers depending on business components From fd9a1e8b92807003dcc366902d26042301cca172 Mon Sep 17 00:00:00 2001 From: iteye Date: Tue, 14 Jul 2026 12:03:40 +0800 Subject: [PATCH 4/5] fixing the obfuscated runtime --- L1/qkc-slave-migration-design.md | 29 +++++++++++++++-------------- 1 file changed, 15 insertions(+), 14 deletions(-) diff --git a/L1/qkc-slave-migration-design.md b/L1/qkc-slave-migration-design.md index fea606a..73c98b3 100644 --- a/L1/qkc-slave-migration-design.md +++ b/L1/qkc-slave-migration-design.md @@ -35,15 +35,15 @@ This phase does not implement business logic, such as: ## 3. Current Status -| PR | Content | Status | -|-----|--------------------------------------------|-------------| -| PR1 | Wire Frame encoding/decoding | ✅ Completed | -| PR2 | Metadata and Opcode definitions | ✅ Completed | -| PR3 | Message definitions and protocol constants | Review | -| PR4 | RpcConn + XshardConn foundation | Review | -| PR5 | MasterConn, handler registration | Review | -| PR6 | PeerConn, Dispatcher, and peer routing | Planned | -| PR7 | Slave runtime and lifecycle management | Planned | +| PR | Content | Status | +|-----|------------------------------------------------------|-------------| +| PR1 | Wire Frame encoding/decoding | ✅ Completed | +| PR2 | Metadata and Opcode definitions | ✅ Completed | +| PR3 | Message definitions and protocol constants | Review | +| PR4 | RpcConn + XshardConn foundation | Review | +| PR5 | MasterConn, handler registration | Review | +| PR6 | PeerConn, Dispatcher, and peer routing | Planned | +| PR7 | Slave communication runtime and lifecycle management | Planned | ## 4. Communication Flow @@ -171,21 +171,22 @@ Main contents: - PeerConn does not own a TCP socket and uses MasterConn as its transport. - PeerConn is an independent RPC channel with its own RPC ID namespace. - PeerConn lifecycle is controlled by Master commands. -- Dispatcher: routes frames to the correct PeerConn based on `cluster_peer_id` (`cluster_peer_id == 0` routes to MasterConn - itself) +- Dispatcher: routes frames to the correct PeerConn based on `cluster_peer_id` (`cluster_peer_id == 0` routes to + MasterConn itself) - Peer Handler registration and dispatch framework - Stub implementations for Handlers depending on business components -### PR7: Slave Runtime +### PR7: Slave Communication Runtime Corresponds to Python: `SlaveServer` Main contents: -- Slave lifecycle management + TCP listener +- Slave communication service lifecycle management +- TCP listener for cluster connections - Unified connection management (MasterConn / PeerConn / XshardConn) - Dispatcher orchestration -- Wiring of all components +- Wiring of communication components ## 6. Handler Implementation Strategy From 6f856d3a4b8bbd4fec306f9fadca05e7af908e1e Mon Sep 17 00:00:00 2001 From: iteye Date: Wed, 22 Jul 2026 10:46:43 +0800 Subject: [PATCH 5/5] update design --- L1/qkc-slave-migration-design.md | 142 +++++++++++++++++++++---------- 1 file changed, 95 insertions(+), 47 deletions(-) diff --git a/L1/qkc-slave-migration-design.md b/L1/qkc-slave-migration-design.md index 73c98b3..26890a2 100644 --- a/L1/qkc-slave-migration-design.md +++ b/L1/qkc-slave-migration-design.md @@ -16,14 +16,14 @@ This phase implements: - Frame encoding and decoding - Metadata / Opcode / Message definitions -- RpcConn (basic connection framework) +- baseConn (basic connection framework) - MasterConn (Master-Slave communication) - XshardConn (Slave-Slave communication) +- XshardPool (Slave-Slave connection pool) - PeerConn (Peer virtual connection) - Dispatcher (message routing) -- Slave Runtime (lifecycle management) -- Handler Registration (registration mechanism) -- Handler Dispatch (dispatch scheduling) +- SlaveComm Runtime (lifecycle management) +- Handler Registration and Dispatch This phase does not implement business logic, such as: @@ -35,15 +35,15 @@ This phase does not implement business logic, such as: ## 3. Current Status -| PR | Content | Status | -|-----|------------------------------------------------------|-------------| -| PR1 | Wire Frame encoding/decoding | ✅ Completed | -| PR2 | Metadata and Opcode definitions | ✅ Completed | -| PR3 | Message definitions and protocol constants | Review | -| PR4 | RpcConn + XshardConn foundation | Review | -| PR5 | MasterConn, handler registration | Review | -| PR6 | PeerConn, Dispatcher, and peer routing | Planned | -| PR7 | Slave communication runtime and lifecycle management | Planned | +| PR | Content | +|-----|--------------------------------------------| +| PR1 | Wire Frame encoding/decoding | +| PR2 | Metadata and Opcode definitions | +| PR3 | Message definitions and protocol constants | +| PR4 | baseConn + XshardConn + XshardPool | +| PR5 | MasterConn, handler registration/dispatch | +| PR6 | PeerConn, Dispatcher, peer routing | +| PR7 | SlaveComm Runtime, interop testing | ## 4. Communication Flow @@ -62,7 +62,7 @@ QuarkChain cluster communication consists of three communication paths: │ 12B ClusterMetadata + │ │ op + rpc_id + payload │ │ │── cluster_peer_id == 0 - │ │── lookup MASTER_OP_RPC_MAP + │ │── lookup handler │ │── dispatch handler │◄──[TCP Frame]──────────────────│── return response │ 12B ClusterMetadata + │ @@ -71,18 +71,49 @@ QuarkChain cluster communication consists of three communication paths: ### 4.2 Peer Virtual Connection Forwarding +**External view (Master ↔ External Peer):** + ``` External P2P Peer Master Slave (MasterConn) │ │ │ - │──[P2P Frame]─────►│ │ + │──[P2P Frame]─────► │ │ │ │── convert to ClusterMetadata - │ │──[Cluster Frame]─────►│ + │ │──[Cluster Frame]─────► │ │ │ │── cluster_peer_id ≠ 0 - │ │ │── route to PeerConn - │ │◄──[Cluster Frame]─────│── return response - │◄──[P2P Frame]─────│ │ + │ │ │── Dispatcher.RouteFrame + │ │ │── enqueue to PeerConn + │ │◄──[Cluster Frame]───── │── PeerConn writeFrame + │◄──[P2P Frame]───── │── via MasterConn │ +``` + +**Internal view (PeerConn virtual transport):** + +``` + MasterConn (readLoop) Dispatcher PeerConn (readLoop) + │ │ │ + │── frame with cluster_peer_id≠0 │ │ + │── forwarder(frame) ───────────►│ │ + │ │── RouteFrame │ + │ │── lookup peer by │ + │ │ (cluster_peer_id, │ + │ │ branch) │ + │ │── vt.receive(frame) ────►│ + │ │ │── readFrame() from + │ │ │ vt.inbound chan + │ │ │── dispatch handler + │ │ │── handler returns resp + │ │◄── vt.writeFrame(resp) ──│ + │◄── MasterConn.ForwardFrame ────│ │ + │── write to TCP │ │ ``` +PeerConn does not own a TCP socket. Instead: + +- **Inbound**: MasterConn's readLoop calls `forwarder(frame)` → `Dispatcher.RouteFrame` → `vt.receive(frame)` enqueues + into `vt.inbound` channel → PeerConn's readLoop reads from the channel +- **Outbound**: PeerConn's handler returns a response → `vt.writeFrame(resp)` sets `ClusterMetadata` and calls + `MasterConn.ForwardFrame` → MasterConn writes to TCP + ### 4.3 Slave → Slave RPC ``` @@ -91,7 +122,7 @@ QuarkChain cluster communication consists of three communication paths: │──[TCP Frame]──────────────────►│ │ 0B Metadata + │ │ op + rpc_id + payload │ - │ │── lookup XSHARD_OP_RPC_MAP + │ │── lookup handler │ │── dispatch handler │◄──[TCP Frame]──────────────────│── return response │ 0B Metadata + │ @@ -131,24 +162,30 @@ MasterConn routes frames based on `cluster_peer_id`: ``` PR1-PR3 (Protocol foundation layer) ↓ -PR4 (RpcConn / XshardConn) +PR4 (baseConn / XshardConn / XshardPool) ↓ PR5 (MasterConn) ↓ PR6 (PeerConn / Dispatcher) ↓ -PR7 (Slave Runtime) +PR7 (SlaveComm Runtime + interop testing) ``` -### PR4: RpcConn + XshardConn +### PR4: baseConn + XshardConn + XshardPool -Corresponding Python components: `Connection`, `SlaveConnection` +Corresponding Python components: `Connection`, `SlaveConnection`, `SlaveConnectionManager` Main contents: -- RpcConn: connection framework (TCP read/write loops, RPC ID management, request/response matching) -- XshardConn: Slave-to-Slave communication -- Xshard handler registration and Ping/Pong handshake +- **baseConn**: shared RPC connection framework (lifecycle management, RPC ID tracking and monotonic validation, + request/response matching, typed handler dispatch, OpSerializer registration, connection close propagation) +- **XshardConn**: Slave-to-Slave communication (0-byte metadata, PING/PONG identity exchange, shard list validation, + stub handlers for ADD_XSHARD_TX_LIST and BATCH_ADD_XSHARD_TX_LIST) +- **XshardPool**: shard-indexed connection pool (Add/Get/Remove, slave ID deduplication, inbound/outbound tracking, + VerifyAndAdd with PING verification, broadcast send) + +Testing: unit tests only (`xshard_test.go`), validating Go-side connection behavior (RPC ID validation, pending RPC +lifecycle, connection close semantics, pool lifecycle). ### PR5: MasterConn @@ -156,8 +193,8 @@ Corresponds to Python: `MasterConnection` Main contents: -- Master-Slave communication -- MasterConn serves as the transport channel for forwarded peer traffic +- Master-Slave communication (12-byte ClusterMetadata) +- MasterConn serves as the transport channel for forwarded peer traffic via `forwarder` callback - Master Handler registration and dispatch framework - Stub implementations for Handlers depending on business components @@ -167,26 +204,30 @@ Corresponds to Python: `PeerShardConnection`, `VirtualConnection` Main contents: -- PeerConn: virtual connection for forwarded external peer traffic. -- PeerConn does not own a TCP socket and uses MasterConn as its transport. -- PeerConn is an independent RPC channel with its own RPC ID namespace. -- PeerConn lifecycle is controlled by Master commands. -- Dispatcher: routes frames to the correct PeerConn based on `cluster_peer_id` (`cluster_peer_id == 0` routes to - MasterConn itself) +- **PeerConn**: virtual connection for forwarded external peer traffic. Does not own a TCP socket; uses + `virtualTransport` (inbound channel + outbound forwarding through MasterConn) as its transport. Independent RPC ID + namespace. Lifecycle controlled by Master commands. +- **Dispatcher**: routes frames to the correct PeerConn based on `cluster_peer_id` and `branch` (`cluster_peer_id == 0` + stays with MasterConn). Two-layer map: `cluster_peer_id → branch → *PeerConn`. Manages `CreatePeerConns` and + `DestroyPeerConns`. - Peer Handler registration and dispatch framework - Stub implementations for Handlers depending on business components -### PR7: Slave Communication Runtime +### PR7: SlaveComm Runtime + Interop Testing Corresponds to Python: `SlaveServer` Main contents: -- Slave communication service lifecycle management -- TCP listener for cluster connections -- Unified connection management (MasterConn / PeerConn / XshardConn) -- Dispatcher orchestration -- Wiring of communication components +- **SlaveComm**: runtime orchestration layer (TCP listener, accept loop with first-connection-as-MasterConn semantics, + XshardConn lifecycle management, CONNECT_TO_SLAVES_REQUEST handler, unified Start/Stop) +- Wiring of all communication components: MasterConn, XshardPool, Dispatcher, PeerConns +- **Interop testing**: end-to-end tests using a real Python Master (`pyquarkchain`) and Go Slaves + - `TestRealMasterBootstrap`: verifies Master↔Slave PING/PONG, slave-to-slave xshard connections via + CONNECT_TO_SLAVES_REQUEST + - `TestRealMasterPeerLifecycle`: verifies external peer connect → CREATE_CLUSTER_PEER_CONNECTION_REQUEST → + Dispatcher.peers populated → peer disconnect → DESTROY_CLUSTER_PEER_CONNECTION_COMMAND → Dispatcher.peers cleaned + up ## 6. Handler Implementation Strategy @@ -225,10 +266,14 @@ The Python implementation is the protocol authority. Testing is performed in two stages: -- PR1–PR6: Use Python-generated test vectors and lightweight Python protocol peers to validate serialization, RPC flows, - and connection lifecycles. These protocol peers are used solely for protocol compatibility testing and do not require - a full Python Master or business components. -- PR7: Run end-to-end interoperability tests using a real Python Master and the Go Slave implementation. +- **PR1–PR6**: Unit tests validate Go-side communication components, including RPC + lifecycle, connection handling, handler dispatch, pool lifecycle, and virtual + transport routing. Protocol behavior is implemented according to the Python + reference implementation. +- **PR7**: End-to-end interoperability tests using a real Python Master + (`pyquarkchain`) and Go Slave processes. Tests validate the complete + communication lifecycle: bootstrap handshake, slave-to-slave connection + establishment, and peer connection create/destroy lifecycle through Dispatcher. Testing focuses on: @@ -237,7 +282,7 @@ Testing focuses on: - RPC request/response flow - Master-Slave communication - Slave-Slave communication -- Peer communication +- Peer virtual connection forwarding ## 8. Expected Result @@ -247,5 +292,8 @@ After completing PR1-PR7: - Master-Slave, Slave-Slave, and Peer virtual connections are all ready - All RPC Opcodes have completed registration and dispatch - Business Handlers provide compatible Stub implementations +- End-to-end interoperability with real Python Master validated + +This phase only completes the communication layer infrastructure. Business logic will be implemented in later phases. -This phase only completes the communication layer infrastructure. Business logic will be implemented in later phases. \ No newline at end of file +--- \ No newline at end of file