The cluster has four moving parts.
- A client
- A router
- Three storage nodes
- One Redis instance behind each node
Client
│
▼
Router (:8000)
/ | \
/ | \
▼ ▼ ▼
Node1 Node2 Node3
:8001 :8002 :8003
│ │ │
▼ ▼ ▼
Redis6379 Redis6380 Redis6381
Clients never talk to nodes directly.
Every request goes through the router. The router decides which node owns the key and forwards the request.
The node stores the data locally and replicates it to its configured replica.
The first problem is deciding where a key belongs.
The simplest approach is modulo hashing.
node = hash(key) % 3
It works until the number of nodes changes.
hash("user:42") % 3 = node1
hash("user:42") % 4 = node3
Adding a single node changes the destination for most keys.
That means moving most of the data.
This project uses consistent hashing instead.
Both nodes and keys are mapped into the same hash space.
0 -------------------------------------- 2³²
\______________________________________/
Each key belongs to the first node encountered while moving clockwise around the ring.
Adding another node only affects the keys between that node and its predecessor.
Everything else stays where it is.
Using only one position per node produces uneven distribution.
Instead, every physical node owns multiple virtual positions.
type Ring struct {
Nodes []uint32
Owners map[uint32]*node.Node
}During startup each node is inserted 100 times.
ring.AddNode(node1, 100)
ring.AddNode(node2, 100)
ring.AddNode(node3, 100)That creates 300 virtual positions around the ring.
Each position hashes a unique string built from the node ID and a virtual node index.
If two virtual nodes hash to the same value, a small salt is added until an unused position is found.
Once all positions are inserted, they're sorted.
Looking up a key becomes a binary search followed by a clockwise walk.
idx := sort.Search(len(r.Nodes), func(i int) bool {
return r.Nodes[i] >= hash
})If the search reaches the end of the slice, it wraps back to index zero.
The slice behaves like a circle without actually storing one.
Each node owns exactly one Redis client.
The storage layer does nothing except talk to Redis.
type Storage struct {
rdb *redis.Client
}It exposes three methods.
Set(ctx, key, value)
Get(ctx, key)
Ping(ctx)It knows nothing about routing, replication, or HTTP.
Keeping this layer small makes everything above it easier to reason about.
Each node is an independent HTTP server.
/write
/read
/replicate
/health
One mistake I made early was using Go's default HTTP mux.
http.HandleFunc(...)That registers handlers globally.
With three nodes in one process, every node would share the same routes.
The fix was giving every node its own ServeMux.
mux := http.NewServeMux()
server := &http.Server{
Addr: addr,
Handler: mux,
}Each node runs independently in its own goroutine.
go node1.Start()
go node2.Start()
go node3.Start()The router is intentionally simple.
It has two things.
type Router struct {
ring *hash.Ring
nodes []*node.Node
}For every read or write request it:
- It Reads the key.
- Looks up the owner in the hash ring.
- Forwards the request to that node.
- Returns the node's response unchanged.
The router stores no data.
That means multiple routers can exist without coordinating with each other.
As long as they all use the same ring, every router makes the same routing decision.
A write follows two steps.
First the primary node stores the value locally.
Client
│
▼
Router
│
▼
Primary
Then the primary sends the same write to every configured replica.
Primary
│
└────► Replica
In this project the replication topology forms a ring.
node1 → node2
node2 → node3
node3 → node1
Each write therefore exists on two nodes.
The write handler returns a response containing both the primary node and the replication results.
{
"primary_node": "node2",
"replications": [
{
"node": "node3",
"status": "ok"
}
]
}If replication fails, the client still receives a successful write, but the response clearly reports which replica failed.
That makes the failure visible instead of hiding it behind a log entry.
The node waits for replication before replying.
write
│
▼
primary
│
▼
replica
│
▼
client
This increases write latency because another network hop is required.
The benefit is stronger durability.
If the client receives a successful response, the data exists on both the primary and its replica.
The alternative is asynchronous replication.
write
│
▼
primary
│
├──► client
│
└──► replica
That is faster.
It also creates a window where the primary can fail before replication finishes.
Neither approach is universally correct.
The choice depends on whether latency or durability matters more.
Reads are much simpler.
The router hashes the key exactly the same way.
The request reaches the primary node.
The node fetches the value from Redis and returns it.
Because the hash function is deterministic, the same key always maps to the same owner.
hash("user:42")
│
▼
node2
Every router reaches the same conclusion.
Routing only works if the router knows which nodes are alive.
Every node continuously checks its Redis instance.
ticker := time.NewTicker(3 * time.Second)Every three seconds it performs a Redis ping.
If the ping succeeds, the node is marked alive.
If it fails, the node is marked unavailable.
The alive flag is protected by a mutex because it is accessed from multiple goroutines.
aliveMu sync.RWMutexThe router never checks Redis directly.
Instead, the hash ring skips nodes whose health check has marked them unavailable.
node1 ✓
node2 ✗
node3 ✓
If the key belongs to node2, the lookup simply continues clockwise until it finds a healthy node.
The router exposes one additional endpoint.
GET /status
Example response:
{
"nodes": [
{
"id": "node1",
"alive": true
},
{
"id": "node2",
"alive": false
},
{
"id": "node3",
"alive": true
}
],
"ring": {
"total_vnodes": 300
}
}This provides a quick snapshot of the cluster without inspecting individual nodes.
This implementation intentionally leaves out many problems that production systems solve.
- No rebalancing
- No read repair
- No quorum
- No consensus