Redis is one of those tools most backend developers use and few have looked inside. You type SET greeting hello EX 60, the key is there, and a minute later it is gone. Between those two moments sit a wire protocol, a keyspace, an expiry algorithm and a durability story. Each is a topic you can read about for hours without really understanding it.
I wanted to understand them by writing them. redis-clone is a small Redis-compatible server in Go. It speaks the RESP2 protocol, keeps a keyspace with strings, lists and hashes, expires keys, and persists data in two ways: an append-only file (AOF) with rewrite, and snapshots. Real Redis clients work with it, and the tests talk to it through go-redis.
It is a learning project, not a Redis replacement. This article is about what is in it and what each part teaches.
What it does
The server supports a practical set of commands:
| group | commands |
|---|---|
| connection | PING, ECHO, QUIT, SELECT 0, CLIENT (no-op), HELLO (refused, so clients fall back to RESP2), COMMAND, INFO |
| keys | GET, SET (EX, PX, EXAT, PXAT, NX, XX, KEEPTTL, GET), DEL, EXISTS, TYPE, KEYS (glob), DBSIZE, FLUSHALL/FLUSHDB |
| counters | INCR, DECR, INCRBY, DECRBY (with overflow checks) |
| expiry | EXPIRE, PEXPIRE, EXPIREAT, PEXPIREAT, TTL, PTTL, PERSIST |
| lists | LPUSH, RPUSH, LPOP/RPOP (with count), LLEN, LRANGE, LINDEX |
| hashes | HSET, HGET, HDEL, HGETALL, HLEN, HEXISTS, HKEYS, HVALS, HINCRBY |
| persistence | SAVE, BGSAVE, BGREWRITEAOF |
The code is split into four folders. resp/ has the RESP2 reader and writer; the reader also accepts inline commands such as a plain SET a b line. store/ is the keyspace: a map of entries, a map of expiry times in milliseconds, and a glob matcher for KEYS. server/ has the TCP accept loop, one goroutine per connection, the command table, the cron (active expiry, the every-second fsync, auto-rewrite, periodic save) and the AOF and snapshot code. cmd/ is the binary.
What I built from scratch, and what it teaches
Wire protocols. RESP2 is a streaming, length-prefixed protocol. The lesson is in the word "streaming": a TCP read can end in the middle of a command, so the parser must not lose data across reads. Clients also pipeline, which means they send several commands before reading any reply. In this server, replies are buffered and only flushed once the client's pipeline is drained.
Single-threaded execution. Redis runs commands on one thread, which is why its data structures need no locks. I reproduced that with one mutex: each connection reads commands and calls Server.Exec, which holds the server mutex for the duration of one command. Many connections, one command at a time.
Expiry. There are two mechanisms working together. Lazy expiry checks a key's TTL every time it is accessed. That alone would leave dead keys in memory forever if nobody reads them, so there is also Redis's probabilistic active expiry cycle: sample 20 keys that have a TTL, delete the expired ones, and repeat while more than 25% of the sample was expired.
Durability. This is the largest part. Write commands append their RESP encoding to appendonly.aof. Relative expiries are rewritten as absolute ones (SET k v PXAT …, PEXPIREAT), so replaying the log days later still gives the right answer, and keys that expire are logged as DEL. On startup the log is replayed through the normal command table. If the server crashed in the middle of a write, the partial final command is truncated away.
A log that only grows needs compaction. For an AOF rewrite, the keyspace is deep-copied under the lock, then a goroutine writes the minimal commands (SET, RPUSH, HSET, PEXPIREAT) to a temp file. Writes that arrive in the meantime are kept in a rewrite buffer and appended when the goroutine finishes, and the temp file is atomically renamed over the log.
Snapshots use a custom binary format: an RCDB0001 magic header, typed records, uvarint lengths and a CRC-32 trailer. The file is written to a temp file, fsynced and renamed, so a reader never sees half a snapshot. On startup, the AOF is used if it is enabled and present; otherwise the snapshot is.
How to use it
Install the server binary with Go:
go install github.com/AbdeldjalilChougui/redis-clone/cmd/redis-clone@latest Or clone the repository and run it from source, on a port that does not collide with a real Redis:
go run ./cmd/redis-clone -addr :6380 -dir ./data redis-cli -p 6380 set greeting hello EX 60 redis-cli -p 6380 ttl greeting The flags follow Redis's own names: -aof (on by default), -appendfsync always|everysec|no, -auto-aof-rewrite-percentage 100, -auto-aof-rewrite-min-size 67108864 and -save-interval 5m.
The fsync policy is the classic trade-off. always syncs every write to disk and is the slowest. everysec syncs once a second, so a crash can lose about a second of writes. no leaves it to the operating system.
To run the tests:
go test -race ./... They need no external services. The server tests start the server on a random port and talk to it through go-redis, and a fake clock drives TTLs and restarts.
Things to know
The README is direct about the limits, and they are worth reading because each one points at something real Redis does better:
- Only RESP2, a single database (index 0), and three data types. There are no sets, sorted sets, streams, pub/sub, transactions (MULTI/EXEC), Lua, replication, auth or cluster.
- The snapshot format is its own format. It is not RDB-compatible, so you cannot load these files in Redis or the other way round.
- Background saves and rewrites deep-copy the keyspace while holding the lock. Redis uses
fork()and copy-on-write instead. With a large dataset, the copy pauses clients for its duration. - The
everysecfsync runs while holding the command lock, so a slow disk stalls commands for that moment. - Lists are Go slices, so
LPUSHis O(n) instead of O(1). - Some error messages and edge cases differ from real Redis.
What I learned
The first surprise was how much of Redis is about time. A relative TTL is easy in memory and wrong in a log: if you store "expire in 60 seconds" and replay it tomorrow, the key comes back to life. Rewriting every expiry as an absolute timestamp before logging it is a small rule with a big effect.
The second was that durability is mostly about the bad moments: a crash mid-write, a rewrite racing with new commands, a snapshot half on disk. The happy path is an append. The work is in truncating the torn tail, keeping a rewrite buffer, and using write-temp, fsync, rename so that a file is either the old version or the new one.
The third was that the limitations teach as much as the features. Writing the deep copy made it clear why Redis reaches for fork(): the operating system gives it a consistent view of memory almost for free, while my copy has to stop the world.
Get the code
The source and the tests are on GitHub: https://github.com/AbdeldjalilChougui/redis-clone. It is MIT licensed. If you have ever wondered what happens between SET and the disk, reading server/ next to the Redis documentation on persistence is a good afternoon.
Originally published at abdeldjalilchougui.biz.


