A distributed system is one in which the failure of a computer you didn’t even know existed can render your own computer unusable. — Leslie Lamport
Most distributed-systems incidents aren’t exotic. They come from assumptions nobody wrote down: that clocks agree, that a message arrives exactly once, that the network is reliable, that a retry is harmless. Our job is to surface those assumptions early and replace them with guarantees.
What we do
- Design from guarantees. We start by stating what the system promises (ordering, durability, consistency, availability) and under which failures. The architecture follows from that, not the other way round.
- Choose the right consistency model. Linearisable where it must be, eventually consistent where it can be, and a clearly understood boundary between the two.
- Coordination done properly. Leader election, leases, fencing tokens and idempotency keys, plus the consensus protocols underneath them, such as Raft and Paxos variants.
- Data placement. Partitioning and sharding schemes, rebalancing without downtime, and hot-key mitigation.
- Ledger and blockchain infrastructure. Nodes, indexers and services where data integrity and verifiability are the core requirement.
How we verify it
Distributed bugs rarely show up in unit tests; they live in the interleavings. We use deterministic simulation testing, fault injection (network partitions, clock skew, slow disks, paused processes) and property-based tests to explore the orderings that production would otherwise find for you, usually at the worst possible moment.
Typical engagements
- Reviewing an existing architecture before a scale-up, a new region or a new consistency requirement
- Designing and implementing a new replicated service in Rust
- Root-cause analysis of split-brain, lost-write or duplicate-processing incidents, and fixing the design flaw behind them