CAP Theorem and
Why You Can't Have It All
When the network between two parts of a system breaks, the system has to pick between answering every request and giving only correct answers. This episode explains that choice, why it is often misunderstood, and what to think about on the days when nothing is broken.
What this episode covers
- A real story: in 2018, GitHub lost the connection between its East Coast and West Coast data centers for 43 seconds. The site then ran in a reduced state for more than a day.
- The shortener split in two: what goes wrong when the same short link is created on both sides of a broken connection.
- The three letters of the CAP theorem, defined carefully: Consistency, Availability and Partition Tolerance. Each one means something narrower than it sounds.
- The “pick any two” myth. You don't get to choose whether the network breaks. The real choice is what to give up while it is broken.
- The choice does not have to be the same for everything. A system can keep redirecting links while still refusing to create new ones it can't check.
- PACELC: what the trade-off looks like on normal days, when nothing is broken but every request still has to wait on something.
Extra readings
A short article (about six mins read) that goes a little further than the video, in three parts: what the CAP theorem really says, the two meanings of “consistency”, and what PACELC adds.
Exercises
The coding exercise is the one to prioritize. It makes you choose between refusing a request and answering with old data, and writing that choice in code makes it much easier to remember.
1 · Written problem
Your shortener runs in two regions, East and West. A network partition cuts them off from each other for ten minutes. During that time the service also counts clicks on every short link, and it lets people create custom aliases that must belong to exactly one owner. (This is an imagined scenario made for the exercise, not a real incident.)
- For the click counter, should each region keep counting during the partition, or refuse to count until the connection is back? Pick one and explain briefly.
- For custom alias creation, should each region keep accepting new aliases during the partition? Pick one and say what could go wrong with the other choice.
- Now imagine there is no partition. A developer suggests waiting for both regions to confirm every alias creation before replying to the user. Using PACELC, explain what this gives you and what it costs.
Model answer
a) Keep counting on both sides, which favours availability. A click count can be slightly out of date without much harm. Refusing to count would lose clicks for good, and that is worse than two numbers that don't match for a while. When the regions reconnect, the two counts can simply be added together. Any answer that picks a side and says “a wrong count is easy to fix, a lost click is not” is a good one.
b) Refuse to create aliases on the side that can't check with the other one, which favours consistency. If both sides accept, the same alias could end up owned by two people and pointing at two different pages. After the connection returns, someone's link would silently be wrong. Accepting on both sides can still be defended, but only if the team has decided in advance how conflicts get settled (for example, the later one wins, or both owners are told) and accepts that one owner loses their alias.
c) It gives you consistency: both regions always agree, so nobody ever reads an out-of-date alias, even on a normal day. It costs you latency: every creation has to wait for a round trip between the regions, and that wait never goes away, even when nothing is broken. In PACELC terms, this is the “else” half, choosing consistency over latency.
2 · Coding exercise - refuse or answer stale
~20–30 minutes · JavaScript
You will build a tiny alias store that runs in two regions, "east" and
"west". It can run in two modes:
- CP mode: while the regions are cut off from each other, West
refuses to create or look up aliases and throws
Unavailable. East keeps working. When the connection is back, West catches up to East. - AP mode: while the regions are cut off, both keep answering from
their own table, even though the data may be out of date. When the connection is
back, the two tables are merged. If the same alias points to different URLs on the
two sides, the one created later wins, and
heal()returns the list of aliases that conflicted.
Fill in the Cluster class below. Four tests run against your implementation. You can't see or edit the tests themselves, but their names tell you what they
check.
Hint, if you're stuck
Keep one table per region, and give every create a number that goes up by one each
time. Then “the later create wins” just means “the higher number wins”. One small
helper that decides whether a region should refuse a request in CP mode keeps
create and resolve short.
Click "Run tests" to check your implementation.
Reference solution
class Unavailable extends Error {
constructor(message) { super(message); this.name = "Unavailable"; }
}
class Cluster {
constructor(mode) {
this.mode = mode;
this.partitioned = false;
this.data = { east: new Map(), west: new Map() }; // alias -> { url, seq }
this.seq = 0;
}
partition() {
this.partitioned = true;
}
check(region) {
// only CP mode refuses, and only on the west side
if (this.mode === "CP" && this.partitioned && region === "west") {
throw new Unavailable(region);
}
}
create(region, alias, url) {
this.check(region);
if (this.data[region].has(alias)) return false;
const entry = { url, seq: ++this.seq };
this.data[region].set(alias, entry);
if (!this.partitioned) {
const other = region === "east" ? "west" : "east";
this.data[other].set(alias, entry); // copied straight away
}
return true;
}
resolve(region, alias) {
this.check(region);
const entry = this.data[region].get(alias);
return entry ? entry.url : undefined;
}
heal() {
this.partitioned = false;
const { east, west } = this.data;
const conflicts = [];
const aliases = [...new Set([...east.keys(), ...west.keys()])].sort();
for (const alias of aliases) {
const a = east.get(alias), b = west.get(alias);
if (a && b && a.url !== b.url) conflicts.push(alias);
const winner = !a ? b : !b ? a : a.seq > b.seq ? a : b; // later create wins
east.set(alias, winner);
west.set(alias, winner);
}
return conflicts;
}
}
The key idea is that one function decides whether to refuse. In CP mode West says
“I can't check with East, so I won't answer”. In AP mode nobody refuses, so West
may answer with old data, and the merge on heal() cleans up the
conflicts afterwards. That is the same choice the episode describes, written as code.
3 · Quiz
- In the CAP theorem, what does the word “consistency” mean?
Answer
B.In CAP, consistency means every read sees the latest write. It is not the “C” from ACID, and it is not the same as copies agreeing “eventually”.
- Why is “pick any two of C, A and P” misleading?
Answer
C.Partitions happen to you. So the decision is what to give up while one is happening. When there is no partition, you can have both.
- During a partition, a region refuses to create custom aliases because it can't check with the other region. Which property is it giving up?
Answer
C.It returns an error to keep each alias owned by just one person. That protects consistency and gives up availability.
- What does PACELC add that CAP alone doesn't cover?
Answer
B.The “else” half of PACELC covers normal days, when waiting for other copies to confirm makes every request slower.
- The shortener keeps answering redirects (possibly with old data) but refuses to create aliases during a partition. What does that show?
Answer
D.Eric Brewer made this point in 2012: the choice can be different for each kind of operation, each kind of data, or even each user.