Three ideas from this episode that are easy to half-understand. Here is a little more about each.

1 · What the CAP theorem really says

In 2000, the computer scientist Eric Brewer gave a talk suggesting that a system that stores data on several machines cannot promise everything at once. Two researchers, Seth Gilbert and Nancy Lynch, proved a precise version of the claim in 2002. That version is the “theorem”.

The proof is simple enough to picture. Two machines each hold a copy of the same data, and the network between them breaks. A user writes a new value to the first machine. Then another user asks the second machine for that value. The second machine has two options. It can answer with what it has, which is now out of date. Or it can refuse to answer until it hears from the first machine, which it can't do. There is no third option, because the new value simply has not reached it.

That is the whole theorem. Answering with old data gives up consistency. Refusing gives up availability. And the word “partition” only names the broken network that forced the choice.

Why “pick any two” is the wrong summary

The slogan suggests you choose two of three, as if you could simply skip partition tolerance. In practice you can't. Networks fail whether or not your design allows for it, so a system that ignores partitions is not safe from them. It just has no plan for them. Brewer said this himself in 2012, and he added a second point. Since partitions are rare, a good design does not give up anything most of the time. It decides ahead of time what to do when a partition starts, limits what it allows while the partition lasts, and repairs the damage once the network is back.

The same article makes a third point. The choice does not have to be the same for the whole system. A link shortener can keep redirecting while it refuses to create new aliases it can't check. It can even choose differently for different users.

One caution: the words are narrower than they sound. “Available” in the theorem means every working machine gives a real answer, not that the service feels fine to a user. And a label like “this database is CP” is usually too blunt, because many databases let you pick per request.
Where this comes from:

2 · Two meanings of “consistency”

Database people use the word “consistency” for two different things, and mixing them up causes real confusion.

The “C” in ACIDThe “C” in CAP
It is aboutA transaction on a databaseSeveral copies of the same data
It meansAfter a transaction, the data still follows the rules you set: unique values stay unique, references still point to something realEvery read sees the latest completed write, as if there were only one copy
Who guarantees itPartly the database, partly your own rules and codeThe way the copies talk to each other
In the shortenerNo two rows in one table have the same aliasEast and West both agree who owns /sale

A single database on one machine can have the ACID kind and never face the CAP kind, because there is only one copy. A system with many copies can have the CAP kind and still let a transaction break a rule if nobody wrote the rule down.

There is a third use of the word too. “Eventually consistent” means the copies may disagree for a while but will agree once updates stop. That is a weaker promise than the CAP kind, and it is the one many systems that stay available during a partition actually offer.

The ACID acronym itself dates from the early 1980s. Martin Kleppmann, who wrote the well-known book Designing Data-Intensive Applications, points out that of its four letters the “C” depends the most on the application, since the database can only enforce the rules it has been told about.

Where this comes from:

3 · PACELC: the other 99 percent of the time

CAP only talks about the moments when the network is broken. Daniel Abadi pointed out in 2010 that this leaves out nearly all of a system's life. On a normal day nothing is broken, but there is still a trade-off, and it is between speed and consistency.

Here is why. If East and West must agree before a write is confirmed, the user waits for a round trip between two distant data centers, even when everything is healthy. That is usually tens of milliseconds, and it is a limit of physics, not of software. A message can't arrive faster than light could carry it across the distance. To be faster, the system can confirm the write straight away and update the other copy later. Then a read from the other region may be a little behind.

PACELC puts both situations in one sentence. If there is a Partition, choose between Availability and Consistency. Else, choose between Latency and Consistency.

So a system can be described by two choices, one for each half. “PA/EL” means it favours availability in a partition and low latency otherwise. “PC/EC” means it favours consistency both times. Abadi's paper gives Amazon's Dynamo and Apache Cassandra as PA/EL examples. Many modern databases let you change this per request, so treat the labels as a way of thinking, not a fixed fact about a product.

A useful habit: when you read about a database, ask two questions. What does it do when the network breaks? And what does it give up on a normal day to stay consistent, or to stay fast?
Where this comes from: