Module 9 6 lessons 82 min
Growing big
Three years of orders: the orders table has 2 billion rows, though almost every query asks about this week. The app runs in fifty cities, and forty engineers in one codebase keep waiting on each other.
In this module
5 Read · 1 Lab
- 9.1Old data you rarely readAlmost every query asks about this week. Move old orders to cheap storage, and keep what the law says you must keep.Read10 min
- 9.2Partitioning a tableSplit the orders table by month inside one database. Removing a month becomes instant instead of one huge DELETE - and why it barely speeds up a query that already has an index.Read12 min
- 9.3ShardingWhen one machine can't hold it: split the data by city across databases. What a shard key is, what you lose (a query across all cities), and why it comes last. The deep dive is in Part 2.Read15 min
- 9.4Splitting into servicesMicroservices solve a people problem. Here's how to tell when you have it, and what it costs - like following one slow order across five services.Read12 min
- 9.5Proof: clear out an old monthEvery month the oldest orders move to the archive. Remove one month from 30 million orders twice - row by row, then by dropping its partition - while new orders keep coming in. Compare the time, the new orders' speed, and the disk freed.Read8 min
- 9.6Lab: Ten years of clinic bookings400 clinics and ten years of appointments. Archive the old ones and split the rest by month - then decide whether it needs shards. (It doesn't.)Lab25 min