The design process

Most people start system design by drawing boxes: a load balancer here, a database there, a cache because caches are good. That's the wrong place to start. Boxes are answers, and you need the question first.

In this lesson we'll walk through the whole design process with one example. We'll pick a single requirement and follow it from the first conversation all the way down to a line of code.

The three stages

The design process has three stages, and each one zooms in a little more.

Here's a simple way to hold it in your head: requirements tell you what, high-level design tells you where, and low-level design tells you how.

Building a house works the same way. The brief says what the family needs, the floor plan shows where each room goes, and only when you zoom into one room do you decide how it works: where the desk goes, and how many sockets it needs.

Software design zooms in the same way: first what, then where, then how.

The first stage is the one people rush. Fred Brooks, who wrote The Mythical Man-Month book, said the hardest part of building software is "deciding precisely what to build"¹: in other words, getting the requirements right. That's still true. Most bad designs aren't bad engineering. They're good engineering aimed at the wrong problem.

It matters even more now that AI can write code. AI will build whatever you describe, and build it fast. But it doesn't know your business, your users or your limits, and it won't ask about the things you forgot to mention. Deciding what to build is still the engineer's job, and it's becoming the most valuable part of it.

Our example: a food delivery app

It's 8pm on a Friday. Thousands of hungry people across the city open the app at the same time, browse menus, fill their carts and tap Pay. On a slow network, some of them tap it twice.

We will design the system for this app, something like Swiggy or Zomato, through the rest of the course. In this lesson we look at the whole thing from far away. In the next modules we'll go deeper, starting with one server.

Stage 1: Requirements

Start by writing down what the system must do. These are the functional requirements:

Then write down how well it must do it. These are the non-functional requirements:

Non-functional requirements usually shape the design more than functional ones. Lots of apps let you pick something and pay. Very few have to do it for a whole city at dinner time, without ever charging anyone twice.

Add some rough numbers too. Say our app gets 200,000 orders a day, and most of them come between 8 and 10pm. That sounds like a lot, so let's do the maths:

One ordinary server with one database can handle that. How do we know? A typical request keeps a server busy for only a few milliseconds, so one ordinary server can answer hundreds of them every second. These numbers can be rough. Their job is to tell you how big the design needs to be, and here the answer is: not very big. Don't worry if this feels like guesswork for now. In the coming modules, we'll load-test a real server and see exactly where it breaks.

Now pick one requirement to follow. We'll take the bold one: a customer is never charged twice for one order. Call it our thread, and watch what it does at every stage.

Stage 2: High-level design

You'll often hear this stage called HLD, short for high-level design. Now you can draw boxes. But there's one rule: every box has to point back to a requirement.

The numbers told us one server is enough, so here's our design. Each box shows what it does, and, in purple, the requirement that put it there:

Three boxes. That's a real design, and for this app, a good one.

Every box has a reason. Whenever you look at a design, point at any box and ask:

If nobody can answer, the box probably shouldn't be there. Try it:

Now look for our thread, "never charge twice". It didn't add a box. Instead, it changes how the boxes we already have must work.

First, the database is the one place that knows whether an order is paid. Before charging, the server always asks the database, never a copy kept somewhere else. If it asked an old copy that said "not paid", we could charge again. A database that always gives the latest answer is called strongly consistent, and a single database like Postgres gives you that for free.

Second, the server sends the order ID every time it asks the payment provider to charge. On a slow network, the app may send the same "pay" twice. Both requests have the same order ID, so the payment provider can tell the second one is a repeat and charge only once.²

So our thread made two design decisions:

Here's the whole design, with the wiring:

Stage 3: Low-level design

You'll often hear this stage called LLD, short for low-level design. Now zoom into one part of the server: taking a payment. This is where you decide how things actually work.

When it comes to paying, an order can be in one of three states:

Here's how it moves between them:

Now the tricky case. The customer taps Pay, nothing seems to happen, so they tap it again. Two "pay" requests for the same order arrive in the same millisecond, and only one of them may charge.

You let the database decide, by only moving the order to paying if it's still pending:

UPDATE orders
SET status = 'paying',
    payment_started_at = now()
WHERE order_id = :order_id
  AND status = 'pending';

If this updates one row, this request gets to call the payment provider. If it updates zero rows, another request is already paying, so this one just waits for that result.

This pattern is called a conditional update³: change it only if it's still what you expect. We'll come back to it later in the course, when we look at what happens when many requests change the same data at once.

That's our thread again. It started as a sentence in a meeting, became a design decision, and is now one WHERE clause. That's the whole process, end to end.

One more thing: low-level design often sends you back up. Say you test this at dinner-time load and find that orders get stuck in paying when the payment provider is slow. You don't patch it here. You go back to the high-level design and decide who checks on stuck payments. Sometimes you go all the way back to the requirements and ask, "How long can an order wait for payment before we cancel it?"

That's normal. The process is a loop, not a one-way line.

Putting it together

Here's the whole path of our one requirement:

1. Requirement: a customer is never charged twice for one order.
2. High-level design: the database is the one place that knows whether an order is paid, and the server sends the order ID every time it asks the payment provider to charge.
3. Low-level design: an order is pending, paying or paid, and a conditional update decides which request gets to pay.

In the end, the whole requirement comes down to one line in that update: AND status = 'pending'. That's a decision you make in low-level design. Remove it, and customers can be charged twice.

You might be wondering: which type of requirement changes the design, functional or non-functional? The answer is both, and at both stages. In our app, the functional requirement "people can place an order" put a server in the high-level design, and an orders table in the low-level design. The non-functional requirement "never charge twice" decided which part knows whether an order is paid in the high-level design, and added the conditional update in the low-level design.

So functional requirements decide what you build. Non-functional requirements decide how well it has to work. Both shape every stage.

Simple is a design too

Look at what we didn't draw: no microservices, no Kafka, no ten servers. That wasn't laziness. The requirements and the numbers didn't ask for them.

Remember the idea from the first lesson:

Now you've seen it in action. The limits are real: how many users you will have, how much budget you have for this system, how big your team is, and how soon you need to ship. If you get 20 orders a second, building for a million doesn't make the design better. It makes it worse. It costs more, takes longer to build, and has more parts that can break. That's overengineering.

Programmers have a name for avoiding it: YAGNI⁴, short for "You Aren't Gonna Need It". Don't build something because you might need it one day. Build it when you actually need it.

Simple doesn't mean small, either. In 2016, Stack Overflow, one of the busiest websites in the world, ran on nine web servers and a few databases, as one app, not hundreds of services. Its engineers said it could run on a single web server if it had to.⁵

Systems grow one step at a time. Most big systems that work today started as small systems that worked. This is called Gall's law⁶, after John Gall, the writer who noticed it.

So we'll do the same. When dinner traffic grows too big for our one server, we'll add the next piece, and the numbers will show us why. That's what the rest of this course is about.

Next time you draw a box, ask what requirement put it there. If you can't answer, you're not designing yet. You're just drawing.