That second half is what the interviewer is actually grading. Here's how a strong candidate walks through it: requirements, then the algorithm with honest trade-offs, then the distributed parts that turn a toy into a real design.

Step 1: Pin down the requirements (3-5 minutes)

Don't start drawing. Ask questions that change the design:

1. Client-side or server-side? Assume server-side. You can't trust clients to throttle themselves.
2. Throttle by what? User ID or API key for logged-in traffic, IP as the fallback. Many users can share one office or mobile-carrier IP.
3. What are the rules? For example: 100 requests per minute per user, plus a stricter 5 login attempts per 15 minutes per account. Rules differ per endpoint, so they should be configuration, not code.
4. How strict? Is it acceptable if a user occasionally gets 105 requests through instead of 100? For most APIs, yes. For OTP sends or payment attempts, maybe not. This one answer decides a lot.
5. Where does it run? As middleware in each service, or centrally at the API gateway?
6. Scale? Rough users, peak requests per second and number of API servers.

Then say your assumptions out loud: "Distributed limiter at the gateway, keyed by user ID with IP as fallback, rules stored in config, a small amount of over-admission is acceptable for normal endpoints but not for login and OTP."

Step 2: Pick an algorithm, and say why

There are four algorithms worth knowing. You don't need to implement all of them, but you should be able to compare them in one breath.

Fixed window counter

Split time into fixed windows (say, each calendar minute). Keep one counter per user per window. Increment on each request; reject once it passes the limit.

• Good: dead simple. One integer per user. In Redis it's an INCR plus an EXPIRE.
• Bad: the boundary problem. With a limit of 100 per minute, a user can send 100 requests at 10:00:59 and another 100 at 10:01:00. That's 200 requests in about one second, and both windows look fine.

Sliding window log

Store a timestamp for every request. On each new request, drop timestamps older than the window, count what's left, and allow the request only if the count is under the limit.

• Good: exact. No boundary problem at all.
• Bad: memory. A limit of 1,000 requests per hour means up to 1,000 timestamps per user. Multiply by a large user base and it gets expensive. In Redis this is usually a sorted set, so each check costs a removal plus a count instead of one increment.

Sliding window counter

The practical middle ground. Keep the counter for the current fixed window and the previous one, and estimate the rolling count with a weighted sum:

estimated = current_count + previous_count × (fraction of previous window still inside the rolling window)

Say the limit is 100 per minute, it's 15 seconds into the current minute, the previous minute had 80 requests and the current one has 30. Then 45 seconds of the previous window still fall inside the rolling 60 seconds, so the estimate is 30 + 80 × 0.75 = 90. The request is allowed.

• Good: two counters per user, and it smooths out the boundary spike.
• Bad: it's an approximation. It assumes requests in the previous window were evenly spread. Fine for most APIs, not something to use where the limit is a hard contract.

Token bucket

Each user has a bucket with a capacity (say 10 tokens) that refills at a steady rate (say 1 token per second). Each request takes one token. No token, no request.

You don't need a background job to refill buckets. Store two values, tokens and last_refill_time, and compute the refill lazily when a request arrives:

now = current_time()
tokens = min(capacity, tokens + (now - last_refill_time) × refill_rate)
last_refill_time = now
if tokens >= 1:
    tokens -= 1
    allow()
else:
    reject()

• Good: allows short bursts while enforcing a long-run average. That matches real behaviour: a page load fires several calls at once, then nothing. Small state.
• Bad: two parameters to tune instead of one. If you set the capacity too high, bursts can still hurt a fragile downstream service.

The leaky bucket queues requests and drains them at a fixed rate. It smooths traffic rather than rejecting it, at the cost of queueing delay. Mention it and move on.

The comparison you should be able to say out loud

A good answer sounds like: "For the general API I'll use a token bucket, because it tolerates normal bursts and keeps state small. For login and OTP, where the limit is a security control and volume is low, I'll use a sliding window log, because exactness matters more than memory there."

That one sentence shows you know there isn't a single right answer.

Step 3: Where do the counters live?

With one server, an in-memory map is enough. With twenty API servers behind a load balancer, in-memory counters break: each server sees only part of a user's traffic, so the user effectively gets twenty times the limit.

The standard answer is a shared, fast store, usually Redis:

• Key: something like rl:{user_id}:{rule_name}
• Value: a counter, a sorted set, or a small hash (tokens, last refill), depending on the algorithm
• TTL on every key, so idle users don't pile up in memory forever

The race condition interviewers like to poke at

Two requests from the same user hit two servers at the same moment. Both read tokens = 1, both decide to allow, both write tokens = 0. You've let through two requests on one token.

The fix is to make read-check-update atomic:

• For a fixed window, INCR is already atomic. Check the returned value instead of reading first.
• For a token bucket or sliding window, put the logic in a Lua script that Redis executes atomically, so the read, the refill calculation and the write happen as one operation.

Saying "I'd use a Lua script so the check-and-decrement is atomic" is a small sentence that signals real experience.

Scaling the store

• Shard by key. Redis Cluster spreads keys across nodes by hash slot. Each limiter key lives on exactly one node, so every check is a single-shard operation and the Lua script still works.
• Hot keys. One huge customer can overload a single shard. Give them a local pre-limiter on each gateway node, or split their budget across sub-keys.
• Latency budget. Every request now includes a network call to Redis. Keep the store close to the gateway (same region, ideally same zone) and use connection pooling.

Step 4: Failure modes (where SDE-2 answers stand out)

What if Redis is down or slow?

You have to choose:

• Fail open: allow requests when the limiter can't decide. Users aren't hurt, but you lose protection exactly when something is already wrong.
• Fail closed: reject requests. Safe for the backend, but your rate limiter has just become a single point of failure for the whole API.

A common, defensible answer: fail open for general endpoints, with a generous local in-memory limit as a backstop, and fail closed for sensitive endpoints like login and OTP. Also put a tight timeout on the Redis call (a few milliseconds), so a slow limiter can't drag down every request.

Precision vs latency

A purely local limiter needs no network call but is inaccurate when traffic is uneven across nodes. A central one is accurate but adds a hop. A middle path syncs local counts to the central store every few hundred milliseconds, trading slight over-admission for speed.

Step 5: The client-facing contract

Interviewers like candidates who remember that someone is on the other end:

• Return HTTP 429 Too Many Requests when a request is rejected.
• Include a Retry-After header so well-behaved clients know when to try again.
• Many APIs also send headers showing the limit, how many requests remain and when the window resets. Naming conventions vary, so say "remaining-quota headers" rather than inventing an exact standard.
• Log rejections with the rule that triggered them, so "your API is blocking us" takes minutes to answer, not hours.

A 35-minute plan for the interview

If the interviewer wants to go deep somewhere, follow them. The plan just stops you spending 25 minutes on the algorithm and never reaching the distributed part.

Common mistakes

• Naming an algorithm without a reason. "Token bucket" isn't an answer. "Token bucket, because bursts are normal for our clients and the state is small" is.
• Forgetting the multi-server problem. If your counters are in local memory and you have more than one server, the limit doesn't hold.
• Ignoring atomicity. Read-then-write on a shared counter is a race condition.
• Treating the limiter as infallible. It's a dependency. Decide what happens when it fails.
• One rule for everything. Login, search and file upload deserve different limits and possibly different algorithms.

Quick checklist

• Throttle key decided (user ID / API key, IP fallback)
• Rules are per endpoint and live in config
• Algorithm chosen and justified, with an alternative named
• Shared store with TTLs and atomic updates
• Fail-open vs fail-closed decided per endpoint, with a timeout
• 429 plus Retry-After, and rejections are logged

Practise this once end to end, out loud, and the question stops being scary. Thoda structure, thoda trade-off talk, and you're already ahead of most answers.

I build DevInterviewMaster, an interview-prep site for developers that teaches in English, Hindi or Hinglish. A rate limiter is one stop on a bigger path (requirements, estimation, high-level design, deep dives). If you want that whole flow laid out, the system design framework pages are free to read at devinterviewmaster.com/system-design.

Both miss the point. Estimation isn't a maths test. It's how you decide whether you need one database or twenty, whether a cache is optional or essential, and whether the bottleneck is reads, writes or storage. Done well, it takes three to five minutes and makes every later decision easier to defend.

Here's a practical way to do it.

What the interviewer is actually checking

1. Can you turn vague requirements into numbers? "A lot of users" isn't a design input. "About 1,000 writes per second at peak" is.
2. Are your assumptions explicit? Interviewers care far more that you state an assumption than that it's perfectly accurate.
3. Do you use the numbers? An estimate that never shows up again in your design is wasted time.
4. Is your order of magnitude right? Being off by 2× is fine. Being off by 1,000× changes the architecture.

Precision is not on the list. Round aggressively.

The five numbers worth memorising

You don't need a table of fifty constants. These cover most interviews.

1. Seconds in a day: 86,400, so call it ~100,000 (10^5). This one number turns "per day" into "per second". 1 million requests a day is roughly 10 requests per second. 100 million a day is roughly 1,000 per second. Using 10^5 slightly underestimates QPS, which you'll correct with a peak factor anyway.

2. Seconds in a month: about 2.5 million. Useful when the requirement is stated per month (30 × 86,400 ≈ 2.6 million).

3. Powers of two and data sizes.

So "1 billion records of 1 KB each" is about 1 TB. That shortcut alone saves a lot of fumbling.

4. Latency orders of magnitude. These are the widely cited reference figures (often called "latency numbers every programmer should know"). Exact values vary by hardware; the ratios are what matter.

• Main memory reference: ~100 nanoseconds
• Random read from SSD: ~100 microseconds
• Round trip within the same data centre: ~0.5 milliseconds
• Disk seek on a spinning HDD: ~10 milliseconds
• Round trip between continents: ~150 milliseconds

The takeaway: memory is roughly 1,000× faster than SSD, and a cross-continent call costs more than a thousand in-datacenter calls. That's why caches and regional deployments exist.

5. Peak vs average. Traffic isn't flat. A common assumption is peak ≈ 2–3× the average. For spiky Indian workloads such as a cricket final, a flash sale or exam results day, say out loud that peaks can be far higher, and design for that peak.

The four-step method

Do these in order and write each line on the board.

Step 1: Pin down the users and actions

Start from daily active users (DAU) and what each user does.

"Let's assume 50 million DAU. Each user reads about 20 items and creates about 1 item per day."

If the interviewer gives you numbers, use theirs. If not, propose round ones and ask, "Does that sound reasonable?" This turns estimation into a conversation instead of a monologue.

Step 2: Convert to QPS (reads and writes separately)

• Writes/day = DAU × writes per user
• Write QPS ≈ writes/day ÷ 10^5
• Read QPS ≈ reads/day ÷ 10^5
• Peak QPS ≈ average × 2 or 3

Keeping reads and writes separate matters. The read:write ratio is often the most useful single number in the interview. A 100:1 read-heavy system pushes you towards caching and read replicas. A write-heavy system pushes you towards partitioning, queues and append-friendly storage.

Step 3: Storage

• Storage/day = writes/day × size per record
• Total = storage/day × 365 × years of retention
• Multiply by the replication factor (3 is a common default)

Estimate record size by listing fields: an ID (8 bytes), a short URL or username (tens of bytes), a timestamp (8 bytes), some text (a few hundred bytes). Round up to a clean number such as 500 bytes or 1 KB. Media (images, video) should be estimated separately because it dominates everything else and usually goes to object storage.

Step 4: Bandwidth and memory

• Incoming bandwidth ≈ write QPS × record size
• Outgoing bandwidth ≈ read QPS × response size
• Cache size: if a small fraction of items gets most of the traffic (the 80/20 rule is a common working assumption), caching around 20% of a day's read volume is a reasonable starting estimate.

Stop there. Four steps, about five lines of maths.

A worked example: a URL shortener

Assumptions (stated aloud and agreed with the interviewer):

• 100 million new short URLs per month
• Read:write ratio of 100:1 (people click links far more often than they create them)
• Store links for 5 years
• About 500 bytes per record (short code, long URL, owner ID, timestamps)

Writes: 100M per month ÷ 2.5M seconds per month ≈ 40 writes/second on average, so about 100/second at peak.

Reads: 40 × 100 = 4,000 reads/second on average, about 10,000/second at peak.

Storage: 100M per month × 12 × 5 = 6 billion records. 6 billion × 500 bytes = 3 TB. With 3× replication, about 9 TB.

Cache: Reads per day ≈ 4,000 × 10^5 = 400 million. If we cache 20% of daily reads at 500 bytes each: 80 million × 500 bytes = 40 GB, which fits comfortably in memory across a small cache cluster.

Key space (bonus): With a base-62 alphabet (a–z, A–Z, 0–9), 7 characters give 62^7 ≈ 3.5 trillion combinations. That's far more than 6 billion, so 7 characters are plenty.

Now use the numbers:

• 100 writes/second is modest; a single primary database can usually handle this, so sharding for writes isn't the first problem.
• 10,000 reads/second at peak with a 100:1 ratio says the design should be cache-first. Most redirects should never touch the database.
• 9 TB over five years is large for a single machine but manageable with partitioning by short code, which you can plan for later rather than on day one.
• Since key space is ample, you can choose between a counter-based ID with base-62 encoding and random codes with collision checks, and discuss the trade-offs.

That is what estimation is for. Each number led to a design decision.

Common mistakes

• Doing maths with no assumptions on the board. If the interviewer can't see where 4,000 came from, they can't give you credit for it.
• False precision. Writing 1,157.4 QPS signals you're optimising the wrong thing. Say "about 1,000–1,200".
• Forgetting peak traffic. Design for the peak, not the average.
• Mixing units. Bits vs bytes and per-day vs per-second errors are the most common source of 1,000× mistakes. Write units on every line.
• Ignoring replication and indexes. Raw data is rarely the full storage cost.
• Never using the result. End the section with one sentence: "So this is read-heavy, around 10K QPS at peak, a few TB, and caching is the main lever."

How to practise (without a whiteboard partner)

1. Pick a familiar product: a chat app, a food-delivery tracker, a UPI-style payment notification service, a photo feed.
2. Set a five-minute timer and run the four steps on paper.
3. Write the one-sentence conclusion and the two design decisions it implies.
4. Compare your assumptions with a friend's. If yours differ by 10× or more, find out why.

Do this for ten systems and the steps become automatic. In the interview, you'll spend your energy on the design itself, which is where the real discussion happens. Thoda practice, and the numbers stop feeling scary.

Quick checklist

• DAU and actions per user stated and agreed
• Read and write QPS calculated separately, with a peak factor
• Storage with retention and replication
• Bandwidth and cache size, roughly
• Units written on every line
• A one-sentence summary that drives at least two design decisions

Estimation is the step between requirements and architecture, and it fits into a bigger interview framework: clarify requirements, estimate, sketch the high-level design, then dive deep. If you want a structured overview of that whole flow, the system design framework pages on DevInterviewMaster are free to read at devinterviewmaster.com/system-design.

The fix is to learn it in the right order. Design skill stacks: object design sits on principles, and architecture sits on object design and fundamentals. Here's the roadmap I'd follow, and the one DevInterviewMaster's learning path is built around: SOLID → Design Patterns → Low-Level Design → High-Level Design.

Stage 1: SOLID principles (1–2 weeks)

SOLID isn't trivia to recite. It's a vocabulary for talking about the cost of change.

• Single Responsibility: a class should have one reason to change. If your OrderService also formats emails and talks to the payment gateway, every change risks everything.
• Open/Closed: add behaviour by adding code, not by editing a growing if/else chain.
• Liskov Substitution: a subclass must honour the parent's contract. The classic square/rectangle problem is the interview favourite.
• Interface Segregation: many small interfaces beat one fat one that forces empty implementations.
• Dependency Inversion: depend on abstractions, so you can swap a real payment gateway for a fake one in tests.

Exit test: take a 200-line class you wrote and explain which principle it breaks and what that will cost you later.

Stage 2: Design patterns (2–3 weeks)

Don't memorise a catalogue. For each pattern, learn the problem it solves and its trade-off. Prioritise the ones that come up again and again in LLD rounds:

• Strategy: swap algorithms at runtime (pricing rules, payment methods).
• Factory / Abstract Factory: centralise object creation.
• Observer: notify subscribers of events (notifications, order status).
• State: behaviour that changes with state. Vending machines, elevators and order lifecycles are exactly this.
• Decorator: add behaviour without subclass explosion.
• Singleton: know it, and know why it's often a smell (global state, testing pain).
• Builder, Adapter, Command: frequent supporting cast.

Exit test: given "design a vending machine", you can say which pattern you'd use and why, in two sentences.

Stage 3: Low-Level Design (3–4 weeks)

LLD rounds ask you to turn requirements into classes, interfaces and working code. A repeatable approach:

1. Clarify use cases and invariants. "Two cars can never be assigned the same spot" is an invariant, and your design must protect it.
2. Identify entities and responsibilities. Which object owns each rule?
3. Define interfaces and relationships. Composition over inheritance by default.
4. Implement one core flow end-to-end. Interviewers want running logic, not just a class diagram.
5. Test the design against change. "Now add electric-vehicle charging spots": how many classes do you touch?
6. Discuss concurrency if relevant. Two requests reserving the same spot at once is a race condition (see stage 4 thinking).

Classic practice problems: parking lot, elevator system, vending machine, library management, splitwise, rate limiter (LLD version), LRU cache, BookMyShow-style seat booking.

Stage 4: High-Level Design (4–6+ weeks)

Now zoom out to services, data flow and failure. Use the same six steps for every question:

1. Clarify requirements: users, core actions, scale, and what's explicitly out of scope.
2. Estimate scale: QPS, storage, read/write ratio, latency targets, with units.
3. Design APIs and the data model before drawing boxes.
4. Draw the core flow: trace one request end-to-end.
5. Handle scale and failure: caching, queues, partitioning, replication, retries. Add each only when you can say why it's needed.
6. Defend trade-offs: consistency vs availability, cost vs complexity, and what would change your mind.

Fundamentals to cover: load balancing, caching strategies and invalidation, SQL vs NoSQL, indexing, replication and partitioning, consistent hashing, message queues and event-driven design, rate limiting, CDNs, idempotency, and observability.

Start with these five problems, because together they cover most building blocks: URL shortener, rate limiter, notification service, chat system, file storage.

What changes by level

• SDE-1: fundamentals, small object designs, clean code. LLD matters more than HLD.
• SDE-2: both rounds; expect deeper trade-offs and failure handling.
• Senior: HLD dominates, with emphasis on reliability, operability and cost. Real production incidents become great material here.

A realistic 12-week plan

Throughout, practise explaining out loud. System design interviews are conversations. Record yourself, or explain to a friend (or a tutor) and see where you hand-wave.

DevInterviewMaster follows exactly this path, SOLID → design patterns → LLD (70 topics) → HLD (248 topics), with an AI tutor for HLD and LLD that you can talk to in English, Hindi or Hinglish. The overview is free at devinterviewmaster.com/system-design.

The good news: most production incidents are variations of a small set of patterns. While writing up 57 incidents across databases, backend services, frontends, infrastructure and distributed systems, we kept seeing the same shapes. Here are twelve worth knowing, grouped by layer, each with the symptom, the root cause and the usual fix.

Database layer

1. The N+1 query problem

Symptom: A list endpoint that took 40 ms in development takes 4 seconds in production. CPU on the app server looks fine.

Root cause: The ORM loads a list of parent rows with one query, then lazily loads a relationship for each row, so N extra queries. With 10 rows nobody notices. With 2,000 the database is doing 2,001 round trips.

Fix: Eager loading (JOIN, select_related, include), batching, and a test that asserts the query count for critical endpoints so the regression can't sneak back.

2. Connection pool exhaustion

Symptom: The app "hangs". Requests time out, yet CPU and memory are low. Restarting fixes it for an hour.

Root cause: Connections are borrowed and not returned (a missing finally, an exception path, a long-running transaction), or a slow query holds them. Once the pool is empty, every request waits.

Fix: Always release in finally/context managers, set acquisition timeouts so callers fail fast, alert on pool utilisation, and size pools against the database's max_connections across all instances.

3. Deadlocks

Symptom: Intermittent errors like "deadlock detected" under load, usually around payments or inventory.

Root cause: Two transactions lock the same rows in opposite orders and each waits for the other.

Fix: Lock rows in a consistent order, keep transactions short, avoid user I/O inside transactions, and retry the transaction on deadlock errors.

4. Replication lag

Symptom: A user changes a setting, the page reloads, and the old value shows. Support tickets say "your app doesn't save".

Root cause: Writes go to the primary but reads go to a replica that is a few hundred milliseconds behind.

Fix: Read-your-writes consistency. After a write, route that user's reads to the primary for a short window, or use session-level consistency tokens.

Backend services

5. Memory leaks

Symptom: Memory climbs steadily for days, then the process is OOM-killed and restarts. The graph looks like a sawtooth.

Root cause: Unbounded in-memory caches, event listeners that are never removed, or growing global maps keyed by user or request.

Fix: Bound every cache (size and TTL), take heap snapshots before and after a load test, and alert on memory trend rather than just the absolute level.

6. Race conditions

Symptom: Very occasionally a coupon is redeemed twice, or a seat is booked by two users.

Root cause: A check-then-act sequence ("is it available? then reserve it") without atomicity.

Fix: Push the invariant into the database (unique constraints, conditional updates like UPDATE ... WHERE available = true), or use proper locking or idempotency keys.

7. Thread starvation

Symptom: Latency spikes across all endpoints, not just one, whenever a particular downstream service slows down.

Root cause: Blocking calls occupy every worker thread, so even cheap requests can't get a thread.

Fix: Timeouts on every outbound call, separate pools (bulkheads) for slow dependencies, or non-blocking I/O.

8. Retry storms

Symptom: A brief slowdown in one service turns into a full outage that lasts far longer than the original blip.

Root cause: Every caller retries immediately, several times. Load on the struggling service multiplies just when it can least handle it.

Fix: Exponential backoff with jitter, retry budgets, and circuit breakers that stop calling a failing dependency for a while.

Frontend

9. Infinite re-renders

Symptom: The tab freezes and the fan spins. React warns "Maximum update depth exceeded".

Root cause: State updated inside an effect whose dependency changes on every render (often a new object or array literal).

Fix: Stable dependencies (useMemo/useCallback), and derive values during render instead of syncing them into state.

10. Hydration mismatch

Symptom: A server-rendered page flickers, or console warnings say the server and client HTML differ.

Root cause: Rendering something non-deterministic on the server (dates, Math.random(), window checks) that comes out different on the client.

Fix: Render deterministic markup on the server and move client-only values into effects.

Infrastructure and distributed systems

11. DNS and config drift

Symptom: One region or environment behaves differently, or a deploy works in staging and fails in production.

Root cause: Manually changed config, stale DNS records or TTLs, and environments that slowly diverge.

Fix: Infrastructure as code, config in version control, drift detection, and sensible DNS TTLs before planned migrations.

12. Split brain and cascading failures

Symptom: After a network partition, two nodes both accept writes as "leader", and data diverges. Or one failing dependency takes down services that were healthy.

Root cause: Leader election without quorum, missing timeouts, and no isolation between components.

Fix: Quorum-based consensus (Raft and similar), fencing tokens, timeouts everywhere, bulkheads, and graceful degradation.

How to use this in an interview

When an interviewer asks about a production bug, walk through it in this order:

1. Symptom and impact: what users saw, how many, for how long.
2. First hypothesis, and why it was wrong: this makes the story credible.
3. How you found the real cause: metrics, logs, traces, a reproduction.
4. The fix: the immediate mitigation, then the real fix.
5. Prevention: tests, alerts or guardrails so it can't recur.

Even if you've never been on-call, you can study these patterns well enough to discuss them honestly ("I haven't hit this in production, but here's how I'd detect and fix it"). Interviewers respect that far more than a vague story.

We wrote all 57 incidents up, each with root-cause analysis, buggy vs fixed code and prevention steps, on DevInterviewMaster. The first two in the Database, Backend and Frontend categories are free to read. Which classic incident would you add to this list?