System Design Interview: How to Design a Rate Limiter (Algorithms, Redis and Trade-offs)

By Preeti Verma

"Design a rate limiter" is one of the most popular system design questions because it is small enough to finish in a single interview and deep enough to show how you think. It touches algorithms, distributed systems, and failure handling.

This walkthrough follows the order you would use in the interview. If system design is new to you, start with HLD vs LLD and a simple way to start.

Step 1: Clarify the requirements

Ask before you draw anything.

Functional

  • What are we limiting? Requests per user, per API key, per IP, or per endpoint?
  • What are the limits? For example 100 requests per minute per user, with different limits for free and paid plans.
  • What happens when the limit is exceeded? Usually the request is rejected with HTTP 429 Too Many Requests.
  • Is it a hard limit, or can short bursts be allowed?

Non-functional

  • It must be very fast, because it runs on every request. A limiter that adds 50 ms is itself a problem.
  • It must work across many servers, so counts are shared.
  • It must be highly available: a failing limiter should not take the whole API down.
  • It should be reasonably accurate, but perfect precision is rarely needed.

Step 2: Where does it live?

Options:

  • In the application code. Simple, but each server counts alone unless you share state, and every service has to implement it.
  • As middleware or an API gateway. The limiter sits in front of your services, so it is applied consistently in one place. This is the most common answer.
  • At the edge or CDN for coarse protection against abuse.

Say you would put it in the gateway, with shared state in a fast store such as Redis.

Step 3: Choose an algorithm

This is the heart of the question. Compare honestly.

Fixed window counter

Count requests in each calendar window (for example each minute). If the count passes the limit, reject.

  • Pros: very simple and cheap (one counter per key).
  • Cons: allows a burst at the boundary: a user can send the full limit at the end of one minute and the full limit again at the start of the next, doubling the rate for a moment.

Sliding window log

Store a timestamp for every request, and count those in the last N seconds.

  • Pros: accurate.
  • Cons: uses a lot of memory, because you store each request.

Sliding window counter

Combine the current and previous fixed windows, weighting the previous one by how much of it still overlaps the sliding window.

  • Pros: smooth, low memory, good approximation.
  • Cons: an approximation, not exact.

Token bucket

A bucket holds up to capacity tokens and refills at a steady rate. Each request takes one token. If the bucket is empty, the request is rejected.

  • Pros: allows controlled bursts (up to the bucket size) while enforcing an average rate. Memory is tiny: a token count and a last-refill time per key.
  • Cons: needs two numbers per key and careful atomic updates.

Leaky bucket

Requests enter a queue and are processed at a fixed rate. It smooths traffic into a steady flow.

  • Pros: predictable output rate.
  • Cons: bursts fill the queue, and old requests can wait or be dropped.

A good final answer: "I would use a token bucket because it allows short bursts, which real users produce, but still enforces an average rate. It also needs very little memory per user."

Step 4: Make it work across many servers

If each server keeps its own counters, a user whose requests are spread across servers gets many times the limit. So the counters must be shared.

Use Redis:

  • Key per user and window, for example rate:{userId}:{minute}.
  • A simple fixed window uses INCR and sets an EXPIRE the first time.
  • For a token bucket or sliding window, store the state in a hash or sorted set.

The race condition

If the application reads the count, checks it, then writes it back, two servers can both read the same value and both allow a request. The fix is to make the check-and-update atomic: use a single Redis command such as INCR, or run the logic in a Lua script, which Redis executes without interruption. Mentioning this shows real understanding.

Step 5: The response

When a request is rejected, return 429 Too Many Requests with helpful headers:

  • Retry-After: how long to wait.
  • X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset: so clients can slow down before they hit the wall (the exact header names vary between APIs).

Good clients should back off with increasing delays, not retry in a tight loop.

Step 6: What if Redis is down?

This is the question that separates good answers. You have two choices:

  • Fail open: let requests through when the limiter cannot decide. The API stays available, but you lose protection for a while.
  • Fail closed: reject requests. You stay protected, but a limiter outage becomes a full outage.

Most public APIs fail open for general limits and fail closed for sensitive endpoints such as login or payments. Say which you pick, and why. Also mention running Redis with replicas, and a short local fallback limit as a safety net.

Step 7: Different limits for different callers

Store limits in configuration, not in code: per plan, per endpoint, per user. Let the gateway look up the limit for each caller. Cheap endpoints can have higher limits than expensive ones such as search or report generation.

Step 8: Scaling and accuracy trade-offs

  • Latency: keep the Redis call to one round trip per request. Place Redis close to the gateway.
  • Hot keys: one very busy user concentrates load on one Redis key. A single key is still handled well, but you can shard by key across nodes.
  • Multiple regions: a global limit needs synchronisation, which is slow. A common choice is a local limit per region, accepting some inaccuracy.
  • Clock differences: do not depend on every server having the same time. Let Redis supply the time, or use its own expiry.

Common mistakes in this interview

  • Jumping to Redis without choosing and justifying an algorithm.
  • Ignoring the boundary burst problem of fixed windows.
  • Forgetting the read-then-write race.
  • Not saying what happens when the limiter fails.
  • Rate limiting only by IP, which punishes many users behind one office or mobile network.

A short summary you can say out loud

"I would run a token bucket per user in the API gateway, with counters in Redis updated atomically through a Lua script. Rejected requests return 429 with a Retry-After header. If Redis fails, general endpoints fail open and sensitive ones fail closed. Limits come from configuration per plan."

Related reading: Redis and caching interview questions, REST API design questions and our walkthrough of designing a URL shortener.

Practise this on GenQuest

  • Start a voice mock interview: answer out loud to an AI interviewer that knows your resume and the role, get follow-up questions, then read a scored report with a question-by-question review. Your first 15-minute interview is free.
  • Open the question banks: Core Java scenarios, Blind 75 and NeetCode 150, high-level and low-level design questions, and company-wise sheets, with a free preview and progress tracking.
  • Ask the AI Coach: paste any question you are stuck on and get it explained in plain words, with a short answer you could say in an interview. See how it works in our AI Coach introduction.

FAQ

Which algorithm should I pick if the interviewer does not specify? Token bucket is a safe default. Say why, and mention one alternative with its trade-off.

How accurate must a rate limiter be? Usually "close enough" is acceptable. Perfect accuracy across regions is expensive, so state the trade-off.

Can I rate limit without Redis? Yes, for a single server, with an in-memory structure. With several servers you need shared state.

The takeaway

Clarify requirements, pick an algorithm and defend it, make counting atomic and shared, handle failure on purpose, and return clear responses. That is a complete answer, and it shows the engineering judgement interviewers look for.