Rate Limiting

While an API — the agreed window programs use to exchange things — was simply left open, the side receiving requests had no say in how many arrived. Opening the window takes a few lines of code; 300 requests per second arriving through it comes from somebody else's circumstances.

In practice the failures came from mistakes rather than attacks. Response time stretches to four seconds at two in the morning, the alert fires, and the cause turns out to be one account calling the document list several hundred times a second. Their retry logic went wrong and repeated the same request without stopping. No malice required — the outcome is the same, and 300 other customer organizations have a slow product for the duration.

The asymmetry of effort is part of it. One request is nearly free for the caller. Changing a single loop multiplies calls by ten thousand. On the receiving side that ten thousand shows up as server bills and response times. Rate limiting converts that asymmetry into a rule the receiving side gets to set, written down where both sides can read it.

What rate limiting solves is one caller's volume degrading or stopping everything else. Four things get prevented.

1. Capacity protection — requests beyond what can be processed get cut at the door. Rejecting the excess while serving the rest beats a server that buckles and fails everything.
2. Dividing the share — per-account caps stop one customer from eating another's capacity. Without them the busiest organization takes the most resources and the quiet ones pay for it, usually without ever learning why their morning was slow.
3. A ceiling on cost — every request costs server and database time. With a cap, the maximum monthly bill is computable.
4. Containing incidents — a misbehaving loop stops at ten thousand instead of running all night. Without a cap the same bug runs until someone notices it the next morning, and by then the bill and the incident report are both written.

Three things get decided when setting one up.

1. What to count by — account, issued key, or source address. Using the address alone bundles an entire company arriving from one address into a single bucket, where they eat each other's share. Counting by issued key is usually the middle ground, because it survives a company arriving through several addresses while still separating one integration from another.
2. What window to count in — cutting each minute cleanly is simplest but lets double through at the boundary. Sliding the last 60 seconds continuously is accurate, and there's also the approach of banking a little unused allowance to permit short bursts, which suits callers whose traffic arrives in clumps rather than evenly.
3. What to return when the cap is hit — status code 429, meaning too many requests, along with when to try again. Attaching the remaining count and the window reset time to every response lets a caller slow down before hitting the wall.

The place to apply it is usually the API gateway, the layer that funnels several services through one entrance. Apply it per service and the same rule exists in several copies, with one getting fixed while the others stay, which is how a caller ends up capped differently depending on which endpoint they hit.

Several approaches address request floods, so the choice follows whether an immediate answer is needed and who pays the cost. Four get compared.

1. Message queues — line work up and process it behind rather than rejecting it. Suited to jobs like document exports where no answer is needed right now. The caller has to check the result separately later, which means the calling code changes shape rather than just slowing down.
2. Autoscaling — add servers as load rises. Everything gets accepted without rejection, but cost rises with request volume and provisioning takes minutes, by which point a short burst has already passed.
3. Caching — reuse a result already built to reduce actual processing. It only helps when the same request repeats, and does little on a window returning different results per account.
4. Pricing the excess — charge for volume over the cap instead of refusing it. Bundled with a usage limit, the ceiling on what a free tier can consume, this becomes a pricing tier. It resolves the cost problem and leaves the capacity problem untouched, which is why it tends to sit alongside a cap rather than replacing one.

The four aren't exclusive. In practice teams cap at the gateway, push heavy work to a queue, and keep frequent responses in a cache. What the cap contributes in that arrangement is a known worst case, which the other three don't provide on their own.

"This one account is calling 200 times a second. We need a cap on it."

One account is dragging down response times for everyone.

"We return 429 but never tell them when to come back."

Refusing without telling the caller what to do next.

"We agreed to raise this customer's cap to 600 a minute. It's in the contract."

The cap becomes a line item in pricing and contracts.

At Pagedeck, Theo set per-account caps at the gateway. The free plan gets 60 a minute, the team plan 600, and enterprise contracts get a negotiated number. Every response carries the remaining count and the reset time. When Sofia signs an enterprise customer, this value comes up as a negotiation item.

The most common mistake is reading rate limiting purely as attack defense. The name says limit and the act of refusing looks defensive. But what hits the cap most often is legitimate code written by your own customers — calls that grew when they shipped a feature, misconfigured retries, bulk data migrations. Set the cap very low with attackers in mind and the attacker moves to another address while your customers are the ones blocked. Set it as a rule for dividing capacity, and leave malicious traffic to the tooling built to identify it. The two look similar from the outside and need different answers.

Treating it as a value set once is the other one. It's a single number, so it goes in a config file and gets forgotten. But it's tied to server capacity, customer count, and pricing all at once. Double the servers with the cap unchanged and the added capacity goes unused; sign an enterprise customer whose contract names a different number and it has to vary per account. Without recording cap-hit counts per account, you never learn which customer walks into the wall every day, and that customer usually tells sales about it before they tell engineering.