Message Queue

The default shape of a web request is simple: a user presses a button, the server — the program that receives requests, does the work, and sends a response — finishes the job on the spot and sends the answer back. For loading a list or saving a document, work that finishes in a tenth of a second, nothing about that arrangement causes trouble.

Heavy work changes the picture. If exporting a 300-page document to a file takes 40 seconds, the user stares at a blank screen for all 40 of them, and the browser usually gives up and drops the connection somewhere around 30. The worse problem is what happens when requests arrive together. Five hundred export requests at 9 a.m. on a Monday means the server is holding 500 jobs in memory at once, and it falls over. Once it's down, nobody knows which requests finished and which didn't, so users can't even be told that theirs failed.

Running ten servers is the answer that comes to mind first, and it means paying every month for nine machines that sit idle the moment the morning rush passes. Turning requests away is an answer users notice immediately.

A message queue is the third answer. An incoming request isn't processed; it's written onto a list, and an acknowledgment goes back right away. A separate program running behind the scenes pulls items off that list at whatever rate it can sustain. All the queue does is decouple the rate work arrives from the rate work gets done. Five hundred requests at 9 a.m. make the line longer, and the server keeps running at its usual pace.

One piece of infrastructure covers four separate problems.

1. Slow responses. Writing an item onto the list takes a hundredth of a second, so the screen moves on immediately and a notification arrives when the work is finished. The user still waits as long as the job takes; they just aren't pinned to the page for it.
2. Collapsing under bursts. If requests arrive at 300 per second and the server can process 50 per second, the line grows by 250 items every second. What it doesn't do is fall over, and the backlog drains itself once traffic thins out around 11 a.m.
3. Work disappearing mid-flight. An item stays in the queue until whoever took it reports back that it finished. If that program dies partway through, the item reappears on the line a short time later. Because that retry can run the same job twice, the processing side needs idempotency — the property of producing the same result no matter how many times the same request arrives. For sending email, that usually means recording the send first and skipping anything already recorded.
4. Two systems waiting on each other. Wire order intake directly to billing and a five-minute billing outage stops orders too. Put a queue between them and orders keep being accepted, then billing works through the backlog once it's back.

Four approaches do the same job without a queue, and the choice comes down to how heavy the work is and how much of it there is.

1. Handle it inline. For work that finishes in a second and never arrives in bursts, this wins. Adding a queue means operating the queue itself, and it adds one more place to look when something goes wrong.
2. Run it as a background job — work split off from the response and done afterward — without a queue, inside the server process. The code stays simple, but restarting that server loses whatever it was doing, and nothing throttles the work when volume spikes.
3. Hand it to a cron job, a task that runs automatically at set times. It suits work that doesn't need to happen now and can be batched. Totaling yesterday's usage at midnight doesn't need a queue.
4. Use a database table as the queue. Insert rows for pending work and pull from them every minute. It reuses something already running and the contents are easy to inspect by hand, which carries most systems comfortably through a few thousand items a day.

Three questions tell you when to move. Does a single job take more than a few seconds? How many times higher is peak volume than normal? Would a user notice if a job silently vanished? Answer yes to the third and a queue is worth it even at low volume.

"Let's move this to the queue. We're holding the response open for eight seconds."

A proposal to stop making the screen wait.

"The queue is backed up. Twelve thousand items waiting, and we're scaling the workers to three."

Arrivals have outpaced processing, and the fix is more programs pulling from the same line.

"Retries are sending the same email twice. We need an idempotency key on the send."

A queue can hand the same item over more than once, so the receiving side has to filter duplicates.

At Pagedeck, Theo moved document exports onto a queue. Before the change, the server went down most Monday mornings and Sofia wrote apology emails to customers each time. After it, the 500 requests formed a line, two worker programs drained them in order, and the last export finished at 9:18. The server never went down.

Something else changed as well. Because nothing collapsed, the backlog became a number the team could watch, and they set an alert at 5,000 waiting items. Problems that customers used to report first now reach the team first.

The most common belief is that a queue makes things faster. The screen moves on instantly, so it feels that way. What actually got shorter is the response time; the moment the work finishes moves later, because time spent waiting in line is new time that didn't exist before. Adopt a queue without understanding that difference and you get a product where the action appears instant and the result shows up twenty minutes later — and users who can't see progress will press the same button three more times. Adding a queue and showing progress are one piece of work, not two.

The second belief is that a queue delivers everything in order, exactly once. The word queue suggests it. Order breaks the moment you run three workers, since each pulls independently and a later item can finish first. Delivery count works the same way: most queues guarantee at least once, not exactly once. Design on the assumption that the queue enforces order and count, and it comes back as duplicate charges or state changes applied backwards.