Autoscaling

For a long time, the number of servers behind a product was a figure someone set once and left alone. The hard part was choosing which moment to set it for. Size it for the busiest hour of the day and most of those machines sit idle for the other twenty, billing by the hour for doing nothing. Size it for an ordinary afternoon and a single announcement email takes the service down.

So running a product meant keeping a person on watch. The night before a campaign, someone would add servers by hand and take them away the next morning. On a day nobody saw coming, whoever got the 2 a.m. alert opened a laptop and raised the count from bed. Forgetting to lower it again was just as common — the extra machines would stay up for two weeks and surface on the invoice.

What changed the arrangement was cloud computing: renting only as much of somebody else's data center as you need, and paying only for what you use. Getting one machine ready went from weeks of ordering hardware to a couple of minutes. Once a server can be created and destroyed that fast, no person needs to watch for the moment to do it. Autoscaling is that decision written down as a rule and handed to the machine.

It cuts the money spent on idle servers and the outages that hit at peak, without trading one for the other. Those two used to move in opposite directions. Keep the count generous and outages get rare while the bill grows; keep it tight and the reverse happens. When the count moves with the hour, the trade disappears.

Pagedeck sees almost no traffic at three in the morning and its heaviest load around lunch and again at three in the afternoon. Running eight servers around the clock to cover the peak costs $1,200 a month at $150 per server. Add up what each hour needs and the average is roughly 3.5 servers, or $525. The other $675 pays for machines that were switched on while nobody was signed in.

The mechanism is a four-step loop that never stops running.

1. Measure — read the processing load, memory use, or the number of requests waiting in line, once a minute.
2. Compare — check that reading against a threshold written in advance, such as "processing load above 70 percent for more than five minutes."
3. Adjust — add or remove servers when a threshold is crossed, within a floor and ceiling set alongside it, like a minimum of two and a maximum of ten.
4. Wait — pause the next decision for a few minutes so the last change has time to show up in the readings. Without this step the rule adds and removes the same server over and over.

The loop only works if a load balancer sits in front of it. That's the device that spreads incoming requests across several servers, and a newly started machine does no work until the balancer knows to send traffic its way. The software also has to return the same result no matter which machine handles the request — the same condition horizontal scaling asks for applies here without change. Keep a signed-in session in one server's memory and everyone attached to that server gets logged out the moment the count drops.

There are four ways to get through a busy hour without letting a rule move the server count.

1. A fixed count — leave enough machines running for the peak, permanently. There is no rule and no threshold to tune, and when load varies by less than a factor of two across the day, the savings aren't worth the moving parts.
2. Scheduled scaling — change the count by the clock. Six servers at 9 a.m. on weekdays, two at 11 p.m. This suits an internal tool whose load follows office hours, and it never reacts late because it isn't waiting on a reading. It has nothing to offer when traffic arrives unannounced.
3. Vertical scaling — grow one machine instead of adding machines, moving memory from 8 GB to 32 GB, for instance. No code has to change, and for a component that can't simply be duplicated, such as a primary database, it is often the only option available. One machine can only grow so far, though, and it goes offline while the swap happens.
4. Serverless — hand over the scaling decision itself. There is no server count at all: one function runs per request, and nothing runs when no requests arrive. The price is a cold start, the delay on the first call to an idle function.

Which one fits comes down to how predictable the load is. Barely varies, take the first. Regular as clockwork, take the second. No way to know when the rush lands, autoscale. Whichever you pick, the count is beside the point if the software lacks scalability — the property of being able to grow as load grows.

"Autoscaling is set up, so nobody needs to be online when the announcement goes out."

No one has to stand by during the rush waiting to add servers.

"I'm raising the minimum from two to four. The first requests every morning are still slow."

The count falls to two overnight, the commute-hour traffic arrives all at once, and the rule adds servers more slowly than the traffic builds.

"Let's drop the threshold from 70 to 55. We were already throwing errors for three minutes before the count moved."

Waiting until load reaches 70 percent is too late, so the rule should trigger earlier.

At Pagedeck, on the day Ravi sent a new-feature announcement to every user, Theo raised the minimum count from two to six the evening before. Ten minutes after the email went out, sign-ins hit seven times the usual level; the count climbed to nine and settled back to three about two hours later. When Grace asked the next month why that day cost twice what a normal day costs, Theo showed her the ceiling of twelve servers written into the same rule.

The common assumption is that switching it on absorbs any amount of traffic, because "automatic" sounds like "instant." In practice a minute goes by taking the reading, and another two or three before a new server is up and answering requests. Traffic that multiplies tenfold in thirty seconds outruns that. Miss the gap and a ticket sale opens with five minutes of errors, while the record shows only a line saying the service scaled on schedule.

The second assumption is that the bill goes down by itself. Shutting off idle machines makes it look that way, but the same automation works in the other direction. A bad deploy that pins processing load at 100 percent, or a bot hammering an endpoint, gets the same obedient response: more servers. Leave the ceiling unwritten and that obedience shows up at the end of the month.