Vertical Scaling

At some point growth means the server stops coping. A response time of 0.3 seconds at 500 concurrent users turning into 8 seconds at 2,000 is functionally identical to being broken, whatever the features do. Nothing in the code changed; the number of people did.

There are only two things you can do about it. Replace the machine you have with a better one, or run several of the machine you have. The first is vertical scaling and the second is horizontal scaling.

Back when servers were bought and racked, the first option was a project. Adding memory meant ordering parts and visiting the data center. Replacing the machine outright meant a quote, an approval, and a delivery window — six weeks, during which the service kept running slowly.

Cloud turned that into changing a size setting and restarting. When traffic spikes on a Friday evening and responses start backing up, a team with no time to change the architecture can move up two sizes and get through the weekend. One of the two options shrinking from weeks to minutes is what made it a first resort again.

Measure which resource is short before anything else, then move to a size that grows that resource and restart.

1. Identify the resource that is short. There are four candidates: the CPU doing the computation, the memory holding data in reach, disk read and write speed, and network bandwidth. If memory is short and the system is spilling onto disk, adding CPU cores raises the bill and leaves the response time exactly where it was.
2. Pick the next size up. Cloud size lists usually step in doublings, so eight cores and 32 GB is followed by sixteen cores and 64 GB. You can also pick a shape that grows only the resource you need, which matters because a memory-heavy and a compute-heavy shape at the same price differ several-fold in the resource you actually ran out of.
3. Decide how to cut over. Changing the size requires a restart, which means a few minutes of downtime. With a load balancer in front distributing requests across several machines, you can replace them one at a time with no interruption; with a single machine, you pick an hour when few people are using it. The restart itself is short; deciding when to take it is usually the longer part.
4. Measure again afterward. Look at response time and resource utilization together. If you doubled the size and responses got only 1.3 times faster, the bottleneck was not in the resource you grew.
5. Leave a way back. Sizes can go down as well as up, so something overprovisioned this week can be reduced next week. Disk capacity is the exception — it usually only grows — so that one item deserves a slower decision.

Four situations suit it. First, servers that hold state. A relational database needs its data in one place, and adding machines is hard there, so growing the size is the default instrument. Second, code that is not ready to run across several machines: keep session state in server memory and adding a second machine logs users out at random. Third, a situation that has to be survived right now. Fourth, a stage still small enough that one machine is genuinely enough. All four share a shape: the real constraint sits somewhere other than machine size, and growing the machine buys time to deal with it.

Four limits are worth knowing.

1. There is a ceiling. A largest available size exists and there is nothing above it. Reaching it reduces you to one option, so it is worth knowing where that ceiling sits for your provider before you need the number.
2. Cost does not scale with capacity. Price per unit of performance rises as sizes grow, so moving a $70-a-month machine up four steps lands around $540 a month. A range arrives where the same money buys eight small machines instead.
3. One machine is the whole service. If it stops, the service stops. Adding machines means the survivors absorb a failure; growing one machine leaves no such margin.
4. It requires a restart. Autoscaling, which adjusts capacity automatically as load rises and falls, attaches to the add-and-remove-machines approach. Changing a machine's size stays an operation a person decides on and performs.

"Bump the size for now and get through it. Restructuring is next sprint."

An instrument for when there is no time.

"The database goes vertical. Splitting it across machines is a different problem."

Servers holding state are judged by different criteria.

"We doubled the size and the response time barely moved. The bottleneck wasn't CPU."

The result of not measuring what was short before spending.

At Pagedeck, Theo grows the database by size and the web servers by count. The web tier keeps session state in a separate store rather than in server memory, so adding machines there causes no problems. The split is written into the runbook, so nobody has to rederive it in the middle of an incident.

The database size has been raised twice in the past year. The first time CPU utilization looked high, so the compute side was grown, and responses barely improved. Measuring again showed memory was short and the system was reading from disk continuously. The second change moved to a memory-heavy size, and response time halved.

The most common error is treating vertical scaling as a stopgap on the way to going horizontal. Articles describing large services almost always show the diagram with many machines, which invites that reading. But the right answer differs by position. A database, which keeps data in one place so values cannot disagree, acquires an entirely new problem the moment there are several machines: deciding which machine's value is correct. Which is why even large companies grow their database by size for a long time and handle splitting it through separate, deliberate techniques. It is not a question of temporary against permanent but of whether the thing holds state.

The second error is expecting double the size to mean double the speed. The numbers are written that way, so the assumption is natural. But the arithmetic only holds if the program can use the resource you added. Code written to use one core at a time runs at the same speed on sixteen cores as on eight. If the bottleneck was disk, doubling memory shows up first and the CPU figure never moves. Skipping step 1 and going straight to a bigger size is common, and the money spent that way shows up only on next month's invoice. Measuring first costs ten minutes; guessing costs a month of the wrong machine.