Local Maximum

The same pattern kept showing up in teams that ran experiments well. Twenty A/B tests a quarter — split the audience at random between two versions, see which number comes out higher. Button wording, button color, field order. The conversion rate climbed from 22 percent to 26 percent that way, and then every change started landing within a fifth of a percent of where it started. The tests kept winning and a year later the rate was still about 26 percent.

What had happened only became visible when someone looked sideways. Another product in the same market was converting at 41 percent. That team hadn't polished its signup flow — it had reversed the order. Instead of creating an account and then writing a document, you wrote the document first and were asked to log in at the moment you tried to save it. Nothing resembling that idea appeared anywhere in the first team's list of twenty experiments.

The real problem was having no language for the situation. "Every test wins, so why is the number flat?" usually got answered with "we need better ideas," which sends the team into another quarter of ideas the same size. Local maximum is the name that moves the diagnosis off the supply of ideas and onto the size of the candidates.

A local maximum forms whenever the direction of improvement is decided by one number measured right now. A single experiment takes one step from wherever the team is standing, keeps the step if the number goes up, and undoes it if the number goes down. That procedure is very good at finding uphill. It is also only able to find uphill. Anything you can reach by first going down never enters the list of candidates.

It's the same as walking up a hill in fog with your eyes on the slope at your feet. Do that and you will certainly reach the top of the hill in front of you, but a taller mountain across the valley stays out of reach. Getting there means descending first, and a method that checks the altitude after every step reads that first step as a failure and reverses it.

Three mechanisms usually keep a team parked in place.

1. The ranking rule favors small candidates. An experiment backlog holds the queue of things to try, and the order is normally set by multiplying expected impact by confidence and dividing by the effort. A candidate that changes the shape of the product has never been tried, so confidence is low, and it takes four weeks to build. It loses to a two-day copy test every time.
2. The cycle is short. Run a growth sprint on two weeks and the only candidates are things that can be attached and detached inside two weeks. Reversing the signup order does not fit in one cycle, so it never gets written down at all.
3. The scoreboard pays for small wins. A team that wants its win rate to stay high picks the tests it expects to win. Attempts that change the shape of things lose nine times out of ten, which drags that number down.

The phrase comes up most often in the meeting where last quarter's results sit next to the plan for the next one. The experiment list shows eighteen wins, the chart is flat, someone says "I think we're at a local maximum," and the conversation shifts from gathering more ideas to asking which of the fixed things could be moved.

The first piece of work is writing down what has been held fixed. Signup starts with creating an account. Pricing is charged per user. The first screen is an empty list of documents. These are treated as the backdrop of the experiments rather than as subjects of them, which is why they stay invisible until someone lists them.

Getting out means protecting candidate size from outside the ranking rule. Reserving a slot is the common move: one item per quarter is assigned to a change of shape, and that one item is not allowed to slip because its confidence score is low. The criteria and the time window change with it. A large candidate normally won't move the number inside a single cycle, so it gets judged on what the team learned instead of on conversion rate, and something like a reordered signup flow — which people need time to get used to — is kept running even when the first two weeks come in lower.

"Every test wins and the chart doesn't move. Are we at a local maximum?"

A proposal to increase the size of the candidates, since only small ones are being run.

"Let's set one slot aside this quarter for getting off the local maximum."

Reserve room in advance for a candidate that low confidence would otherwise push down the queue.

"This isn't a local maximum, we just haven't run enough tests yet."

A pushback saying the cause of the flat line has been misdiagnosed. If there is still obvious ground to cover, the term doesn't apply.

At Pagedeck the phrase came up over the screen that moves free-plan users onto a paid plan. Ravi spent six months changing its wording and its timing and pushed conversion from 3.1 percent to 3.8 percent, then bounced between 3.7 and 3.9 for three months more. When he put all fourteen experiments on one slide at the quarterly review, Mia pointed out that every one of them had changed when the screen appeared and nothing else.

The assumption underneath turned out to be about who decides. The team had treated the upgrade as an individual choice and shown the screen only to individuals, while the person who actually pays is the team lead. The next quarter went to building a flow that sends a request to the lead instead.

The most common misreading is that a local maximum means small experiments are worthless. The term is used in contrast with big leaps, so the inference is easy to make. In practice the 26 percent that small improvements built is the starting line for the next leap, and the team converting at 41 percent goes right back to tuning wording and order from there. The problem the term points at isn't that small tests are being run; it's that not a single large candidate is on the list. Left uncorrected, the misreading sends a team to the opposite extreme of repeated redesigns with no evidence behind them, and those screens usually come out worse than what they replaced.

Calling any flat line a local maximum is just as common, because the shape of the chart looks identical. Numbers stall for several different reasons. The product may be seasonal, the traffic mix may have shifted so that different people are arriving, or the tests may be running on samples too small to detect the difference. To tell them apart, count the untouched candidates. If ten obvious ones are still sitting there, the team simply hasn't run enough tests.