ML Engineer

A model being good and a model being usable in a service are two different statements. In an experiment environment, two seconds per prediction is fine as long as the answer is right. At the point where a user is waiting for a screen, two seconds is simply a slow screen.

Pagedeck's document recommendation model went that way. The people who built it reported accuracy, and the number looked reasonable. Getting it into the service, one prediction took 2.3 seconds, and the training data was a file assembled by hand. There was no procedure for retraining it at all. Three weeks after it shipped, the recommendation list began showing only old documents, because newly created ones never reached the model. Nothing had broken; the model was simply describing a version of the product from three weeks earlier.

An ML engineer owns that stretch. Wrapping the model in something the service can call, making sure training data arrives in the same shape every day, and noticing when performance drops and retraining it — that's the scope. None of it is about choosing the model.

The day starts by checking what the model returned yesterday and how long it took, and ends with work on a pipeline or a deployment. Six things repeat.

1. Read the metrics in three parts: how often predictions are right, how long responses take, and how often calls fail. Any one of the three going bad looks like the same malfunction to a user, so all three sit on one dashboard.
2. Check that training data arrives in the same shape daily. A column disappearing or a value range shifting doesn't stop training; it only degrades the result. Catching that quiet failure is the most valuable check of the day. Alerts on the shape of the data catch it earlier than alerts on accuracy do.
3. Wrap the model in something the service can call. Set a response time target first and get inside it — by making the model lighter, precomputing answers to common requests, or building results once a day instead of predicting live. Which of those fits depends on how fresh the answer has to be.
4. Prepare deployment and reversal. Send a new model to a fraction of requests rather than all of them, watch for a few days, and revert when it gets worse. Unlike code, a wrong model doesn't announce itself, so keeping a reversible state is permanent work.
5. Automate retraining. Run it on a schedule, or when a signal shows performance dropping. The whole of that automation is what MLOps names.
6. Watch whether the distribution of incoming data has shifted. The model can be unchanged and still return different results once the way people use the product changes. The answer there is retraining, not editing the model. Distinguishing the two saves weeks of looking in the wrong place.

The seat sits between the people who build models and the people who build the service, which means speaking both languages. Most of the friction in the role lives at those two handoffs.

• Data scientists — they design and experiment. The handoff starts by agreeing what has to be reproducible: without the training data and the settings, the same model can't be rebuilt.
• Backend engineers — together you decide where the model gets called and what to show when a response is late or fails. At Pagedeck, Theo and the ML engineer agreed that a failed recommendation falls back to a list of recent documents.
• Analytics engineers — they supply training data in a cleaned-up form. With that seat empty, an ML engineer ends up doing it and loses the time meant for models.
• Product managers — they decide what counts as a good result. Accuracy rising and product metrics rising are different events, so conversations with Mia start with what belongs in the recommendation pool.
• Data analysts and BI analysts — they turn model output into something people read. When it has to be explained why the model recommended what it did, you look at it together with them.
• Security and legal owners — they check whether identifying values sit in the training data and how far that data may be used. That review belongs before training rather than after.

"Accuracy is good but responses take 2.3 seconds. We can't ship it like this."

Model quality and shippability are different problems.

"The training data is a hand-assembled file. The pipeline has to exist before retraining can."

Naming the gap between building something once and running it continuously.

"I'll turn the new model on for 10 percent of requests and watch it for two days."

Deploying a model in a form that can be reversed.

Getting that recommendation model shipped with Theo is what created this seat at Pagedeck. Instead of predicting live, the lists were built once a day per user in advance, which brought responses from 2.3 seconds to 180 milliseconds. Training data was set to collect automatically at the same hour daily and retraining was scheduled weekly. Three weeks later, the problem of only old documents appearing was gone.

What remained was a different kind of problem. The accuracy figure held steady while the rate of people clicking recommendations didn't move. Looking at it with Mia, the model was reliably predicting documents the user already opened often. Predicting correctly and being useful turned out to be separate things, and the fix was removing recently opened documents from the candidate pool. The metric that needed changing was the one the model was optimizing toward.

The most common reading is that this is another name for a data scientist, since both work with models and the titles sound adjacent. The stretch each owns differs. The first searches, through experiments, for which model suits the problem; this one makes that model survive hundreds of calls a second and still perform three months later. Some teams have one person do both, and in those teams one half usually slips. Which half slips depends on what the company measures.

Reading shipping as the end of the job is the other one, because that instinct comes from code. But a model degrades with the code untouched, because the way people use the product changes and the data changes with it. Ship without monitoring and retraining attached and six months later the recommendations are wrong with nobody having noticed.