AI Engineering

Software spent decades as a thing that returned the same output for the same input. Press save and it saves; if it doesn't, that's a bug. Write down the steps, hand them to a developer, and the same symptom appears on their screen. Fix it, and a test confirms the fix without a person watching. Right and wrong were decidable by machine.

Language models broke that assumption the moment they landed inside products. Pagedeck hit it when it added summaries to meeting notes. Theo ran the feature ten times on his own machine, got ten reasonable summaries, and shipped it. Three days later a customer wrote in to say the summary listed a decision the team had never made. Theo fed the same document back in and got a perfectly good summary. Nothing reproduced, so there was no bug report to write and no line of code to point at. The same input twice gives differently worded output, because the model samples among possible continuations instead of following one fixed path. Ten good runs on one laptop had never been evidence; they were ten draws from a distribution nobody had measured.

A second problem sat underneath the first. Rewriting the instruction improved that one document, but nobody knew what it did to the other fifty. Better and worse both got said out loud with no evidence behind either, and review meetings turned into people trading whichever examples they happened to have seen.

Machine learning already existed, but the shape of the job was different. Training rules out of data meant a company collected its own data and trained its own model, which took dedicated specialists and months of runway. Today the model is built by someone else and called across the internet. What is left is not building a model — it is fitting somebody else's model into a product and keeping it under control, and AI engineering is the name for what is left. The difficulty of construction went away, and the difficulty of judgment and containment moved in behind it.

That gives this branch one property the rest of engineering does not have. Everywhere else you confirm a feature works and then ask how fast it is. Here the team has to build the method for deciding whether it works at all before it can ask anything else.

The terms under this heading sit in the order you hit them while building a single feature. What you build with is the foundation. How you put it to work is application. What you run into is constraints. What goes wrong is risk. How you know it improved is quality. Five branches.

The foundation branch is the raw material. A large language model (LLM) is a program that has read an enormous amount of text in advance and picks a plausible continuation for whatever comes before it. It selects likely next words rather than looking facts up, and that one sentence is the reason the risk branch exists at all. Almost nobody in this field trains one from scratch; teams rent access from the company that built it, which means the single most important component of the feature is one a competitor can rent on the same terms tomorrow.

The application branch splits the ways you give that model a job. There are six, and each costs more effort than the one before it.

1. Prompt engineering — designing the instruction the model receives. "Summarize this" and "list only decisions that were actually made, in three lines, and leave out anything still open" produce completely different results from the same model. It is the cheapest lever available, so most work starts here.
2. Retrieval-augmented generation (RAG) — the model has never seen your company's documents. So when a question arrives, the system searches internal material first, pastes the relevant passages into the instruction, and asks the model to answer from those passages only. Internal search and customer-support replies are built this way, because the answer only counts when it comes from your own material.
3. Embedding — a sentence's meaning turned into a list of numbers. Two texts that share no words land close together when they mean similar things, which puts "when do I get my refund" and "how long does a payment cancellation take" in the same neighborhood. The search step in the previous method runs on exactly this comparison.
4. Fine-tuning — training a model further for one narrow use. Teams reach for it to lock in a house tone of voice, or to teach a classification scheme that exists only inside the company. It needs hundreds of curated examples and has to be redone whenever the base model changes, so it comes last, after the first three have failed.
5. Agents — instead of answering once, the model runs several steps on its own using tools: search a document, check a calendar, draft something, read the result and search again. Every added step is another place where one wrong judgment quietly poisons everything after it.
6. Model Context Protocol (MCP) — an open standard for connecting outside tools and data to a model. Connections that used to be hand-written per tool now follow one shape, so a document store and a calendar attach the same way.

The constraints branch sets how large a job can be. A token is the smallest chunk a model splits text into, roughly three or four English characters. The context window is the maximum number of tokens the model can look at in one pass. Together they are both the price and the ceiling. Billing runs per token, a document larger than the window has to be cut before it goes in, and whatever got cut simply does not exist as far as the model is concerned. That is how you end up with a 200-page contract, a question about the penalty clause, and an answer saying there isn't one.

The risk branch covers failures that belong to this field alone. Hallucination is the model stating something untrue in a confident, well-formed sentence, which is what the summary incident was. Rather than saying it does not know, it fills the gap with the most plausible wording available, so a wrong answer looks exactly as certain as a right one. A guardrail is a limit written in code around what the output is allowed to be: filtering answers that break company policy, rejecting and retrying when the required format is missing, blocking anything that carries a claim absent from the source documents.

The quality branch decides whether anything improved. An eval is a fixed set of inputs paired with the answers you expect, rerun every time an instruction or a model changes so the scores can be compared side by side. Scoring is rarely a plain string match, since a correct summary can be worded twenty ways. Teams usually check which facts have to appear and which must not, and grade the borderline cases by hand until the rule holds steady. However well a team handles the other four branches, without this one nobody can prove an improvement happened, and teams missing it argue from anecdotes until the meeting runs out.

The five branches feed each other rather than sitting in a list. A narrow context window forces retrieval instead of pasting whole documents in. Retrieval that pulls the wrong passage raises the hallucination rate. A rising hallucination rate thickens the guardrails. And whether any of those changes actually helped is a question only the eval set answers. Cost and response time cut across all five: pasting in more material usually improves the answer and always raises both, and a feature that takes twelve seconds gets abandoned before the good answer lands.

"This one's for the AI engineering side. The server didn't fall over — the model gave a nonsense answer."

Same-looking incident, different kind of cause, so a different owner and a different response procedure.

"Our AI engineering work this quarter is the eval set, not a new feature. Right now nobody can tell whether the answers got better or worse."

A commitment to build the instrument that measures quality before stacking anything else on top of it.

"AI engineering costs doubled against last month. We either cut how much we paste in or we switch models."

The bill scales with usage, so the feature is a variable cost somebody has to manage rather than a one-time build.

At Pagedeck this work sits with Theo. After the summary incident, the first thing he did was not to fix the summary. He pulled forty real meeting notes and wrote a correct summary by hand for each one. Since then, every change to the instruction gets rerun against all forty, counting how many come out wrong. Thirty-two passing turning into thirty-six passing means the change ships; unchanged or lower means it gets reverted the same day.

Those forty notes do a second job. Whenever a new customer complaint arrives, the case joins the set, so a failure the team has already lived through cannot quietly come back in the next round of edits.

Mia uses the number as a release gate. Every new AI feature gets a required score out of forty agreed in advance, and a feature below the line does not ship even when the date arrives. Grace asks two questions in quarterly planning: what the model bill came to this month, and whether the feature it bought moved paid conversion.

The most common mistake is treating AI engineering as a synonym for writing good prompts. Demos really are built from a single instruction, so the reading is easy to arrive at. In a feature that reaches customers, the instruction is a small share of the work. The rest is deciding what material to retrieve, fitting it inside the length limit, filtering wrong answers, measuring cost and latency, and confirming that a change improved things. Left uncorrected, a team spends months editing instructions. Every week feels slightly better than the last with no way to show it, and six months on the same complaints arrive at the same rate. Editing the instruction is not the problem; editing it with no scale to weigh the result on is.

The second mistake is assuming the job disappears once models get good enough. Models do improve visibly year over year, so the expectation is reasonable on its face. But no model, at any quality, holds your company's documents and customer records. Deciding who is allowed to see what stays on the product side. So does keeping cost and response time inside something the business can absorb, and so does leaving a trail that shows who was accountable when an answer turns out to be wrong. A team holding this belief builds none of that plumbing while it waits for the next release, and on the day a genuinely better model arrives it has no way to put the thing to use.