Large Language Model

For a long time, every text feature needed its own model. Sorting support tickets by category meant collecting thousands of tickets a human had already labeled and training a classifier on them. Wanting the same tickets condensed into three lines meant starting over with a fresh dataset for that job. The two features read the same sentences and shared nothing.

Pagedeck hit that wall directly. When Mia suggested sorting incoming tickets by type, the answer was that the data would have to be gathered first — months of calendar time before anyone could see whether the idea worked. The cost of trying one feature was high enough that ideas which might not pan out never got tried at all.

A large language model (LLM) inverts the order. Instead of training on one task, it trains on a very large amount of text up front and receives the task later, as an instruction. Write "sort these" and it sorts; write "condense these" and it condenses. Months of collecting and training collapse into minutes of writing a few sentences, and the cost of abandoning an idea drops just as far.

Three things change in practice.

1. The cost of starting. You get a first result from an instruction alone, with no training data. Paste in ten tickets, ask for categories, and an answer comes back immediately. Prompt engineering is the craft of shaping that instruction, and the same model produces markedly different output depending on how it's written.
2. Text with no fixed shape. This handles free-form writing rather than numbers in columns — pulling only the decisions out of meeting notes, sorting tickets by sentiment, summarizing a long document. Work that resists being written down as rules.
3. The input itself. People can type what they want instead of learning which buttons on a screen produce it.

Three limits travel with all of that. The first is size. Models chop text into units called tokens, and the maximum number of tokens a model can look at in one pass is its context window. A document larger than the window can't go in whole, so it has to be split, and whatever gets left out does not exist as far as the model is concerned.

The second is accuracy. The model isn't looking anything up; it's choosing a plausible continuation, which makes hallucination — inventing content and stating it confidently — a structural property rather than a bug to be fixed. Not knowing your internal documents is the same issue from another angle, which is why teams often add retrieval-augmented generation (RAG): find the relevant passages first, attach them to the instruction, and have the model answer from those.

The third is price and latency. Billing runs on token counts, so long documents cost more, and answers take seconds to come back. That rules out wiring it directly behind anything that has to respond the instant a button is pressed.

Other approaches solve the same problems, and the choice usually comes down to how well defined the task is.

1. Writing rules. Check whether certain words appear and route accordingly. For sending any ticket containing "refund" to the refunds queue, this is the better tool: it costs almost nothing, the same input always produces the same output, and you can explain every decision.
2. A conventional machine learning classifier. When labeled data already exists, a small model trained on it is faster, cheaper, and steadier. That holds especially for work with a fixed set of categories, like ticket routing.
3. A small specialized model. A common sequence is to prove the idea with a large model first, then use its output as training data for a small one to bring cost and latency down.
4. People. For ten items a day, having someone handle them beats automating them. Automation pays back only when volume justifies it.

The large model earns its place in the gap those four leave: categories that aren't fixed in advance, no data collected yet, or one feature that has to do several different things at once.

"This looks like it could be rules. Do we really need a model?"

Arguing for the simpler option on cost and predictability.

"The document is too long for the context window, so we have to chunk it."

It won't fit in one pass, which means the design has to split it.

"We swapped models — how do we know it got better?"

Asking for a shared basis of comparison instead of an impression.

Pagedeck's ticket summary feature followed that order. Mia wrote a few lines of instruction, ran twenty tickets through, and got a sense that it was usable. Theo then changed it to pull in internal documents alongside the ticket, and added guardrails to reject answers that broke the expected format.

The next thing they built was an eval set: sixty tickets paired with the summary each one should produce. Every time the instruction or the model changes, the same sixty run again and the scores get compared. Without it, "this got better" rests on two or three anecdotes.

The most common mistake is assuming the model looks the answer up somewhere. Question in, answer out resembles search closely enough that the reading is natural. But what the model does is choose the next words, and unless you attach source documents yourself there is no notion of a source at all. Read it as search and people use the output without checking it. Clause numbers that don't exist and feature names nobody shipped end up in documents this way.

The other mistake is thinking the model learns when you correct it. Point out an error mid-conversation and it fixes the answer on the spot, which looks like learning. The correction holds inside that conversation and vanishes with it; training happens on the model maker's side, on their schedule. Not knowing this leads to correcting the same error every day and wondering why it never sticks. What actually reduces a recurring error is changing the instruction, attaching source material, or filtering the output with guardrails.