Token

People measure the length of text in characters or words. Models don't. Miss that difference and both your limits and your costs come out wrong, and the moment you notice is usually in production.

Pagedeck hit this while building document summarization. The feature passes a document to a large language model — a model trained on large volumes of text to work with language — and gets three lines back. Theo capped the input at 5,000 characters. Some documents went through and others returned an over-limit error. The character counts were identical.

What separated them was the language. An English meeting note of 5,000 characters went in fine, and a Korean document of the same length was rejected. The unit the model counts isn't characters but tokens, and the same character count produces two or three times as many tokens in some languages as in others. Capping by characters was the cause, and no amount of tuning the cap would have fixed it — 5,000 became 3,000 and the same documents failed.

Text gets split into chunks that appear frequently, and each chunk counts as one token. The list of chunks is fixed when the model is trained, derived from its training data, so strings that appeared often become single tokens while rare strings break into several pieces.

1. Common words are one token whole. In English, words like "the" or "have" are a single token each, which is why word-counting feels close enough at first.
2. Rare words break apart. A person's name or a piece of domain jargon becomes two or three pieces, and a random string can be split per character.
3. Whitespace and line breaks count too. Writing the same content without line breaks uses slightly fewer tokens, which matters at scale and almost never on one call.
4. Languages differ a lot. English runs around four characters per token; Korean sits between one and two tokens per character. The same meaning in a different language can cost two or three times more.
5. Input and output count separately. The text sent to the model and the text it produces are each measured, and the limit applies to their sum rather than to either one. Billing treats them separately at different rates.

Exact counts differ by model. Splitting rules vary, so a sentence that is 12 tokens in one model is 15 in another. So you design with estimates and confirm with real values. The response from the API you call the model through — the agreed interface programs use to talk to each other — reports how many tokens were used, and that usage field is the only number worth trusting. Counting locally with a library is close, and only the response is exact.

The first place it bites is cost estimation, because pricing attaches to tokens. A typical shape is $5 per million input tokens and $20 per million output tokens, with output the more expensive side. A summarization call at 8,000 input tokens and 500 output tokens comes to about $0.05. Two thousand calls a day is roughly $3,000 a month. Running that arithmetic before building the feature is what tells you which part needs shrinking, and it is almost always the input.

Three other places.

1. Input limits — set the length a user can submit in tokens. The screen can still say characters, but the check that blocks has to count tokens. In documents mixing languages, a character-based check will be wrong in one direction or the other, and users only report the direction that blocks them.
2. Choosing what to send — instead of passing a long document whole, find the relevant parts and attach those. RAG, the approach of retrieving outside material and attaching it so the model can answer, exists half for quality and half for tokens. You cannot put 300 pages into every call.
3. Output limits — cap the maximum tokens the model may produce. Without a cap, a request for a summary can come back as long as the original, and since output is the expensive side the cost spikes before anyone notices the quality problem.

Prompt engineering watches this unit as well. Designing instructions so a model produces the answer you want often means adding examples, and every example is counted again on every call. Five examples that improve the answer also multiply the bill, so the two have to be measured together rather than separately.

The unit doesn't belong in anything users read. "Up to 8,000 tokens" tells a user nothing at all. Put "about ten pages" on the screen and judge in tokens behind it. The two numbers never have to match.

"You can't cap this in characters. Korean documents get blocked first."

The unit being counted isn't characters.

"This feature is about five cents a call, so two thousand a day is three thousand a month."

Estimating operating cost from token pricing.

"There's no output cap. The summary comes back longer than the source."

Response length was never limited.

Theo reset the limit. The code counts tokens before sending, truncates anything over 8,000, and the screen notes that long documents get their opening section summarized. The output cap went to 400 tokens.

Over-limit errors disappeared after that, and the per-call cost fell from about $0.05 to $0.04. Mia's question came next: whether summarizing only the opening section was useful to anyone. That question is why the following quarter rebuilt the feature to split a document and call the model several times.

The most common belief is that a token is a word. Common English words really are one token, so the assumption forms naturally. But one person's name frequently becomes three tokens and a single emoji can become several. Estimating by words is off by more than half on input that mixes languages and code. Left in place, the cost estimate comes out at half the real number, and the feature approved on that estimate overruns its budget a month after launch.

The other common belief is that the limit applies only to input. Attention goes to what you send, so that reading follows. But the limit covers input and output together. Fill the window nearly to the limit with input and the model has no room left to answer, so the response truncates mid-sentence or errors outright. Features handling long input set aside a share for output first and derive the input cap from what remains.