Training Data

The original way to make a program judge something was to write the rules down. To catch spam accounts, a person wrote conditions: more than 100 documents created in a day and over 80 percent external links means spam. The advantage was that whoever wrote the condition could explain why they chose it.

The problem was how fast the conditions multiplied. Every exception adds one. There are teams that create a lot of documents and are legitimate, and teams with a lot of links that are legitimate. Six months in there are forty conditions, nobody knows which of them is actually doing work, and nobody dares delete one because what breaks isn't knowable. Then the spammers change tactics and all forty need revisiting.

Training data inverts the order. Instead of writing conditions, you collect examples that already carry answers. Feed in 2,000 accounts judged to be spam and 46,000 legitimate ones and the program finds the boundary between them itself. The human work moved from writing rules to gathering cases and attaching answers. Which is why a question about a model's quality is usually a question about data rather than about algorithms.

Start with the shape of one row: a set of input values paired with the answer for that row. For an account, the inputs might be days since signup, average documents per day, people invited and share of external links, with spam-or-not as the answer. Teaching from answer-labeled data has a name of its own, supervised learning.

The collected data gets split into three parts.

1. Training set — around 70 percent. The portion the model actually uses to find its boundary.
2. Validation set — around 15 percent. Used to compare settings against each other. Looked at repeatedly.
3. Test set — the remaining 15 percent. Looked at once, at the end. Peek at it midway and you start tuning toward that result, which destroys its ability to estimate performance on new data.

The reason for the split is catching overfitting — the state of being tuned so tightly to the training data that nothing generalizes, like memorizing answers to past exam papers. 98 on the training set and 71 on the test set points here first.

Attaching the answers is human work, and what varies is the standard. When one person calls an account spam and two call it legitimate, a model trained on that data wavers too. So the judgment criteria get written down, and the same 200 accounts get labeled by two people independently. Disagreement above 10 percent means rewriting the criteria.

Ahead of volume comes representativeness. The data has to look like what will actually arrive. A model trained on 50,000 enterprise accounts produces nonsense on individual accounts, and the failure is quiet — the scores on the test set look fine, because the test set came from the same skewed pool.

Three situations bring it up in practice. The first is building a model yourself, most often for a classification task — deciding which category something falls into, like whether an incoming question is about a refund or a bug. Here, 30,000 past questions with categories attached are the training data.

The second is adapting someone else's model. Fine-tuning means feeding your own data into an already-trained model to point it at a specific use. Far less volume is needed: building from scratch takes tens of thousands of examples, while fine-tuning often works on a few hundred to a few thousand well-chosen ones. The selection standard is stricter, though, because inconsistent answer formatting teaches the model the formatting too.

The third is refreshing. One dataset doesn't last. Add features to the product and the pattern of normal usage changes, so a model trained on older records goes dull without anything in the model having changed. Recent cases get added each quarter and old ones get retired, which means the dataset needs an owner the way a piece of code does.

"Before we change the model, let's look at the training data. There are only 2,000 spam examples."

Suspecting the data before the algorithm.

"Labeling without written criteria means two people disagree. Let's write the criteria first."

Asking for consistency in the answers.

"Don't open the test set until the end. Look now and we'll start tuning toward that number."

Protecting the evaluation data from contamination.

At Pagedeck, Theo runs a model that screens spam accounts. The training data is 2,100 accounts suspended over the last two years plus 20,000 randomly sampled legitimate ones. Suspension reasons are recorded in five categories, and accounts suspended for payment failure were excluded — a different kind of suspension that blurs the boundary when mixed in.

Mia maintains the labeling criteria document. Each quarter she and Theo independently judge 100 newly suspended accounts and count the disagreements. Last quarter twelve diverged, eight of them teams exchanging many documents with outside partners. A sentence went into the criteria saying that case counts as legitimate.

The most common mistake is assuming more is better. More examples sounds like more to learn from. But when the additions are all the same kind of example, scores don't move. Grow spam cases from 2,000 to 4,000 and if the new 2,000 use one identical tactic, the model gets better at that tactic only. Mix in inaccurately labeled records and scores actively drop. What to look at before adding volume is which kinds of case the model currently gets wrong, and filling those specific gaps beats doubling the whole set.

The other mistake is treating everything on your servers as available for training. It's your product's data, so it feels that way. But documents customers upload belong to those customers, and what they can be used for is set by the scope written into the terms of service. Put customer documents into a model with no such language in place and a breach claim follows. The issues attaching here are called training data rights. Enterprise contracts frequently demand a clause forbidding use of that company's data for training, and once that clause is in, the account's data has to be separated and managed apart.