AI Governance

For a while, adding AI to a feature was a technical decision. Engineering picked the model, checked how many seconds an answer took and how much it cost per month, and the feature shipped on that. Nobody thought legal had anything to look at here. The contract and the terms of service were written on the assumption that the company's own program does only what someone wrote into it, and inside that assumption there was nothing left to ask.

The day the assumption broke at Pagedeck was the day Sofia came back with an enterprise buyer's security questionnaire, about two months after the meeting-note summary feature shipped. One item asked the company to state whether customer documents fed into that feature were used to train the model by the outside company that built it. Theo did not know. Grace asked why they could not simply write that it was not, and the answer was that nothing in the company stood behind that sentence. They read the model provider's terms end to end that evening for the first time. Paid tiers do not train on what customers send. What Pagedeck had wired up was the free developer tier.

One answered item pulled a row of others behind it. Whether the privacy policy said anywhere that customer documents cross into another company's servers. Whether the data processing agreement (DPA) that has to be signed before handing processing to an outside party existed with that provider at all. How the names and job titles of meeting attendees that end up inside a generated summary should be treated under personal data protection. None of it was decidable by the person who built the feature, and none of it was answerable by legal without opening the feature.

The questionnaire went back two months later. What changed in between was not the feature but the sentences around it. The account moved to a paid tier, a line that had not existed went into the terms, and a record was attached showing which document went to which company on which day.

AI governance is the name that grew in that gap. Taken one at a time the items above are a privacy question, a contract question and a disclosure question. They arrive together for a single reason: a judgment the company does not control now sits inside the product. Standards here are still hardening, so a finished rulebook is not something a small company can write. What it can do is put in writing what it has decided not to do, hold that line, and move the product and the contracts together whenever the line moves.

The terms under this name fall into three branches. Law draws a line from outside, which is regulation. The problems that actually blow up sit inside the product, and those are the open issues. The design choices that keep those problems from reaching a user are the principles. Read in that order, the rules trace one path from where they come from to where they land.

Regulation covers the floor a government sets. The AI Act scores each way of using AI by risk level and attaches different obligations to each level. Sorting by use rather than by technology is the character of this law. The same model behind two features gets treated differently: summarizing a document carries light obligations, while screening job applications or deciding a loan carries much heavier ones, because one person's life turns on that judgment.

Obligations usually come in three layers.

1. Uses that are off the table — reading a person's emotions without their knowledge, or scraping faces from across the internet into a database for identity lookups. There is no set of conditions that makes these allowed.
2. Uses allowed only under conditions — hiring, lending, admissions, and anything else that splits people's opportunities. The company has to document what data the system was built on, leave a place where a person checks the result, and keep a record of which input produced which output.
3. Uses that only need disclosure — summaries, drafts, suggestions, where a wrong answer costs little. Saying plainly that a machine produced it is enough.

Which layer a feature lands in is decided by what the feature decides, not by which model it calls. Europe legislated in this shape first and other countries are following it, so a company selling into several markets rewrites less later if it builds one version against the strictest market.

Open issues covers the places where the accidents actually happen. There are two.

1. Training data rights — the copyright and permitted use of the material a model learned from. It runs in two directions, which is what makes it confusing. In one, somebody else's model trains on material belonging to our customers. In the other, material that model learned from other people surfaces inside our product's output. The first breaks a promise we made to a customer; the second touches a stranger's rights. Pagedeck's move from the free tier to the paid tier closed the first one.
2. AI bias — training material tilted to one side, and results that land against a particular group because of it. A model pulls its rules out of past records, so an uneven past gets learned exactly as it was. Take a company whose hires over five years were mostly one gender, then automate résumé screening on that record: the model learns to tie getting hired to that gender and filters new applications the same way. Deleting the gender field often changes nothing, because school names, club names and anything else that moves together with gender stay in the file, and the model uses those instead.

Principles covers what gets built into the product to stop the two problems above. There are two.

1. Model transparency — telling the user that AI was involved and what the answer rests on. In practice that is a line above the summary saying it was generated, plus a pointer from each sentence back to the part of the meeting note it came from. The reason the label is needed is hallucination: a model states things that are not true in sentences just as confident as the true ones, so a reader has no way of knowing where to be suspicious.
2. Human in the loop — keeping a person in the seat that confirms a judgment before it takes effect. The common shape is that the model drafts and a person approves. A typo inside a summary and an automatically rejected refund are not the same kind of failure, so which judgments go through that step is settled before the feature is built. Added afterwards, the step means stopping something that already runs on its own.

The three branches run one way. Regulation draws the line that has to be cleared, open issues name the causes that push a product over it, and principles block those causes inside the product. Building the feature itself belongs to AI engineering; this branch decides whether what got built can reach a customer.

"Let's put this through AI governance review before we wire it up. Customer documents leave our servers in this design."

Asking to settle how far the data travels, separately from whether the feature works well.

"It isn't that our AI governance is finished. It's that what we decided not to do is written down."

What a company without a rulebook says when a buyer or an investor asks.

"This one can't run automatically. AI governance has it set up so a person looks at it once."

The decision touches a customer's money or contract, so a confirmation step is built into the design.

Since the questionnaire, Mia keeps a fixed four-line block in every AI feature spec at Pagedeck. Where the input goes, whether the receiving side trains on it, what the customer is told and where, and who reverses the result when it comes out wrong. Theo fills in the first two lines and Grace checks the contract and disclosure ones. A feature whose four lines are not filled does not ship, even when the code has been done for a week. The line that comes back empty most often is the last one. Who reverses a wrong answer is the question nobody asks while a feature is being built, so it usually gets settled only after a customer has already found the wrong answer and written in about it.

Sofia treats the same document as sales material. When an enterprise buyer asks the questions she has seen before, she copies the answers across instead of starting the research over, and when an item has no answer she does not promise anything in the room — she checks and writes back the next day. The answers themselves get re-read once a quarter, because a provider that quietly changes its terms turns an answer that was true last quarter into a wrong one this quarter. What Grace reviews each quarter is two things: whether the items written down as not-to-do stayed undone, and whether any change in law or in a contract has moved the line since.

The most common reading is that a company which does not build models has nothing to do here. Regulation in the news mostly points at the handful of companies training enormous models, so it lands that way. But the rules in this field attach to what a model is used for and who it reaches, not to who trained it. The company that put the feature on a customer's screen is us, and the company holding the contract with that customer is us. Leave the assumption in place and the order reverses the moment something goes wrong. The customer asks us rather than the provider, and the only thing we can hold up is a page of the provider's terms, which does not stand in for the promise between us and that customer.

Waiting until the law settles is the other one. Standards differ by country and shift every year, so matching them once at the end looks like the cheaper plan. Parts of this work, though, cannot be made retroactively. Documents already sent to an outside model cannot be recalled. Material collected without consent can only be made lawful by going back and asking every user who already signed up. And if no record was kept of which input went where, there is no way to explain afterwards what actually happened. The deferred bill then arrives all at once on the day a rule takes effect. What would have been one disclosure line and one annex to a contract becomes re-consenting hundreds of thousands of users and switching a live feature off for several weeks.