Data Mart

Once a company gathers its data in one place, the next problem shows up immediately: the place is enormous. A data warehouse holds payment, signup, session and support records spread across hundreds of tables, and the sales team wants exactly one thing out of it — payment status by account. Getting that one thing meant first working out which tables held which columns and how to stitch them together.

So a single question took days to turn into an answer. Sofia would ask how many accounts upgraded their plan this month, and whoever owned the data would write a SQL query joining six tables. SQL is the language used to ask a stored set of tables for particular rows and columns. Two weeks later the same question came back, and somebody else wrote their own version of it. One of them excluded free trial accounts and the other included them, so the same month came back as 41 and as 57.

A data mart exists to collapse that repetition into one pass. The answers to the questions that keep coming back get built into tables ahead of time and kept in one place, and from then on people read those tables. The work moves from calculating on every request to looking up something already calculated.

A data mart removes the need to redo the same calculation every time. Someone picks out the questions a department asks repeatedly, builds the tables that answer them, and schedules the refresh to run on a fixed rhythm, usually once a night.

Three costs disappear.

1. Time spent searching. If the sales mart holds three tables, Sofia never wonders which of several hundred she needs. She opens a table where one account is one row and the plan, lifetime payments and last login all sit on that row.
2. Numbers that disagree. A judgment call like whether free trial accounts count gets made once, while the table is being built, and is then baked in. It puts the metric definition inside the table instead of inside a document, so everyone who reads it gets the same value.
3. Waiting and spend. A query that sweeps the whole warehouse reads hundreds of millions of rows. The pre-narrowed table is tens of thousands of rows, finishes in seconds, and on a store that bills by bytes scanned it costs proportionally less. On a team where twenty people open the same dashboard every morning, that gap is paid twenty times a day rather than once.

Building one usually means adding a step to the data pipeline — the automated flow that pulls data from source systems, cleans it up, and loads it into the store. A step that joins six warehouse tables into one sales table goes on the end of that flow and runs before the workday starts.

Three other routes reach the same destination without a separate mart. Which one fits depends on how many people will read the table and how fresh it has to be.

1. Define a view instead. A view is a saved query with a name attached, and the query runs against the original tables each time someone opens it. Nothing extra is stored and the result is always current, but a heavy calculation makes you wait for it on every open.
2. Add one more table to the mart layer. If the store is already split into raw, cleaned and finished layers, there's no new container to build — the work is one table added to the last layer. On a team with that layering in place, this is the first option to try.
3. Handle it inside the dashboard tool. Modern tools can shape and hold data themselves, so when a BI analyst assembles a table inside the tool, readers can't tell it apart from a mart. The catch is that the definition lives only inside that tool and leaves with it if the tool is replaced.

When one person needs a number once a quarter, a mart costs more to build than it returns. Run the query once and export the result. Marts start paying off at the point where the same question comes back every week.

"Could you add one column to the sales mart for plan change history?"

Not a request for a new store — a request to widen a table that already exists.

"Is that number from the mart or straight from the source tables?"

The two paths can disagree, so this is a check on which one was read.

"Before we cut a new mart, let's see whether the existing one covers it."

More stores means more definitions to keep in line, so this is a brake on adding another.

At Pagedeck the sales mart came first, because Sofia brought the same questions every week. Which accounts upgraded this month, which accounts haven't logged in for three weeks, how many contracts renew next month — the same three, over and over. Theo merged the answers into a single table and scheduled it to refresh at four in the morning.

Meetings changed shape after that. Sofia used to file a request on Tuesday, get numbers on Thursday, and carry them into the Friday meeting. Now she opens the table during the meeting. One rule came with it: when a number looks wrong, fix the step that builds the table, not the table. Values edited directly in the table are gone by the next refresh, and the mistake they were papering over comes back with them.

It's tempting to give every department its own mart. Each team wants different things, so a container per team feels natural, and building one isn't hard. But as marts multiply, a metric with the same name starts being calculated a little differently in each one. Active accounts in the sales mart means accounts that paid this month; active accounts in the marketing mart means accounts that logged in at least once.

Left alone, two different active account numbers arrive at the same meeting, and half the meeting goes to deciding which is right instead of deciding what to do about either. The problem came from splitting, so more splitting doesn't fix it. Calculation rules belong in one place on the warehouse side, with each mart slicing from that result. This is what single source of truth points at — for any given value, exactly one place is the reference.