Extract-Transform-Load

Looking at one number requires the data to sit in one place, and it never does. Payment records live with the payment processor, signups and usage live in the service database, ad spend lives in the ad platform. The shapes disagree too. One source writes the date as 2026-09-16 and another writes 09/16/26. Payments arrive in dollars while reporting happens in the local currency, and internal test accounts sit mixed into the user list.

So early on a person did the work. When Mia wanted last month's paid conversion rate, someone downloaded files from three places, lined up the date formats, deleted the test accounts, and produced a number. It took half a day, and next month the same question started from scratch. The worse part was that two people doing it got different answers: depending on which accounts each of them filtered out, the conversion rate came back as 3.8 percent or as 4.1 percent.

Extract, transform, and load — ETL — is that manual work frozen into code. The rules a person used to decide each time get written once, and a machine repeats them in the same order. There's a reason the transform step comes before loading. When this approach took hold, storage and compute in an analytics warehouse were expensive, so loading data nobody would query was waste. Teams shrank it on the outside and loaded only what they needed.

The three letters are the order. In practice two more steps hang off the end.

1. Extract. Pull the data out of each source. The payment processor hands it over through an API, the service database exposes a replica to read from, the ad platform lets you download a file. Fetching everything every time gets expensive, so most jobs take only what changed since the last run, usually by asking for rows with an updated timestamp after the last successful run.
2. Transform. Reshape the data on a worker outside the store. Standardize the date format, convert dollar payments at that day's rate, filter out internal test accounts, join payment records to user records on a shared key. Masking values like personally identifiable information (PII) that must never enter the store also happens here.
3. Load. Write the cleaned result into the data warehouse, a store that holds analytics data in an organized structure. The usual method is to delete the date range being written and write it fresh, which is what gives the job idempotency — run the same day twice and revenue doesn't double.
4. Schedule it. The three steps break into dozens of small jobs with an order between them. A workflow orchestrator holds that map and refuses to start a downstream job when an upstream one fails. The cadence is usually batch processing: three in the morning, the previous day in one pass.
5. Verify. After each run, compare row counts and totals against the source and alert someone when they disagree. When a quietly failed stretch turns up later, a backfill reprocesses that period and fills it in.

It fits best when the source data is large and the part you actually use is small. If server records pile up at 400 million lines a day and only the two million payment-related lines get analyzed, shrinking on the outside saves real money on the store, since warehouses bill for both what you keep and what you scan. It's also the required order when the data contains values that must not enter the store at all. Deleting something after it landed and never letting it land are different arguments in front of a regulator.

There are four limits.

1. What you dropped is gone. Six months later someone asks to count those filtered test accounts after all, and the warehouse doesn't have them — you go back to the source. If the source only retains 90 days, not even that works.
2. Changing a definition is expensive. Adjust how exchange rates get applied and you edit the transform code, then reprocess every past period before the old and new numbers agree.
3. You have to run the place where transforming happens. That means a worker outside the store, and it has to grow as the data does.
4. A new metric takes a while to appear. The pipeline gets edited and the backfill has to finish before any number shows up.

Which is why early teams still deciding what to count usually start with ELT instead. In a first year, metric definitions change faster than data volume grows.

"That column gets filtered out during ETL, so it isn't in the warehouse. We'd have to re-pull from the source."

The value was discarded during transform, so searching the store won't find it.

"Changing the exchange rate basis isn't just a code edit — we'd reprocess the last twelve months."

A definition change spreads into reprocessing all of the historical data.

"The overnight pipeline finishes at three, so the morning dashboard is only accurate through yesterday."

The cadence is daily, which means today's numbers arrive tomorrow.

At Pagedeck, Theo builds this flow and Mia reads the dashboards it produces. The day they decided to change the paid conversion definition from "paid within 30 days of signup" to "within 14 days," Mia expected the number to move immediately. Editing the transform code and backfilling the past year took two days. This is exactly where changing a definition costs more than it would under ELT.

Since ELT appeared, people tend to read this one as the outdated approach. The newer thing arrived later, so the assumption is natural, but these are options rather than generations. Cheaper warehouse compute genuinely pushed teams toward ELT. Even so, when regulation says certain data can't enter the store, when the source produces hundreds of millions of rows a day and almost none of it gets queried, or when every unit of compute has to be justified, this order still wins. Plenty of companies run both, picking per pipeline according to what each one carries.

The other misreading is that moving data is the whole job. In practice something like half the time spent building a pipeline goes not into the code that moves data but into the machinery that makes a human notice when the moved result looks wrong. One step drifting quietly means every calculation above it is slightly off, and that number reaches a meeting with nobody aware. A pipeline built without data quality checks looks healthy the whole time it's accumulating months of wrong numbers.