Continuous Integration and Delivery

There was a long stretch when shipping was one person's job. A developer copied the files they had edited on their own laptop straight onto the server. Which files went up, in what order, what had to be deleted first, what had to be restarted afterward — all of it lived in that person's head. Skip a step and the service went down. Take a week off and nobody on the team could ship at all.

Merging was manual in the same way. Everyone kept their work on their own machine for two or three weeks, then combined it all right before a deadline. Three people had edited the same file, so someone had to sit down and choose line by line which version survived, and that alone ate two days. Even once it was merged, features that worked fine in isolation often broke when they were put together. The stretch of time spent untangling all of this picked up a name: integration hell.

Put those two problems next to each other and they feed each other. Merging is painful, so teams merge rarely. Shipping is frightening, so teams ship rarely. Rare shipping means each release carries a bigger pile of changes. A bigger pile means that when something breaks, nobody can tell which change did it, and that makes the next release more frightening still. A team that ships once a quarter puts three hundred commits into a single deploy. When the page comes back blank, there are three hundred suspects.

Pagedeck started out this way too. Theo would push an edited file to the server on a Friday evening, reload the page once to confirm it appeared, and go home. On Monday morning Sofia would arrive with a report from a customer that payments were failing. The first job was working out what had changed the week before, and nothing recorded that except Theo's memory. Rolling back was not an option either, because there was no saved point to roll back to — so Theo edited the code back by hand, one line at a time, from what he could recall.

CI/CD is the idea of taking both of those jobs away from people and handing them to a machine. Merging gets chopped into small pieces and happens several times a day rather than once every two weeks, because the less code collides at once, the more a conflict becomes a five-minute task instead of a two-day one. Every merge triggers an automatic check, and code that passes the check keeps moving all the way to a server. Deploying stops being an event somebody has to work up to and becomes something that simply follows from pushing code. None of this requires a larger team. It requires writing down, once, the steps that used to live in one person's head, in a form a machine can repeat without getting tired or distracted on a Friday evening.

The vocabulary underneath this heading splits three ways. How far the automation reaches is the stages branch. What the automated flow is made of is the components branch. How the last step swaps things over for real users is the deployment strategies branch.

Stages is the branch people mix up most, because the three terms in it sound nearly identical. They are not competing approaches — they differ only in how far down the road the automation goes.

1. Continuous integration — everyone merges their edits into a shared branch at least once a day, and every merge kicks off an automatic build and test run. It goes by the initials CI. Nothing reaches users at this stage; the machine only checks.
2. Continuous delivery — whatever passes the checks is kept permanently in a state where it could go out at any moment. A person still presses the button for the actual release. Teams that have to announce changes to enterprise customers, or line a release up with a marketing date, stop here on purpose.
3. Continuous deployment — the button goes away as well. Anything that passes the checks reaches production servers without a human touching it. Teams that ship ten times a day work like this, but they get there by first building the machinery that notices a problem quickly and reverses it.

The last two both abbreviate to CD, so in writing they are impossible to tell apart. The difference is whether a person presses anything at the end. If a conversation gets ambiguous, asking "does it go to production on its own?" settles it.

Components is the branch about what the flow is actually made of. Six terms sit here.

1. Pipeline — the automated sequence that strings build, test, and deploy together in order. Code arrives, the sequence runs stage by stage, and a failure at any stage stops it right there rather than passing the work along. That stop is the thing keeping broken code off a server.
2. Build — the step that turns source code, written for people to read, into a form a machine can execute. A failure here means the syntax is wrong or a dependency is missing, and the run ends before testing even begins.
3. Artifact — the packaged output a build produces. The thing that was tested and the thing that lands on the server have to be identical, so a single artifact is reused for staging and production alike. Rebuilding separately per environment risks shipping something other than what passed the checks.
4. Environment — one of several execution spaces with different purposes, usually development, staging, and production. Staging is set up to resemble production as closely as the budget allows, and the team walks through it instead of real customers. Problems caught there never reach anyone paying.
5. Environment variable — a configuration value injected differently per environment. One copy of the code points at a test payment provider in staging and the real one in production, and this value is what makes the difference.
6. Secret management — handling the subset of those values that must never leak, such as payment keys and database passwords, by keeping them in a vault and releasing them only at the moment they are needed. Write a key into the code repository and everyone with access to that repository now holds the key.

Deployment strategies is the branch about the final step. The swap happens while people are using the product, so this is where downtime and the speed of reversing a bad release get decided.

1. Blue-green deployment — two environments of equal size are kept side by side. New code goes onto the idle one in full, and then user traffic is switched across in a single move. If something looks wrong, switching traffic back is the whole rollback, which takes seconds. The cost is running double the servers.
2. Rolling deployment — with ten servers, two are replaced at a time until all ten are on the new code. No spare environment is needed, so it is much cheaper, but for the duration of the rollout old and new code are running at the same time and have to tolerate each other.
3. Preview deployment — this one sits before users entirely. Every proposed change gets a temporary address showing that change and nothing else. Designers and product managers can look at it without reading code, which makes this the cheapest place in the whole sequence to decide something should be undone.

The three branches are not a menu to pick from independently. They chain. Frequent integration is what makes the pipeline run often. A pipeline that produces one artifact is what lets the same package be checked as it moves across environments. And what to do with that checked package at the very end is the deployment strategy.

"CI is red on this pull request, so it can't merge yet. Two tests are failing."

The automated check failed, so the author has to fix it before asking anyone to review.

"Let's stop at delivery for this one instead of deploying automatically. The customer announcement goes out Wednesday."

Run the pipeline up to the edge of production and let a person press the final button on Wednesday.

"I attached the preview link. Have a look there before we merge."

A temporary address with only this change is live, so review the screen rather than the code.

At Pagedeck, Theo pushes a fix to how the document list is sorted. Three minutes later the pipeline reports that one of four hundred and twelve tests broke. He corrects it, pushes again, and eight minutes after that the run goes green and a preview address for this change alone appears on the pull request. Noa opens it to check the spacing and Mia reads the message shown when the list is empty. Once the approval lands and the change merges, the same pipeline carries it through staging and on to production. Theo never connects to a server himself.

Reversing works through the same machinery. If the error rate jumps half an hour after a release, putting the previous artifact back is the entire fix, which means the screen can be restored before anyone has finished diagnosing the cause. That is the sharpest contrast with the Friday evenings Theo used to spend editing code back from memory.

The most common mistake is believing that installing the tooling is the same as having CI/CD. Drop a configuration file into the repository, watch a green check appear, and it looks finished. But how much an automated check catches is decided by test coverage, not by the tool. In a repository with no tests, the only thing the pipeline confirms is that the code compiles, and it will happily go green while login is broken. Left alone, this produces a team that trusts the green check, merges, and then learns from customers that a feature is down. After that happens twice, nobody trusts the check any more, and people start verifying by hand on top of automation they are still paying to run.

The other frequent misreading is that shipping often and automatically must cause more incidents, since it sounds like removing a human review step. The evidence points the other way. Comparing engineering organizations through DORA metrics shows that the ones deploying most often also have lower change failure rates and recover faster, because each release carries less change. When a deploy of three commits breaks, there are three places to look; when a deploy of three hundred commits breaks, working out where to start is its own project. A team that holds this belief bundles releases into one per quarter in the name of safety, and that quarterly release becomes an all-night event for everyone. The risk did not shrink. It got concentrated, and concentrated risk is the kind that arrives at the worst possible hour. The same team, shipping the same amount of code in small pieces through the week, would have hit each problem on a Tuesday afternoon with one change to examine and a rollback that takes a minute.