Duplicate Content

A rumour followed this condition around for years: that search engines penalize it. There is no penalty rule. The damage is real anyway, and it comes from a choice rather than a punishment — a search engine will not show the same content twice in its results, so it picks one version.

Pagedeck's product page ran into it. Adding filters by company size and industry created an address per filter combination, about two thousand of them, with nearly identical content. What surfaced in search was not the product page but one of those combinations, so arrivals landed on a view filtered to a single industry.

The routes by which addresses multiply are predictable: tracking parameters, a version with a trailing slash alongside one without, http and https, list pages with a different sort order, print versions. None of them is a mistake, which is why they accumulate unnoticed.

The search engine picks a primary among the versions and drops the rest from results. Three kinds of damage follow.

1. Credit splits. Links other sites gave the article spread across four addresses, so the value one article earned arrives in four pieces.
2. The wrong address becomes primary. When a parameter-laden or filtered URL is what appears in results, the screen arrivals see is not the one you intended.
3. Crawl capacity gets spent. A bot sees a fixed number of pages a day, so while it works through two thousand shells, a newly published article waits its turn.

Content does not have to be identical to count. Twenty pages generated by swapping a city name, with the same skeleton and a few different words, get grouped together too.

It happens off your own site as well. When another site republishes an article verbatim, the same content exists on two domains, and if that site carries more weight the original gets pushed down.

Where addresses can be collapsed into one, collapse them; where both have to stay alive, add a canonical tag naming the primary; where an old address is finished, hand it over with a redirect.

"This is duplicate content. Every filter combination is generating a URL."

The same content is splitting across addresses as we speak.

"There's no penalty. The problem is the credit splitting."

Correcting what kind of damage this actually does.

"Searching the site turns up twelve pages with the same title."

Reporting the result of a check on the condition.

At Pagedeck, Ravi runs this check quarterly. He searches with the site restricted to see how many identical titles come back, and reads the list of reasons pages failed to get indexed alongside it. Theo takes the other side, making filter combinations stop turning into addresses, because that costs less than naming a primary after the fact.