What's the most Google-friendly way to clean up a site's duplicate content?
 by Kristi Hagen

What's the most Google-friendly way to clean up a site's duplicate content?

  • I'm cleaning up a site that has many duplicate pages that we'd like removed from Google's index.

    I'm concerned that, if I just delete them, all sorts of 404's will be generated. And, if I NO INDEX them while they are still in the YOAST generated xml sitemaps, that could cause issues too.

    So, I wonder, what's the best solution to this problem?


The fact is, 404 errors are not always bad. A 404 is a natural occurrence on sites. The only "bad" thing about a 404 is the length of time it takes to get that missing content removed from the search index.

Unfortunately, search engines are often slow to perform housekeeping. And, Google provides no tools for us to deal with this in bulk, so we suggest you focus your efforts as follows;

  1. If your old URLs have some traffic and/or important inbound links you'd like to salvage, then you should 301 redirect them to an on-topic page within your site instead of deleting them.

  2. If they have no traffic and no links, and you're just getting rid of junk, then go ahead and delete them. Let them go 404 and ignore the Google Search Console (GSC) reporting error. Eventually they'll clear out.

  3. If you are in a rush to get the old URLs out of the index, then use the Removal Tool in Search Console – after the URLs have been removed/redirected. Btw, the tool was recently moved. You'll now find it in Search Console under the Index menu right below Sitemaps. Go here for Removals Tool info.SEN article end