How to find & fix duplicate content on your website

October 4, 2018October 4, 2018 Dominic Lill canonicalization, Copyscape, duplicate content, fix, Google, links, Moz, pages, quality, redirects, Screaming Frog, search engines, SEO, site crawl, Siteliner, tools, visibility, Website

If you’re worried about duplicate content, you’re not alone. According to estimates, almost 3 in 10 web pages have some sort of duplication problem.

While search engines won’t explicitly penalise your site for having duplicate content, it could impact your rankings indirectly. So, what can you do?

Start at the beginning…

Duplicate content – as the name suggests – is content that appears in multiple places online. This could be the same content on several different websites or on multiple webpages on the same site.

Either way, if there’s duplicate content on your website, you should find it and fix it.

Why is duplicate content a problem? In short, Google doesn’t want to present its users with the same content over and over again. When duplicate content is found, Google’s algorithms attempt to filter out the duplicates and find the original source. As a result, the visibility of any pages using duplicate content is diluted.

If this sounds all too familiar, make sure you act quickly to fix it before your site tumbles down the search results and into obscurity. Read on as we look at the best duplicate content tools and how to better optimise your site for search engines.

What are the best duplicate content tools?

1. Copyscape

First on the list is Copyscape, one of the longest-running duplicate content checking tools available. It’s the go-to site for a wide range of businesses – and with good reason too. You can quickly check a specific web page on your site against external content across the web.

Simply enter the URL of the page you want to check and press ‘Go’. Copyscape provides a list of results from around the web. We’ve used Google’s SEO Guide as an example to show how the results will look…

Clicking on one of the results will bring up a page of text from the external site with any duplicate copy highlighted. The tool also provides a summary of any plagiarised content, with the number of matching words and what percentage of the page that represents.

2. Siteliner

The Copyscape results page points users in the direction of another duplication tool – namely, Siteliner. This time, it’s to check internal pages – those on your own site. When you’re building a website, it’s hard, bordering on impossible, to keep track of what content is used on which pages.

The result? You may end up with the same content being used on multiple pages. Using Siteliner, you can check one page against the rest of your website to find any internal duplication. Again, insert the URL of the page you want to check into the search box.

Among the results is a handy chart showing how much of your site is made up of duplicate content. Clicking through to see your duplicate content gives you a table of the most heavily duplicated pages.

3. Screaming Frog

If you’re looking for more than just a duplication checker, Screaming Frog provides a range of useful tools within its SEO Spider. As the name playfully suggests, the Spider ‘crawls’ all the pages of your website and brings back a variety of insights to help you assess onsite SEO.

As well as exact duplicate pages – which you can identify by checking for matching ‘hash’ values, you get an overview of duplicate blocks of text, page titles, headings or meta descriptions. But that’s not all. You can also:

Check the length of page titles and meta data
Highlight any broken links or errors
Identify any redirects or redirect chains
Collect and review all HTML data including SKUs, prices or social meta tags

4. Moz Site Crawl

Another hugely useful site crawler comes from the SEO experts at Moz. The tool looks at a variety of elements on your site which can impact SEO. Because duplicate content can affect the performance of your site on search engines, Moz has made sure to include it on their list.

What’s great about the Moz Site Crawl is that it provides an overview of your site along with issues for specific pages. Your site’s problem areas are broken down into different categories, such as missing H1, title length or duplicate content, so you know where to focus your attention first.

How to fix duplicate content

There’s no need to worry if you find duplicate content on your site. Despite popular beliefs, Google doesn’t seek out and penalise duplication. They understand that most duplicate content isn’t deceptive or malicious. That’s why they have provided a guide to duplicate content and how to fix things. Here are the options:

Canonicalization

Don’t run a mile. Canonicalization isn’t as scary as it sounds. It simply means declaring which page you want Google to crawl when you have multiple pages with similar content.

That page is known as the canonical, which Google will choose for you if you don’t choose one yourself. The result? Visibility will be diluted for all pages with some unwanted results.

Let’s say you have a main service page and several other pages for sub-categories of this service, which use a lot of the same content. You would benefit much more from the main service page showing on the first search engine results page (SERP) compared to a more specific service. By setting the main page as a canonical, you can let Google know it’s your preferred page for SERP.

301 redirects

301s, for short, are a way of redirecting users from one URL to another. They’re useful when your site has been restructured, to ensure users are directed to the active pages you want them to see.

As well as redirecting users, they redirect search engines, which stops them crawling pages with duplicate content. By using 301s, you can ensure all the ranking power from your old content and pages is redirected to the right page.

More quick fixes

Canonicals and 301s are two of the most popular ways to eliminate duplicate content. However, there are a variety of other methods recommended by Google:

Consolidate pages – If multiple pages have similar content, it may be worth combining them, so you can minimise similar or duplicated content.
Back-links – As well as using content on multiple pages of the same site, you may have used it on several different sites – known as a syndicate. If that’s the case, it’s important to link back to the original article or web page.
Understand your CMS – Content may be displayed in more places than one when you publish it, such as a blog post being archived. It’s important to be aware of this and rectify it where possible.
Minimise repetition – Some content, like the terms and conditions or copyright text, will need to be included on multiple pages. That doesn’t mean you have to include the full text. Try to summarise on each page and link to another page for more details.

Don’t forget to create new content

Don’t overlook the importance of unique quality content. While the quick fixes help reduce the impact of duplication, they don’t provide the same benefit of original, organic content.

Where possible, identify key pages on your site and produce unique content for those pages. Be careful not to simply rewrite the content, however, as Google can easily detect it – and it could even be more harmful than duplicates.

Now it’s over to you. Good luck finding and fixing your duplicate content.

Alexis Pratsides is the Managing Director at MintTwist, an award-winning creative digital agency in London that delivers campaigns across web design, paid media, SEO, content & social.