Tell me more ×
Webmasters Stack Exchange is a question and answer site for pro webmasters. It's 100% free, no registration required.
up vote 2 down vote favorite

Most review sites, for example sites that review games with different platforms, will often have three URL's with identical content, only differing in color schemes, and of course comments. This is true of some large sites like ign.com.

How do these sites deal with duplicate content? Do they use a rel=canonical link to point to only one URL?

share|improve this question
Thanks @dan ! Good edit – André Lucas Jun 9 at 21:23
:-) Good question! – dan Jun 10 at 4:36
Just because ign.com has some duplicate content does not mean that it is unavoidable. – DisgruntledGoat Jun 10 at 9:22
thanks for the comment @DisgruntledGoat , i agree as i used their situation as an example. Cheers – André Lucas Jun 10 at 17:16

1 Answer

up vote 3 down vote accepted

Yes, this is what they should use to indicate where the original content can be found, especially if they want that content to be indexed only for the main site/page that it can be found on.

In the case of ign.com, I can see they have in their HTML source code:

<link rel="canonical" href="http://www.ign.com/" />

Without knowing the other URL's that you're referring to, I can only guess that they have this link in them also whenever duplicate content can be found. The alternative would be to specify in robots.txt not to crawl these other review pages.

On the other hand, if they want the comments to also be indexed, and they are not dynamically generated or too complex for the search engine bots to index, they might just leave it to the search engine bot to decide. In this case, they would have pages competing against each other for the same content, with the potential of the main site/page losing out.

Here's a link from a Google Webmaster Tools answer for more information on this: About rel="canonical"

share|improve this answer
That was really good, @dan ! Thank you! – André Lucas Jun 9 at 21:25
Hi, @dan , It's interesting though, because they (IGN) have three identical pages (only color and a few user related social stuff different) and everyone of them have a canonical link to themselves. Check those two for instance: ign.com/games/bioshock-infinite/pc-14258733 and ign.com/games/bioshock-infinite/xbox-360-14258637 – André Lucas Jun 9 at 22:59
Thanks, you're welcome! The pages for those URL's have iframes in them, so without dissecting them too much, it's possible they iframe the same source, which in most cases would not get indexed for the containing page. Also, take a look at this tag: <meta name="robots" content="noodp, noydir" /> This tells Yahoo and Google not to list them in their directories. Lastly, just because they're a large site and are or are not doing something doesn't mean that they're doing it right :-) – dan Jun 10 at 4:30
Hahaha, @dan, that's true! Maybe that's just because they dind't fully optmize their site for SEO. Those are interesting approaches, though. I'll look deeper on that! Thanks :) – André Lucas Jun 10 at 5:53

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.