Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Magento Dublicate Content (Noindex and Rel"canonical")
-
Hi All,
Just looking for some advice regarding my website on magento.
We by mistake didnt enable canonical tags and noindex tags so had a big problem with dublicate content from filter pages but also have URLs to Cats as Yes so this didnt help with not having canonical tags enabled.
We now have everything enabled for a few weeks now but dont see much drop in indexed pages in google. (currently 27k and we have only 5k products)
My question basically is how do we speed up noindexation of dublicate content and also would you change URL to cats as No so google just now sees the url to products? (my concerns with this is would leaving it to Yes help because it will hopefully read the canonical tags on products now)
Thank you in advance
Michael
-
Hi Carson
Thank you for replying and the indepth answers.
I did read somewhere that dublicate content on your own website isnt too bad but im glad you have helped me clear things up.
So would you change cat urls to no or leave them to yes for now till google can see all the canoical tags on products?
Thanks
Mike
-
I think there's an underlying assumption here that duplicate content will harm your site, and that's not necessarily true. There's no "duplicate content penalty" - it's more than a filter. Google is better than most at recognizing this, especially with common CMS like Magento and WP. Google attempts to look at the links going to both pages and understand their authority together.
Duplicate content is more of an issue if you're pulling content that others are using as well, e.g. on product descriptions provided by manufacturers and other types of content. Google won't "penalize" you, but they will sometimes filter your site out in favor of the most authoritative site with that content. It's also an issue (mostly for Panda) if you're creating keyword pages that contain duplicate of even very-similar content just to rank for a bunch of very similar keywords.
So my first bit of advice is, "don't obsess over intra-site duplicate content."
That said, it's best to reduce and avoid duplicate content 1) for less-sophisticated search engine, 2) for the sake of your own analytics data integrity and simplicity, 3) just in case Google doesn't get it (very rare).
Set the categories up however you think is best for the user (generally just the product name without categories), double-check the canonical URLs, and wait for Google to catch up on the canonical and noindex. It can take many months depending on your site's authority, but it's unlikely to move the needle either way. Keep in mind that Google may keep pages in the index even if they are honoring the canonical tag - they'll just show the canonical version but keep both indexed. That's working as intended - don't worry about that

Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
"5XX (Server Error)" - How can I fix this?
Hey Mozers! Moz Crawl tells me I am having an issue with my Wordpress category - it is returning a 5XX error and i'm not sure why? Can anyone help me determine the issue? Crawl Issues and Notices for: http://www.refusedcarfinance.com/news/category/news We found 1 crawler issue(s) for this page. High Priority Issues 1 5XX (Server Error) 5XX errors (e.g., a 503 Service Unavailable error) are shown when a valid request was made by the client, but the server failed to complete the request. This can indicate a problem with the server, and should be investigated and fixed.
Technical SEO | | RocketStats0 -
Product Variations (rel=canonical or 301) & Duplicate Product Descriptions
Hi All, Hoping for a bit of advice here please, I’ve been tasked with building an e-commerce store and all is going well so far. We decided to use Wordpress with Woocommerce as our shop plugin. I’ve been testing the CSV import option for uploading all our products and I’m a little concerned on two fronts: - Product Variations Duplicate content within the product descriptions **Product Variations: - ** We are selling furniture that has multiple variations (see list below) and as a result it creates c.50 product variations all with their own URL’s. Facing = Left, Right Leg style = Round, Straight, Queen Ann Leg colour = Black, White, Brown, Wood Matching cushion = Yes, No So my question is should I 301 re-direct the variation URL’s to the main product URL as from a user perspective they aren't used (we don't have images for each variation that would trigger the URL change, simply drop down options for the user to select the variation options) or should I add the rel canonical tag to each variation pointing back to the main product URL. **Duplicate Content: - ** We will be selling similar products e.g. A chair which comes in different fabrics and finishes, but is basically the same product. Most, if not all of the ‘long’ product descriptions are identical with only the ‘short’ product descriptions being unique. The ‘long’ product descriptions contain all the manufacturing information, leg option/colour information, graphics, dimensions, weight etc etc. I’m concerned that by having 300+ products all with identical ‘long’ descriptions its going to be seen negatively by google and effect the sites SEO. My question is will this be viewed as duplicate content? If so, are there any best practices I should be following for handling this, other than writing completely unique descriptions for each product, which would be extremely difficult given its basically the same products re-hashed. Many thanks in advance for any advice.
Technical SEO | | Jon-S0 -
Why do some URLs for a specific client have "/index.shtml"?
Reviewing our client's URLs for a 301 redirect strategy, we have noticed that many URLs have "/index.shtml." The part we don'd understand is these URLs aren't the homepage and they have multiple folders followed by "/index.shtml" Does anyone happen to know why this may be occurring? Is there any SEO value in keeping the "/index.shtml" in the URL?
Technical SEO | | FranFerrara0 -
"nofollow pages" or "duplicate content"?
We have a huge site with lots of geographical-pages in this structure: domain.com/country/resort/hotel domain.com/country/resort/hotel/facts domain.com/country/resort/hotel/images domain.com/country/resort/hotel/excursions domain.com/country/resort/hotel/maps domain.com/country/resort/hotel/car-rental Problem is that the text on ie. /excursions is often exactly the same on .../alcudia/hotel-sea-club/excursion and .../alcudia/hotel-beach-club/excursion The two hotels offer the same excursions, and the intro text on the pages are the exact same throughout the entire site. This is also a problem on the /images and /car-rental pages. I think in most cases the only difference on these pages is the Title, description and H1. These pages do not attract a lot of visits through search-engines. But to avoid them being flagged as duplicate content (we have more than 4000 of these pages - /excursions, /maps, /car-rental, /images), do i add a nofollow-tag to these, do i block them in robots.txt or should i just leave them and live with them being flagged as duplicate content? Im waiting for our web-team to add a function to insert a geographical-name in the text, so i could add ie #HOTELNAME# in the text and thereby avoiding the duplicate text. Right now we have intros like: When you visit the hotel ... instead of: When you visit Alcudia Sea Club But untill the web-team has fixed these GEO-tags, what should i do? What would you do and why?
Technical SEO | | alsvik0 -
NoIndex/NoFollow pages showing up when doing a Google search using "Site:" parameter
We recently launched a beta version of our new website in a subdomain of our existing site. The existing site is www.fonts.com with the beta living at new.fonts.com. We do not want Google to crawl the new site until it's out of beta so we have added the following on all pages: However, one of our team members noticed that google is displaying results from new.fonts.com when doing an "site:new.fonts.com" search (see attached screenshot). Is it possible that Google is indexing the content despite the noindex, nofollow tags? We have double checked the syntax and it seems correct except the trailing "/". I know Google still crawls noindexed pages, however, the fact that they're showing up in search results using the site search syntax is unsettling. Any thoughts would be appreciated! DyWRP.png
Technical SEO | | ChrisRoberts-MTI0 -
How is a dash or "-" handled by Google search?
I am targeting the keyword AK-47 and it the  variants in search (AK47, AK-47, AK 47) .  How should I handle on page SEO?  Right now I have AK47 and AK-47 incorporated. So my questions is really do I need to account for the space or is Google handling a dash as a space? At a quick glance of the top 10 it seems the dash is handled as a space, but I just wanted to get a conformation from people much smarter then I at seomoz. Thanks, Jason
Technical SEO | | idiHost0 -
How to structure rel=canonical for a e commerce site
Hello, So I have searched the Q & A , Google, the zen cart forum and at this point I am looking for some one to give a concrete answer on what I should do.  There is a lot of different opinions on " rel=canonical" and how to apply it , since there are many other variable in place. I have a zen cart site.  I am using the latest 1.3.9 version.  The default setting ( seem to me) uses the  rel=canonical to point back to the specific link product or category respectively. Most of the time I have two scenarios. 1. Main category --->  Sub category----> Product 2. Main Category----> Product I'll give an example http://www.perfectindesign.com/awards ---main category http://www.perfectindesign.com/awards/acrylic-awards sub category http://www.perfectindesign.com/awards/acrylic-awards/slanted-award product (this example has three sub categories with maybe 12 products in one 4 in the second and 5 in the third) From looking at the source code for each url it the rel=canonical just points back to its own url. I want to avoid competing against my self, for the example above keyword "acrylic awards" so should the use of the re=canonical be changes site wide to have products point back to sub categories when they exist and have products point back to main categories when no sub categories exist? I am very new to seo, specifically eCommerce seo.  If you have experience and have done this to a site you manage for a client or your own please advise how to proceed. Also if I'm missing some thing that will give me a better understanding of the bigger seo picture that would be great. Thanks, Yevgeny
Technical SEO | | Yevgeny0 -
Meta tag "noindex,nofollow" by accident
Hi, 3 weeks ago I wanted to release a new website (made in WordPress), so I neatly created 301 redirects for all files and folders of my old html website and transferred the WordPress site into the index folder. Job well done I thought, but after a few days, my site suddenly disappeared from google. I read in other Q&A's that this could happen so I waited a little longer till I finally saw today that there was a meta robots added on every page with "noindex, nofollow". For some reason, the WordPress setting "I want to forbid search engines, but allow normal visitors to my website" was selected, although I never even opened that section called "Privacy". So my question is, will this have a negative impact on my pagerank afterwards? Thanks, Sven
Technical SEO | | Zitana0