• majorAlexa

        See all notifications

        Skip to content
        Moz logo Menu open Menu close
        • Products
          • Moz Pro
          • Moz Pro Home
          • Moz Local
          • Moz Local Home
          • STAT
          • Moz API
          • Moz API Home
          • Compare SEO Products
          • Moz Data
        • Free SEO Tools
          • Domain Analysis
          • Keyword Explorer
          • Link Explorer
          • Competitive Research
          • MozBar
          • More Free SEO Tools
        • Learn SEO
          • Beginner's Guide to SEO
          • SEO Learning Center
          • Moz Academy
          • MozCon
          • Webinars, Whitepapers, & Guides
        • Blog
        • Why Moz
          • Digital Marketers
          • Agency Solutions
          • Enterprise Solutions
          • Small Business Solutions
          • The Moz Story
          • New Releases
        • Log in
        • Log out
        • Products
          • Moz Pro

            Your all-in-one suite of SEO essentials.

          • Moz Local

            Raise your local SEO visibility with complete local SEO management.

          • STAT

            SERP tracking and analytics for enterprise SEO experts.

          • Moz API

            Power your SEO with our index of over 44 trillion links.

          • Compare SEO Products

            See which Moz SEO solution best meets your business needs.

          • Moz Data

            Power your SEO strategy & AI models with custom data solutions.

          Let your business shine with Listings AI
          Moz Local

          Let your business shine with Listings AI

          Learn more
        • Free SEO Tools
          • Domain Analysis

            Get top competitive SEO metrics like DA, top pages and more.

          • Keyword Explorer

            Find traffic-driving keywords with our 1.25 billion+ keyword index.

          • Link Explorer

            Explore over 40 trillion links for powerful backlink data.

          • Competitive Research

            Uncover valuable insights on your organic search competitors.

          • MozBar

            See top SEO metrics for free as you browse the web.

          • More Free SEO Tools

            Explore all the free SEO tools Moz has to offer.

          NEW Keyword Suggestions by Topic
          Moz Pro

          NEW Keyword Suggestions by Topic

          Learn more
        • Learn SEO
          • Beginner's Guide to SEO

            The #1 most popular introduction to SEO, trusted by millions.

          • SEO Learning Center

            Broaden your knowledge with SEO resources for all skill levels.

          • On-Demand Webinars

            Learn modern SEO best practices from industry experts.

          • How-To Guides

            Step-by-step guides to search success from the authority on SEO.

          • Moz Academy

            Upskill and get certified with on-demand courses & certifications.

          • MozCon

            Save on Early Bird tickets and join us in London or New York City

          Unlock flexible pricing & new endpoints
          Moz API

          Unlock flexible pricing & new endpoints

          Find your plan
        • Blog
        • Why Moz
          • Digital Marketers

            Simplify SEO tasks to save time and grow your traffic.

          • Small Business Solutions

            Uncover insights to make smarter marketing decisions in less time.

          • Agency Solutions

            Earn & keep valuable clients with unparalleled data & insights.

          • Enterprise Solutions

            Gain a competitive edge in the ever-changing world of search.

          • The Moz Story

            Moz was the first & remains the most trusted SEO company.

          • New Releases

            Get the scoop on the latest and greatest from Moz.

          Surface actionable competitive intel
          New Feature

          Surface actionable competitive intel

          Learn More
        • Log in
          • Moz Pro
          • Moz Local
          • Moz Local Dashboard
          • Moz API
          • Moz API Dashboard
          • Moz Academy
        • Avatar
          • Moz Home
          • Notifications
          • Account & Billing
          • Manage Users
          • Community Profile
          • My Q&A
          • My Videos
          • Log Out

        The Moz Q&A Forum

        • Forum
        • Questions
        • My Q&A
        • Users
        • Ask the Community

        Welcome to the Q&A Forum

        Browse the forum for helpful insights and fresh discussions about all things SEO.

        1. Home
        2. Digital Marketing
        3. Web Design
        4. Best method to stop crawler access to extra Nav Menu

        Moz Q&A is closed.

        After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

        Best method to stop crawler access to extra Nav Menu

        Web Design
        3
        4
        1571
        Loading More Posts
        • Watching

          Notify me of new replies.
          Show question in unread.

        • Not Watching

          Do not notify me of new replies.
          Show question in unread if category is not ignored.

        • Ignoring

          Do not notify me of new replies.
          Do not show question in unread.

        • Oldest to Newest
        • Newest to Oldest
        • Most Votes
        Reply
        • Reply as question
        Locked
        This topic has been deleted. Only users with question management privileges can see it.
        • DWJames
          DWJames last edited by

          Our shop site has a 3 tier drop down mega-menu so it's easy to find your way to anything from anywhere. It contains about 150 links and probably 300 words of text.

          We also have a more context-driven single layer of sub-category navigation as well as breadcrumbs on our category pages.

          You can get to every product and category page without using the drop down mega-menu.

          Although the mega-menu is a helpful tool for customers, it means that every single page in our shop has an extra 150 links on it that go to stuff that isn't necessarily related or relevant to the page content. This means that when viewed from the context of a crawler, rather than a nice tree like crawling structure, we've got more of an unstructured mesh where everything is linked to everything else.

          I'd like to hide the mega-menu links from being picked up by a crawler, but what's the best way to do this?

          I can add a nofollow to all mega-menu links, but are the links still registered as page content even if they're not followed? It's a lot of text if nothing else.

          Another possibility we're considering is to set the mega-menu to only populate with links when it's main button is hovered over. So it's not part of the initial page load content at all.

          Or we could use a crude yet effective system we have used for some other menus we have of base encoding the content inline so it's not readable by a spider.

          What would you do and why?

          Thanks,

          James

          1 Reply Last reply Reply Quote 0
          • blacey
            blacey @Click2Rank last edited by

            I agree Alan,

            Mega Menu's are a good way to dilate the link equity of your page and in most cases it isn't needed at all. Keep the top-level navigation simple and have a submenu on all pages that contain links relevant to that section.

            EG: Mega Menu could be:

            Home Mens (Mens Tops, Mens Jeans, Mens Coats), Women (Womens Tops, Womens Jeans etc) Contact us

            In this example it would be better to have one top level menu for:

            Home | Mens | Women | Contact us

            Then when your in the men or women section show links to "Tops", "Jeans" and "Coats". That way those links are relevant to the section you're in and reinforces the structure of that section to search engines.

            After giving it further thought I would suggest not having a mega menu at all, because it may harm your SEO on-page optimisation efforts in the long term.

            1 Reply Last reply Reply Quote 0
            • Click2Rank
              Click2Rank last edited by

              Ben's partially correct.  Unfortunately Google has been claiming they do process Javascript for a while, and they recently stated they've begun reading AJAX.  Of course they do a lousy job of it and don't always get it right, which just makes things even more muddy.

              So from an SEO best practices perspective, you shouldn't have the menu(s) in the first place, at all.

              You may also THINK their good for users but has any significant study been performed to confirm that?  You'd need to check click-through rates on all the links to know for sure.

              What I've found through years of auditing sites that have such menus is that it almost always turns out to be the case where most of the deeper links NEVER get clicked on from within these menus.  Instead, they're overwhelming to users.  This is why it's better to not have them from a UX perspective.

              If you abandon them and go with more traditional hierarchical index and sub-index pages, and if those are properly optimized, you'll not only eliminate the massive SEO problem but in fact get more of your category pages to have higher ranking strength and authority over time.

              IF you're going to keep them in any form because you don't want to go to the extreme I recommend, then yes - AJAX would likely be the only scenario that offers the least likelihood of search engines choking on the over-use of links.

              And for the record, the real current problem with all those links on every page is duplicate content confusion - all of those URLS at the source level dilutes the uniqueness of content on every page of the site.  And that also means you're harming the topical focus of every page as well.  So whatever you do, AJAX or doing away with them altogether is going to be of high value long term.

              • Alan Bleiweiss - Click2Rank's Search Team Director
              blacey 1 Reply Last reply Reply Quote 1
              • blacey
                blacey last edited by

                From my experience I don't think you can really 'hide' the megamenu links from a crawler if they are generated using a content management system (code server side). If the link is on the page in the HTML then it will be crawled by a bot etc.

                The general method of getting a mega menu to work is through the use of CSS and JavaScript, so you might want to have a look at using AJAX to get the relevant links from the database and then use JavaScript to put the links into the page.

                This isn't a great solution, but bots cannot load JavaScript, so what they will see is only the links that are served up from the content management system.

                1 Reply Last reply Reply Quote 0
                • 1 / 1
                • First post
                  Last post

                Browse Questions

                Explore more categories

                • Moz Tools

                  Chat with the community about the Moz tools.

                • SEO Tactics

                  Discuss the SEO process with fellow marketers

                • Community

                  Discuss industry events, jobs, and news!

                • Digital Marketing

                  Chat about tactics outside of SEO

                • Research & Trends

                  Dive into research and trends in the search industry.

                • Support

                  Connect on product support and feature requests.

                • See all categories

                Related Questions

                • Joe_Stoffel

                  Have Your Thoughts Changed Regarding Canonical Tag Best Practice for Pagination? - Google Ignoring rel= Next/Prev Tagging

                  Hi there, We have a good-sized eCommerce client that is gearing up for a relaunch. At this point, the staging site follows the previous best practice for pagination (self-referencing canonical tags on each page; rel=next & prev tags referencing the last and next page within the category). Knowing that Google does not support rel=next/prev tags, does that change your thoughts for how to set up canonical tags within a paginated product category? We have some categories that have 500-600 products so creating and canonicalizing to a 'view all' page is not ideal for us. That leaves us with the following options (feel it is worth noting that we are leaving rel=next / prev tags in place): Leave canonical tags as-is, page 2 of the product category will have a canonical tag referencing ?page=2 URL Reference Page 1 of product category on all pages within the category series, page 2 of product category would have canonical tag referencing page 1 (/category/) - this is admittedly what I am leaning toward. Any and all thoughts are appreciated! If this were in relation to an existing website that is not experiencing indexing issues, I wouldn't worry about these. Given we are launching a new site, now is the time to make such a change. Thank you! Joe

                  Web Design | | Joe_Stoffel
                  1
                • QBSEO

                  What’s the best tool to visualize internal link structure and relationships between pages on a single site?

                  I‘d like to review the internal linking structure on my site. Is there a tool that can visualize the relationships between all of the pages within my site?

                  Web Design | | QBSEO
                  0
                • LauraFalls

                  Have an eBook. What is best practice for SEO?

                  Hello We have a free eBook - its a great resource and great piece of content. It is available to download on our website here - http://re-timer.com/the-product/how-to-sleep-better/ The book is available as a whole or as individual chapters (i.e. http://re-timer.com/app/uploads/2015/07/Chapter8.pdf?b0df38). The PDF chapters appear to be doing well in Google search for certain keywords. I can't measure this in GA though. I would like the eBook to assist the SEO of my website overall. If I create a web page and 'embedded' the PDF into it will Google still crawl this page? At the moment we are also using this to collect email addresses, this is a nice to have and it is OK if people get the eBook without doing this (if they find a chapter in Google they currently don't have to enter their email address). I'm sure lots of people have eBooks now. What is best practice and the best way to use this as a tool to maximise SEO for the whole website (http://re-timer.com)? Thank you! Laura

                  Web Design | | LauraFalls
                  1
                • johncurlee

                  Best practice for multilanguage website ( PHP feature based on Browser or Geolocalisation)

                  Hi Moz Experts I would like to know what does it the best practice for multilanguage website for the default language ? There are several PHP features to help users to get the right language when they come from SEO and direct; present the default language by browser language, by gelolocalisation, etc. However, which one is the most appropriate for Quebec company that try to get outside Canada ? PRO and CONS. Thank you in advance.

                  Web Design | | johncurlee
                  0
                • Kingalan1

                  Lots of Listing Pages with Thin Content on Real Estate Web Site-Best to Set them to No-Index?

                  Greetings Moz Community: As a commercial real estate broker in Manhattan I run a web site with over 600 pages. Basically the pages are organized in the following categories: 1. Neighborhoods (Example:http://www.nyc-officespace-leader.com/neighborhoods/midtown-manhattan)  25 PAGES Low bounce rate 2. Types of Space (Example:http://www.nyc-officespace-leader.com/commercial-space/loft-space) 
                  15 PAGES Low bounce rate. 3. Blog (Example:http://www.nyc-officespace-leader.com/blog/how-long-does-leasing-process-take 
                  30 PAGES Medium/high bounce rate 4. Services (Example:http://www.nyc-officespace-leader.com/brokerage-services/relocate-to-new-office-space)  High bounce rate
                  3 PAGES 5. About Us (Example:http://www.nyc-officespace-leader.com/about-us/what-we-do
                  4 PAGES High bounce rate 6. Listings (Example:http://www.nyc-officespace-leader.com/listings/305-fifth-avenue-office-suite-1340sf)
                  300 PAGES High bounce rate (65%), thin content 7. Buildings (Example:http://www.nyc-officespace-leader.com/928-broadway
                  300 PAGES  Very high bounce rate (exceeding 75%) Most of the listing pages do not have more than 100 words.  My SEO firm is advising me to set them "No-Index, Follow". They believe the thin content could be hurting me. Is this an acceptable strategy? I am concerned that when Google detects 300 pages set to "No-Follow" they could interpret this as the site seeking to hide something and penalize us. Also, the building pages have a low click thru rate. Would it make sense to set them to "No-Follow" as well? Basically, would it increase authority in Google's eyes if we set pages that have thin content and/or low click thru rates to "No-Follow"? Any harm in doing this for about half the pages on the site? I might add that while I don't suffer from any manual penalty volume has gone down substantially in the last month. We upgraded the site in early June and somehow 175 pages were submitted to Google  that should not have been indexed. A removal request has been made for those pages. Prior to that we were hit by Panda in April 2012 with search volume dropping from about 7,000 per month to 3,000 per month. Volume had increased back to 4,500 by April this year only to start tanking again. It was down to 3,600 in June. About 30 toxic links were removed in late April and a disavow file was submitted with Google in late April for removal of links from 80 toxic domains. Thanks in advance for your responses!! Alan

                  Web Design | | Kingalan1
                  0
                • hwade

                  Side Nav. Vs. Top Nav

                  I have  a client that currently has a side navigation and wants to know how changing to a top nav will affect her SEO. We always recommend top nav for user experience but I am not sure if there is a direct effect on SEO. Would the change affect it? Thoughts?

                  Web Design | | hwade
                  0
                • BigBlaze205

                  Best Website Builder - Help Me Choose

                  I need to built a multi language site (to built a Pilates, Yoga site) and I will use a site builder. After posting questions on wix.com I came to the fact I should continue my research because there are not SEO friendly. Do you have a suggestions? Limited to html knowledge, using a website builder is my only option. Here are some of the features I need: Multilanguage Web Site Mobile version SEO Friendly Nice Template Selections( this is important) HTML customization Twitter, Facebook, Blog... I'm not looking at free website builder, when you want good features, there is a price to paid. Thank you for your help and suggestions, BigBlaze

                  Web Design | | BigBlaze205
                  0
                • wazza1985

                  What CMS system is best?

                  Hi Guys, I am in the process of developing some new areas of work and a good CMS is going to be required to make my SEO and everything included work to its best. What CMS systems are the best ones to use from an SEO prospective and generally all round? Thanks

                  Web Design | | wazza1985
                  0

                Get started with Moz Pro!

                Unlock the power of advanced SEO tools and data-driven insights.

                Start my free trial
                Products
                • Moz Pro
                • Moz Local
                • Moz API
                • Moz Data
                • STAT
                • Product Updates
                Moz Solutions
                • SMB Solutions
                • Agency Solutions
                • Enterprise Solutions
                • Digital Marketers
                Free SEO Tools
                • Domain Authority Checker
                • Link Explorer
                • Keyword Explorer
                • Competitive Research
                • Brand Authority Checker
                • Local Citation Checker
                • MozBar Extension
                • MozCast
                Resources
                • Blog
                • SEO Learning Center
                • Help Hub
                • Beginner's Guide to SEO
                • How-to Guides
                • Moz Academy
                • API Docs
                About Moz
                • About
                • Team
                • Careers
                • Contact
                Why Moz
                • Case Studies
                • Testimonials
                Get Involved
                • Become an Affiliate
                • MozCon
                • Webinars
                • Practical Marketer Series
                • MozPod
                Connect with us

                Contact the Help team

                Join our newsletter
                Moz logo
                © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                • Accessibility
                • Terms of Use
                • Privacy

                Looks like your connection to Moz was lost, please wait while we try to reconnect.