Skip to content
    Moz logo Menu open Menu close
    • Products
      • Moz Pro
      • Moz Pro Home
      • Moz Local
      • Moz Local Home
      • STAT
      • Moz API
      • Moz API Home
      • Compare SEO Products
      • Moz Data
    • Free SEO Tools
      • Domain Analysis
      • Keyword Explorer
      • Link Explorer
      • Competitive Research
      • MozBar
      • More Free SEO Tools
    • Learn SEO
      • Beginner's Guide to SEO
      • SEO Learning Center
      • Moz Academy
      • MozCon
      • Webinars, Whitepapers, & Guides
    • Blog
    • Why Moz
      • Digital Marketers
      • Agency Solutions
      • Enterprise Solutions
      • Small Business Solutions
      • The Moz Story
      • New Releases
    • Log in
    • Log out
    • Products
      • Moz Pro

        Your all-in-one suite of SEO essentials.

      • Moz Local

        Raise your local SEO visibility with complete local SEO management.

      • STAT

        SERP tracking and analytics for enterprise SEO experts.

      • Moz API

        Power your SEO with our index of over 44 trillion links.

      • Compare SEO Products

        See which Moz SEO solution best meets your business needs.

      • Moz Data

        Power your SEO strategy & AI models with custom data solutions.

      Track AI Overviews in Keyword Research
      Moz Pro

      Track AI Overviews in Keyword Research

      Try it free!
    • Free SEO Tools
      • Domain Analysis

        Get top competitive SEO metrics like DA, top pages and more.

      • Keyword Explorer

        Find traffic-driving keywords with our 1.25 billion+ keyword index.

      • Link Explorer

        Explore over 40 trillion links for powerful backlink data.

      • Competitive Research

        Uncover valuable insights on your organic search competitors.

      • MozBar

        See top SEO metrics for free as you browse the web.

      • More Free SEO Tools

        Explore all the free SEO tools Moz has to offer.

      NEW Keyword Suggestions by Topic
      Moz Pro

      NEW Keyword Suggestions by Topic

      Learn more
    • Learn SEO
      • Beginner's Guide to SEO

        The #1 most popular introduction to SEO, trusted by millions.

      • SEO Learning Center

        Broaden your knowledge with SEO resources for all skill levels.

      • On-Demand Webinars

        Learn modern SEO best practices from industry experts.

      • How-To Guides

        Step-by-step guides to search success from the authority on SEO.

      • Moz Academy

        Upskill and get certified with on-demand courses & certifications.

      • MozCon

        Save on Early Bird tickets and join us in London or New York City

      Unlock flexible pricing & new endpoints
      Moz API

      Unlock flexible pricing & new endpoints

      Find your plan
    • Blog
    • Why Moz
      • Digital Marketers

        Simplify SEO tasks to save time and grow your traffic.

      • Small Business Solutions

        Uncover insights to make smarter marketing decisions in less time.

      • Agency Solutions

        Earn & keep valuable clients with unparalleled data & insights.

      • Enterprise Solutions

        Gain a competitive edge in the ever-changing world of search.

      • The Moz Story

        Moz was the first & remains the most trusted SEO company.

      • New Releases

        Get the scoop on the latest and greatest from Moz.

      Surface actionable competitive intel
      New Feature

      Surface actionable competitive intel

      Learn More
    • Log in
      • Moz Pro
      • Moz Local
      • Moz Local Dashboard
      • Moz API
      • Moz API Dashboard
      • Moz Academy
    • Avatar
      • Moz Home
      • Notifications
      • Account & Billing
      • Manage Users
      • Community Profile
      • My Q&A
      • My Videos
      • Log Out

    The Moz Q&A Forum

    • Forum
    • Questions
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. Home
    2. SEO Tactics
    3. Technical SEO
    4. Diagnosing Canonical Errors Is Screaming frog reliable?

    Moz Q&A is closed.

    After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

    Diagnosing Canonical Errors Is Screaming frog reliable?

    Technical SEO
    3
    9
    2217
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with question management privileges can see it.
    • Nightwing
      Nightwing last edited by

      Morning from suny & warm wetherby UK 🙂

      On this page http://www.goldsboroughestates.co.uk/how-we-care-for-you/right-to-manage/ screaming frog is citing a canonical error but I'm confused as this piece of code is in place:

      http://www.goldsboroughestates.co.uk/About/right-to-manage" />

      So my question is please - "Does this page  http://www.goldsboroughestates.co.uk/how-we-care-for-you/right-to-manage/ have a caninical error or is screaming frog useless?

      Other examples where screaming frog is picking up canonical errors include:
      http://www.goldsboroughestates.co.uk/what-our-customers-say/right-to-manage/
      http://www.goldsboroughestates.co.uk/buying-a-home/right-to-manage/

      Oh forgot to say the preffered version is http://www.goldsboroughestates.co.uk/About/right-to-manage/

      Any insights welcvome 🙂

      1 Reply Last reply Reply Quote 0
      • Keszi
        Keszi @Nightwing last edited by

        Hey,

        Long time since the Question, I was just wondering if you worked it out or not.

        Gr.,

        Istvan

        1 Reply Last reply Reply Quote 0
        • Dr-Pete
          Dr-Pete Staff @Nightwing last edited by

          I think Screaming Frog is just warning you that the canonical version doesn't seem to match the display URL. They can't really tell (we have the same problem in SEOmoz tools) what the "right" canonical is - they can just warn of a mismatch.

          I'm a bit confused as to the purpose of the dual URLs here. The best canonical implementation is to use one URL. The canonical tag can act as a band-aid, but  consistency is still the best defense. Having multiple paths to the same page is rarely beneficial.

          1 Reply Last reply Reply Quote 0
          • Nightwing
            Nightwing @Nightwing last edited by

            Having spoke to oiur internal helpdesk (Who I trust & do know what theyre talking about) theyve taken a look at:

            http://www.goldsboroughestates.co.uk/footer-links/left/right-to-manage/

            http://www.goldsboroughestates.co.uk/how-we-care-for-you/right-to-manage/

            http://www.goldsboroughestates.co.uk/buying-a-home/right-to-manage/

            http://www.goldsboroughestates.co.uk/what-our-customers-say/right-to-manage/

            and I'm afraid they have a different perspective which is they see no canonical problem 😞 Hey ho think I'll just set my head on fire then maybe things will be more clearer 😉

            1 Reply Last reply Reply Quote 0
            • Nightwing
              Nightwing @Nightwing last edited by

              Hi Istvan - your advice is good but ive just discovered its not been implemented! Time to kick some ass, I'll update you 🙂

              1 Reply Last reply Reply Quote 0
              • Keszi
                Keszi @Nightwing last edited by

                Hey,

                Any news on how it went? I am curious if that was the problem or not. 🙂

                Gr.,

                Istvan

                1 Reply Last reply Reply Quote 0
                • Keszi
                  Keszi @Nightwing last edited by

                  Hey,

                  Maybe this helps you a littlebit: http://www.seomoz.org/blog/an-seos-guide-to-http-status-codes

                  Dr. Pete's article explains well how the status codes work.

                  Gr.,

                  Istvan

                  1 Reply Last reply Reply Quote -1
                  • Nightwing
                    Nightwing @Keszi last edited by

                    Wow great anser, I'm on to this now & will updat you with how things went 🙂

                    Keszi Nightwing Dr-Pete 6 Replies Last reply Reply Quote 0
                    • Keszi
                      Keszi last edited by

                      Hey there!

                      I think I have found what your problem is with you canonical link 🙂

                      In your code you have:

                      And probably you are somewhere forcing the URls to have a / at the end.

                      So basically you are confusing browsers and search engine bots, because they now cannot tell which is the real version:

                      SE enters the page. Then it sees that the right version should be the one WITHOUT a "/" at the end, then that pages has a 301 redirect to the version which HAS a "/" at the end of the URL (but that has a canonical which points out that the preffered version should be ). So it is a non-ending circle.

                      So if you add a / to the end of your URl, your problem should be solved.

                      Final thought: Screaming Frog is working well.

                      I hope this was a solution.

                      Cheers,

                      Istvan

                      Nightwing 1 Reply Last reply Reply Quote 2
                      • 1 / 1
                      • First post
                        Last post

                      Got a burning SEO question?

                      Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.


                      Start my free trial


                      Browse Questions

                      Explore more categories

                      • Moz Tools

                        Chat with the community about the Moz tools.

                      • SEO Tactics

                        Discuss the SEO process with fellow marketers

                      • Community

                        Discuss industry events, jobs, and news!

                      • Digital Marketing

                        Chat about tactics outside of SEO

                      • Research & Trends

                        Dive into research and trends in the search industry.

                      • Support

                        Connect on product support and feature requests.

                      • See all categories

                      Related Questions

                      • andrew-mso

                        Canonical or hreflang?

                        I have four English sites for four different countries, UK, Ireland, Australia and New Zealand and I want to share some content between the sites. On the pages that share the content, which is essentially exactly the same on all 4 sites, do I use the hreflang tags like: or do I add a canonical tag to the other three pointing to the "origin", which would be the UK site? I believe it is best practice to use one or the other, but I'm not sure which make sense in this situation.

                        Technical SEO | | andrew-mso
                        0
                      • ReSEOlve

                        404 errors

                        Hi I am getting these show up in WMT crawl error any help would be very much appreciated | ?escaped_fragment=Meditation-find-peace-within/csso/55991bd90cf2efdf74ec3f60 | 404 | 12/5/15 |
                        | | 2 | mobile/?escaped_fragment= | 404 | 10/26/15 |
                        | | 3 | ?escaped_fragment=Tips-for-a-balanced-lifestyle/csso/1 | 404 | 12/1/15 |
                        | | 4 | ?escaped_fragment=My-favorite-yoga-spot/csso/5598e2130cf2585ebcde3b9a | 404 | 12/1/15 |
                        | | 5 | ?escaped_fragment=blog/c19s6 | 404 | 11/29/15 |
                        | | 6 | ?escaped_fragment=blog/c19s6/Tag/yoga | 404 | 11/30/15 |
                        | | 7 | ?escaped_fragment=Inhale-exhale-and-once-again/csso/2 | 404 | 11/27/15 |
                        | | 8 | ?escaped_fragment=classes/covl | 404 | 10/29/15 |
                        | | 9 | m/?escaped_fragment= | 404 | 10/26/15 |
                        | | 10 | ?escaped_fragment=blog/c19s6/Page/1 | 404 | 11/30/15 | |   |

                        Technical SEO | | ReSEOlve
                        0
                      • Merkle-Impaqt

                        Can you use Screaming Frog to find all instances of relative or absolute linking?

                        My client wants to pull every instance of an absolute URL on their site so that they can update them for an upcoming migration to HTTPS (the majority of the site uses relative linking). Is there a way to use the extraction tool in Screaming Frog to crawl one page at a time and extract every occurrence of _href="http://" _? I have gone back and forth between using an x-path extractor as well as a regex and have had no luck with either. Ex. X-path: //*[starts-with(@href, “http://”)][1] Ex. Regex: href=\”//

                        Technical SEO | | Merkle-Impaqt
                        0
                      • SeaCandyTackle

                        Rel=canonical Weebly

                        My problem is with my website as it says I have duplicate page titles and contents because of a /index.html. It says the duplicate content is due to the fact that my homepage on my website is www.seacandytackle.com but it is also www.seacandytackle.com/index.html because I use weebly. How can I use the tag to fix this? It won't let me do a 301 redirect because it is a home page. How can I fix this? What code would I have to use and which url? Also it says that I have duplicate page content between http://www.seacandytackle.com/index.html and http://www.seacandytackle.comhttp://www.seacandytackle.com but I don't recall having any page that looks like http://www.seacandytackle.com http://www.seacandytackle.com from weebly. How can I fix this issue as well? Thank you for any help. Step by step implementation would be particularly helpful in using the rel= tags to fix these duplicate issues.

                        Technical SEO | | SeaCandyTackle
                        0
                      • DonnaDuncan

                        Exclude status codes in Screaming Frog

                        I have a very large ecommerce site I'm trying to spider using screaming frog. Problem is I keep hanging even though I have turned off the high memory safeguard under configuration. The site has approximately 190,000 pages according to the results of a Google site: command. The site architecture is almost completely flat. Limiting the search by depth is a possiblity, but it will take quite a bit of manual labor as there are literally hundreds of directories one level below the root. There are many, many duplicate pages. I've been able to exclude some of them from being crawled using the exclude configuration parameters. There are thousands of redirects. I haven't been able to exclude those from the spider b/c they don't have a distinguishing character string in their URLs. Does anyone know how to exclude files using status codes? I know that would help. If it helps, the site is kodylighting.com. Thanks in advance for any guidance you can provide.

                        Technical SEO | | DonnaDuncan
                        0
                      • sidstar

                        The Mysterious Case of Pagination, Canonical Tags

                        Hey guys, My head explodes when I think of this problem. So I will leave it to you guys to find a solution... My root domain (xxx.com) runs on WordPress platform. I use Yoast SEO plugin. The next page of root domain -- page/2/ -- has been canonicalized to the same page -- page/2/ points to page/2/ for example. The page/2/ and remaining pages also have this rel tags: I have also added "noindex,follow" to page/2/ and further -- Yoast does this automatically. Note: Yoast plugin also adds canonical to page/2/...page/3/ automatically. Same is the case with category pages and tag pages. Oh, and the author pages too -- they all have self-canonicalization, rel prev & rel next tags, and have been "noindex, followed." Problem: Am I doing this the way it should be done? I asked a Google Webmaster employee on rel next and prev tags, and this is what she said: "We do not recommend noindexing later pages, nor rel="canonical"izing everything to the first page." (My bad, last year I was canonicalizing pages to first page). One of the popular blog, a competitor, uses none of these tags. Yet they rank higher. Others following this format have been hit with every kind of Google algorithm I could think of. I want to leave it to Google to decide what's better, but then again, Yoast SEO plugin rules my blog -- okay, let's say I am a bad coder. Any help, suggestions, and thoughts are highly appreciated. 🙂 Update 1: Paginated pages -- including category pages and tag pages -- have unique snippets; no full-length posts. Thought I'd make that clear.

                        Technical SEO | | sidstar
                        0
                      • MagicDude4Eva

                        Duplicate title-tags with pagination and canonical

                        Some time back we implemented the Google recommendation for pagination (the rel="next/prev"). GWMT now reports 17K pages with duplicate title-tags (we have about 1,1m products on our site and about 50m pages indexed in Google) As an example we have properties listed in various states and the category title would be "Properties for Sale in [state-name]". A paginated search page or browsing a category (see also http://searchengineland.com/implementing-pagination-attributes-correctly-for-google-114970) would then include the following: The title for each page is the same - so to avoid the duplicate title-tags issue, I would think one would have the following options: Ignore what Google says Change the canonical to http://www.site.com/property/state.html (which would then only show the first XX results) Append a page number to the title "Properties for Sale in [state-name] | Page XX" Have all paginated pages use noindex,follow - this would then result in no category page being indexed Would you have the canonical point to the individual paginated page or the base page?

                        Technical SEO | | MagicDude4Eva
                        2
                      • JohnBerger

                        Rel = prev next AND canonical?

                        I have product category pages that correctly have the prev next but the moz crawl is giving me duplicate content errors.  I would not think I also need to have canonical - but do I ?

                        Technical SEO | | JohnBerger
                        0

                      Get started with Moz Pro!

                      Unlock the power of advanced SEO tools and data-driven insights.

                      Start my free trial
                      Products
                      • Moz Pro
                      • Moz Local
                      • Moz API
                      • Moz Data
                      • STAT
                      • Product Updates
                      Moz Solutions
                      • SMB Solutions
                      • Agency Solutions
                      • Enterprise Solutions
                      • Digital Marketers
                      Free SEO Tools
                      • Domain Authority Checker
                      • Link Explorer
                      • Keyword Explorer
                      • Competitive Research
                      • Brand Authority Checker
                      • Local Citation Checker
                      • MozBar Extension
                      • MozCast
                      Resources
                      • Blog
                      • SEO Learning Center
                      • Help Hub
                      • Beginner's Guide to SEO
                      • How-to Guides
                      • Moz Academy
                      • API Docs
                      About Moz
                      • About
                      • Team
                      • Careers
                      • Contact
                      Why Moz
                      • Case Studies
                      • Testimonials
                      Get Involved
                      • Become an Affiliate
                      • MozCon
                      • Webinars
                      • Practical Marketer Series
                      • MozPod
                      Connect with us

                      Contact the Help team

                      Join our newsletter
                      Moz logo
                      © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                      • Accessibility
                      • Terms of Use
                      • Privacy

                      Looks like your connection to Moz was lost, please wait while we try to reconnect.