• majorAlexa

        See all notifications

        Skip to content
        Moz logo Menu open Menu close
        • Products
          • Moz Pro
          • Moz Pro Home
          • Moz Local
          • Moz Local Home
          • STAT
          • Moz API
          • Moz API Home
          • Compare SEO Products
          • Moz Data
        • Free SEO Tools
          • Domain Analysis
          • Keyword Explorer
          • Link Explorer
          • Competitive Research
          • MozBar
          • More Free SEO Tools
        • Learn SEO
          • Beginner's Guide to SEO
          • SEO Learning Center
          • Moz Academy
          • MozCon
          • Webinars, Whitepapers, & Guides
        • Blog
        • Why Moz
          • Digital Marketers
          • Agency Solutions
          • Enterprise Solutions
          • Small Business Solutions
          • The Moz Story
          • New Releases
        • Log in
        • Log out
        • Products
          • Moz Pro

            Your all-in-one suite of SEO essentials.

          • Moz Local

            Raise your local SEO visibility with complete local SEO management.

          • STAT

            SERP tracking and analytics for enterprise SEO experts.

          • Moz API

            Power your SEO with our index of over 44 trillion links.

          • Compare SEO Products

            See which Moz SEO solution best meets your business needs.

          • Moz Data

            Power your SEO strategy & AI models with custom data solutions.

          Let your business shine with Listings AI
          Moz Local

          Let your business shine with Listings AI

          Learn more
        • Free SEO Tools
          • Domain Analysis

            Get top competitive SEO metrics like DA, top pages and more.

          • Keyword Explorer

            Find traffic-driving keywords with our 1.25 billion+ keyword index.

          • Link Explorer

            Explore over 40 trillion links for powerful backlink data.

          • Competitive Research

            Uncover valuable insights on your organic search competitors.

          • MozBar

            See top SEO metrics for free as you browse the web.

          • More Free SEO Tools

            Explore all the free SEO tools Moz has to offer.

          NEW Keyword Suggestions by Topic
          Moz Pro

          NEW Keyword Suggestions by Topic

          Learn more
        • Learn SEO
          • Beginner's Guide to SEO

            The #1 most popular introduction to SEO, trusted by millions.

          • SEO Learning Center

            Broaden your knowledge with SEO resources for all skill levels.

          • On-Demand Webinars

            Learn modern SEO best practices from industry experts.

          • How-To Guides

            Step-by-step guides to search success from the authority on SEO.

          • Moz Academy

            Upskill and get certified with on-demand courses & certifications.

          • MozCon

            Save on Early Bird tickets and join us in London or New York City

          Unlock flexible pricing & new endpoints
          Moz API

          Unlock flexible pricing & new endpoints

          Find your plan
        • Blog
        • Why Moz
          • Digital Marketers

            Simplify SEO tasks to save time and grow your traffic.

          • Small Business Solutions

            Uncover insights to make smarter marketing decisions in less time.

          • Agency Solutions

            Earn & keep valuable clients with unparalleled data & insights.

          • Enterprise Solutions

            Gain a competitive edge in the ever-changing world of search.

          • The Moz Story

            Moz was the first & remains the most trusted SEO company.

          • New Releases

            Get the scoop on the latest and greatest from Moz.

          Surface actionable competitive intel
          New Feature

          Surface actionable competitive intel

          Learn More
        • Log in
          • Moz Pro
          • Moz Local
          • Moz Local Dashboard
          • Moz API
          • Moz API Dashboard
          • Moz Academy
        • Avatar
          • Moz Home
          • Notifications
          • Account & Billing
          • Manage Users
          • Community Profile
          • My Q&A
          • My Videos
          • Log Out

        The Moz Q&A Forum

        • Forum
        • Questions
        • My Q&A
        • Users
        • Ask the Community

        Welcome to the Q&A Forum

        Browse the forum for helpful insights and fresh discussions about all things SEO.

        1. Home
        2. SEO Tactics
        3. Intermediate & Advanced SEO
        4. Generating 404 Errors but the Pages Exist

        Moz Q&A is closed.

        After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

        Generating 404 Errors but the Pages Exist

        Intermediate & Advanced SEO
        6
        15
        9882
        Loading More Posts
        • Watching

          Notify me of new replies.
          Show question in unread.

        • Not Watching

          Do not notify me of new replies.
          Show question in unread if category is not ignored.

        • Ignoring

          Do not notify me of new replies.
          Do not show question in unread.

        • Oldest to Newest
        • Newest to Oldest
        • Most Votes
        Reply
        • Reply as question
        Locked
        This topic has been deleted. Only users with question management privileges can see it.
        • Found
          Found last edited by

          Hey

          I have recently come across an issue with several of a sites urls being seen as a 404 by bots such as Xenu, SEOMoz, Google Web Tools etc. The funny thing is, the pages exist and display fine.

          This happens on many of the pages which use the Modx CMS, but the index is fine. The wordpress blog in /blog/ all works fine.

          The only thing I can think of is that I have a conflict in the htaccess, but troubleshooting this is difficult, any tool I have found online seem useless.

          Have tried to rollback to previous versions but still does not work.

          Anyone had any experience of similar issues?

          Many thanks

          K.

          1 Reply Last reply Reply Quote 0
          • OptimaD
            OptimaD @RyanKent last edited by

            FYI, we finally found our error. The short URL turned out to be the same name as the folder (photo-gallery) so once this was changed, wordpress was able to access the correct path. A bit of custom javascript had to be amended as well, but that was limited to our custom code. Using your web-sniffer.net link we were able to test immediately and fix it fairly quickly. Thank you for your help!

            1 Reply Last reply Reply Quote 1
            • seo-wanna-bs
              seo-wanna-bs @RyanKent last edited by

              That's true Ryan I guess it is coding related really.

              Issues like this are a real pain in the ass. And most people don't even check WMT to realise the issues exist. TBH, I don't check as often as I should.

              1 Reply Last reply Reply Quote 1
              • RyanKent
                RyanKent @seo-wanna-bs last edited by

                I agree with you Paul.

                As you pointed out one possible cause is a CMS-related issue which I would refer to as "coding" meaning something in the code which was used to present the website. Perhaps there is a better way to phrase it but nothing comes to mind at the moment.

                Another possibility you mentioned is Litespeed which would be a server-side issue directly. Either way, it is a legitimate issue which should be addressed.

                seo-wanna-bs 1 Reply Last reply Reply Quote 0
                • seo-wanna-bs
                  seo-wanna-bs last edited by

                  FWIW, I don't think it's a coding issue. If it were coding, it would either show a 200OK or it would show a 404. It wouldn't sometimes serve a 404.

                  If you're using Litespeed, I'd guarantee that is the issue and if you're using Joomla, it's another prime culprit.

                  RyanKent 1 Reply Last reply Reply Quote 0
                  • seo-wanna-bs
                    seo-wanna-bs last edited by

                    Please keep in mind, that 404 error does not mean the page doesn't exist. It means your server, is sending a response code to indicate that it doesn't exist.

                    When I installed Litespeed on my server, this issue happened over and over again.

                    I believe Joomla for example, has some kind of security module that serves a 404 if a single IP requests a page too many times. I remember running SEOFrog on a friends Joomla site and tons of 404's were showing up.

                    1 Reply Last reply Reply Quote 0
                    • Found
                      Found @RyanKent last edited by

                      Dev team are looking into it, must be quite a complex htaccess issue. Will get to the bottom of it this week and post any findings.

                      1 Reply Last reply Reply Quote 0
                      • OptimaD
                        OptimaD @RyanKent last edited by

                        Thanks Ryan! I will get it looked at...Sue

                        1 Reply Last reply Reply Quote 0
                        • RyanKent
                          RyanKent @OptimaD last edited by

                          @DentalID, the same reply I offered to Guy applies for you as well. This is an SEO issue which does need to be fixed. Something on your end is causing the page to show with a 403 response code. You really need a programmer to get in there and determine the root cause of the issue. You could try asking your web host if you have managed hosting, but this level of assistance would normally be outside the support of managed hosting.

                          OptimaD Found 3 Replies Last reply Reply Quote 0
                          • RyanKent
                            RyanKent last edited by

                            Guy,

                            In looking at the page this appears to be a legitimate problem. Your server settings allow you to present a page with any header code you wish. You can 301 a page but still present the page with a 200 code if you want. Presently it appears the page is being presented fine but your server is offering a 404 header code.

                            I can't tell the actual source of the problem other then to say it appears to be on your end and should be fixed. I originally looked at the code with the MOZbar but then checked independently with another tool as well. http://web-sniffer.net/

                            All tools show a 404 header code for the page. This response code is generated by your web server.

                            1 Reply Last reply Reply Quote 2
                            • OptimaD
                              OptimaD last edited by

                              We are having a similar problem with this URL:  http://dentalimplantsportland.com/photo-gallery/  and also the following locations:

                              http://cosmeticdentistportland.net/photo-gallery/

                              http://dentalveneersportland.com/photo-gallery/

                              SEO Moz  and Google webmaster tools show it as a 403 error but the pages display fine. I am not able to tell if this is really a problem for SEO or if we should reconstruct this gallery system and would really love your input.

                              This is Wordpress with a Spry gallery...

                              Thanks so much!

                              RyanKent 1 Reply Last reply Reply Quote 0
                              • Found
                                Found @RyanKent last edited by

                                It is just a small affiliate site I am looking at - this page creates a 404.

                                http://www.insure-uk.com/post-office-car-insurance.html

                                Currently testing on some beta servers. Hopefully should fix soon as otherwise it will lose indexation.

                                1 Reply Last reply Reply Quote 0
                                • AlanMosley
                                  AlanMosley last edited by

                                  I also see this now and again, but next crawl they fix themselfs. i assume robots can not always reach page for a number of reasons

                                  1 Reply Last reply Reply Quote 0
                                  • RyanKent
                                    RyanKent last edited by

                                    Can you offer an example of a URL which is causing this problem?

                                    Found 1 Reply Last reply Reply Quote 0
                                    • Netboost
                                      Netboost last edited by

                                      I have had the same issues, I think it is often the bot's problem

                                      Just to be certain check your links are correct and manually test them. Also ensure your sitemap is up to date and that you are not blocking the crawlers with metarobots, robots.txt, or some weird stuff in htaccess.

                                      I have found that renaming pages or moving them will often cause 404 issues with crawlers

                                      1 Reply Last reply Reply Quote 0
                                      • 1 / 1
                                      • First post
                                        Last post

                                      Browse Questions

                                      Explore more categories

                                      • Moz Tools

                                        Chat with the community about the Moz tools.

                                      • SEO Tactics

                                        Discuss the SEO process with fellow marketers

                                      • Community

                                        Discuss industry events, jobs, and news!

                                      • Digital Marketing

                                        Chat about tactics outside of SEO

                                      • Research & Trends

                                        Dive into research and trends in the search industry.

                                      • Support

                                        Connect on product support and feature requests.

                                      • See all categories

                                      Related Questions

                                      • henrycabrown

                                        Moved company 'Help Center' from Zendesk to Intercom, got lots of 404 errors. What now?

                                        Howdy folks, excited to be part of the Moz community after lurking for years! I'm a few weeks into my new job (Digital Marketing at Rewind) and about 10 days ago the product team moved our Help Center from Zendesk to Intercom. Apparently the import went smoothly, but it's caused one problem I'm not really sure how to go about solving: https://help.rewind.io/hc/en-us/articles/***    is where all our articles used to sit https://help.rewind.io/***    is where all our articles now are So, for example, the following article has now moved as such: https://help.rewind.io/hc/en-us/articles/115001902152-Can-I-fast-forward-my-store-after-a-rewind- https://help.rewind.io/general-faqs-and-billing/frequently-asked-questions/can-i-fast-forward-my-store-after-a-rewind This has created a bunch of broken URLs in places like our Shopify/BigCommerce app listings, in our email drips, and in external resources etc. I've played whackamole cleaning many of these up, but these old URLs are still indexed by Google – we're up to 475 Crawl Errors in Search Console over the past week, all of which are 404s. I reached out to Intercom about this to see if they had something in place to help, but they just said my "best option is tracking down old links and setting up 301 redirects for those particular addressed". Browsing the Zendesk forms turned up some relevant-ish results, with the leading recommendation being to configure javascript redirects in the Zendesk document head (thread 1, thread 2, thread 3) of individual articles. I'm comfortable setting up 301 redirects on our website, but I'm in a bit over my head in trying to determine how I could do this with content that's hosted externally and sitting on a subdomain. I have access to our Zendesk admin, so I can go in and edit stuff there, but don't have experience with javascript redirects and have read that they might not be great for such a large scale redirection. Hopefully this is enough context for someone to provide guidance on how you think I should go about fixing things (or if there's even anything for me to do) but please let me know if there's more info I can provide. Thanks!

                                        Intermediate & Advanced SEO | | henrycabrown
                                        1
                                      • KJ-Rodgers

                                        Location Pages On Website vs Landing pages

                                        We have been having a terrible time in the local search results for 20 + locations. I have Places set up and all, but we decided to create location pages on our sites for each location - brief description and content optimized for our main service. The path would be something like .com/location/example. One option that has came up in question is to create landing pages / "mini websites" that would probably be location-example.url.com. I believe that the latter option, mini sites for each location, would be a bad idea as those kinds of tactics were once spammy in the past. What are are your thoughts and and resources so I can convince my team on the best practice.

                                        Intermediate & Advanced SEO | | KJ-Rodgers
                                        0
                                      • friendoffood

                                        What if page exists for desktop but not mobile?

                                        I have a domain (no subdomains) that serves up different dynamic content for mobile/desktop pages--each having the exact same page url, kind of a semi responsive design, and will be using "Vary: User-Agent" to give Google a heads up on this setup. However, some of the pages are only valid for mobile or only valid for desktop.  In the case of when a page is valid only for mobile (call it mysite.com/mobile-page-only ), Google Webmaster Tools is giving me a soft 404 error under Desktop, saying that the page does not exist, Apparently it is doing that because my program is actually redirecting the user/crawler to the home page.  It appears from the info about soft 404 errors that Google is saying since it "doesn't exist" I should give the user a 404 page--which I can make it customized and give the user an option to go to the home page, or choose links from a menu, etc.. My concern is that if I tell the desktop bot that mysite.com/mobile-page-only basically is a 404 error (ie doesn't exist), that it could mess up the mobile bot indexing for that page--since it definitely DOES exist for mobile users.. Does anyone here know for sure that Google will index a page for mobile that is a 404 not found for desktop and vice versa?  Obviously it is important to not remove something from an index in which it belongs, so whether Google is careful to differential the two is a very important issue.  Has anybody here dealt with this or seen anything from Google that addresses it?  Might one be better off leaving it as a soft 404 error? EDIT: also, what about Bing and Yahoo?  Can we assume they will handle it the same way? EDIT: closely related question--in a case like mine does Google need a separate sitemap for the valid mobile pages and valid desktop pages even though most links will be in both?  I can't tell from reading several q&a on this. Thanks, Ted

                                        Intermediate & Advanced SEO | | friendoffood
                                        0
                                      • Philip-DiPatrizio

                                        Putting "noindex" on a page that's in an iframe... what will that mean for the parent page?

                                        If I've got a page that is being called in an iframe, on my homepage, and I don't want that called page to be indexed.... so I put a noindex tag on the called page (but not on the homepage) what might that mean for the homepage?  Nothing?  Will Google, Bing, Yahoo, or anyone else, potentially see that as a noindex tag on my homepage?

                                        Intermediate & Advanced SEO | | Philip-DiPatrizio
                                        0
                                      • lcourse

                                        Should I noindex the site search page? It is generating 4% of my organic traffic.

                                        I read about some recommendations to noindex the URL of the site search.
                                        Checked in analytics that site search URL generated about 4% of my total organic search traffic (<2% of sales). My reasoning is that site search may generate duplicated content issues and may prevent the more relevant product or category pages from showing up instead. Would you noindex this page or not? Any thoughts?

                                        Intermediate & Advanced SEO | | lcourse
                                        0
                                      • PottyScotty

                                        Does an H1 have to be at the top of a page?

                                        Because H1 "may" carry some  weight with Google does it have to be placed at the top of the page?  Can I place it towards the bottom of the page instead in normal body size?  My goal is to keep the main keywords in the H1 but create a much friendlier title for the customer to read at the top of the page.

                                        Intermediate & Advanced SEO | | PottyScotty
                                        0
                                      • Prime85

                                        Stuck on Page 2 - What Would You Do?!?

                                        My site is : http://goo.gl/JgK1e My main keyword is : Plastic Bins i have been going back and forth between page 1 and 2 for this keyword and i was wondering if any of you could provide any guidance as to why i can't get on the top of page 1, and stay there... My site has been around for a while, we believe we have a great user experience, all unique, fresh content, and the lowest prices... I must be missing out on something major if I cannot get a steady page 1 ranking... Any thoughts? Thanks in advance...

                                        Intermediate & Advanced SEO | | Prime85
                                        0
                                      • pulseseo

                                        Does Google crawl the pages which are generated via the site's search box queries?

                                        For example, if I search for an 'x' item in a site's search box and if the site displays a list of results based on the query, would that page be crawled? I am asking this question because this would be a URL that is non existent on the site and hence am confused as to whether Google bots would be able to find it.

                                        Intermediate & Advanced SEO | | pulseseo
                                        0

                                      Get started with Moz Pro!

                                      Unlock the power of advanced SEO tools and data-driven insights.

                                      Start my free trial
                                      Products
                                      • Moz Pro
                                      • Moz Local
                                      • Moz API
                                      • Moz Data
                                      • STAT
                                      • Product Updates
                                      Moz Solutions
                                      • SMB Solutions
                                      • Agency Solutions
                                      • Enterprise Solutions
                                      • Digital Marketers
                                      Free SEO Tools
                                      • Domain Authority Checker
                                      • Link Explorer
                                      • Keyword Explorer
                                      • Competitive Research
                                      • Brand Authority Checker
                                      • Local Citation Checker
                                      • MozBar Extension
                                      • MozCast
                                      Resources
                                      • Blog
                                      • SEO Learning Center
                                      • Help Hub
                                      • Beginner's Guide to SEO
                                      • How-to Guides
                                      • Moz Academy
                                      • API Docs
                                      About Moz
                                      • About
                                      • Team
                                      • Careers
                                      • Contact
                                      Why Moz
                                      • Case Studies
                                      • Testimonials
                                      Get Involved
                                      • Become an Affiliate
                                      • MozCon
                                      • Webinars
                                      • Practical Marketer Series
                                      • MozPod
                                      Connect with us

                                      Contact the Help team

                                      Join our newsletter
                                      Moz logo
                                      © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                                      • Accessibility
                                      • Terms of Use
                                      • Privacy

                                      Looks like your connection to Moz was lost, please wait while we try to reconnect.