Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Googlebot being redirected but not users?
-
Hi,
We seem to have a slightly odd issue.
We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc.
One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc.
On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc.
Thanks.
-
Can anyone offer any insight on this? The issue shows no sign of improving, and our organic is tanking.
If this really is an issue on Google's side, and it appears based on our technical set up, how can I force Google to take note and reindex the pages?
Here are some more examples, from across a number of categories..
Category A:
Page that has dropped out of index: https://goo.gl/KEQ8Yh
Cached version of that page: https://goo.gl/DPzFWMPage that has dropped out of index: https://goo.gl/5KiQ4s Cached version of that page: https://goo.gl/myRWNg Category B: Page that has dropped out of index: https://goo.gl/pr3YQs
Cached version of that page: https://goo.gl/8SEYi5 Page that has dropped out of index: https://goo.gl/LqzDrg
Cached version of that page: https://goo.gl/iwPs45 Category
Page that has dropped out of index: https://goo.gl/YBZS7c Cached version of that page:https://goo.gl/n33QzG Page that has dropped out of index: https://goo.gl/Ht4gfO
Cached version of that page: https://goo.gl/u81vbA -
Weirdly, at the time this seemed to correct itself overnight without any fixes our end. However 6 months on and it has happened again... Only this time, all location pages have dropped out of index, but latest cache versions all redirect to the Kansas page.
Have checked log files for these relevant URLs, and all show 200 (and being crawled regularly). When you fetch them in SC the show as complete, and they are all still indexed in Bing.
Very confused!
-
Hi Nicola,
Have you checked your canonical tags to be sure that they are pointing to the correct URL? Sometimes with storefront pages people just copy/paste the same HTML, swapping out the contact information. But maybe they forgot to update the canonical tags?
Keep us updated!
-
Thanks for advice Chris. Have checked, and can't see any issues with htaccess or robots seem. A very strange one indeed!
-
Hi Anthony,
Desktop returns 200, whilst mobile returns a 302 to the mDOT site.
Thanks.
-
Hi Nicola,
Sounds like a very strange one indeed. I can't say I've ever heard of Google giving a redirect when other 3rd party crawlers give you a 200.
My only suggestion to begin the troubleshooting would be to have a look at your htaccess and maybe Robots to see if there are any directives in there addressing Google specifically.
Anthony's suggestion is also a great idea. At least you'll see what Google is picking up on those pages right now rather what they saw last time it was cached.
-
You mention these pages are returning 200 in SEO tools, but would you please share what are these URLs are returning when you Fetch as Googlebot in Google Search Console? Test it in both desktop and mobile versions.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Htaccess - Redirecting TAG or Category pages
Hello Fellow Moz's, We have an issue redirecting some /TAG and /Category pages to inner pages. As an example we use: RedirectMatch 301 /category/Sample-Category(.*) https://OurDomain.com.au/New-Page//$1 That works well. The issue is we have other categories and tags that are named similar to /Sample-Category As an example, if we try to redirect /Sample-Category-1 to /New-Page-1 - it will not work, and redirects to /New-Page I assume this is because /Sample-Category is already being redirected, so anything after /Sample-Category like -1 or -2 or -3 etc, will not be recognized. Anyone know of a workaround?
Intermediate & Advanced SEO | | Jes-Extender-Australia0 -
301 redirect hops from non-https and www
It's best practice to minimize the amount of 301 redirect hops. Ideally only one redirect hop. It's also best practice to 301 redirect (or at least canonical) your non-https and/or your non-www (or www) to the canonical protocol/subdomain. The simplest (and possibly the most common) way to implement canonical protocol/subdomain redirects is through a load balancer or before your app processes the request. Both of which will just blanket 301 to the canonical domain/protocol regardless if the path exists or not In which case, you could have: Two hops. i.e. hop #1 http://example.com/foo to https://example.com/foo, hop #2 https://example.com/foo to https://example.com/bar 301 to a 404. Let's say https://example.com/dog never existed, but somebody for whatever reason linked to it (maybe a typo). If I request https://www.example.com/dog, the load balancer would 301 to a 404 page. Either scenario above should be fairly rare. However, you can't control how people link to you. Should I care about either above scenario? I could have my app attempt to check if the page exists before forwarding, but that code could be complicated.
Intermediate & Advanced SEO | | dsbud0 -
Does removal of internal redirects(301) help in SEO
I am planning to completely remove 301 redirects manually by replacing such links with actual live pages/links. So there will be no redirects internally in the website. Will this boost our SEO efforts? Auto redirects will be there for incoming links to non-existing pages. Thanks, Satish
Intermediate & Advanced SEO | | vtmoz0 -
Several 301 Redirects to Same Page
Hi, I have 3 Pages we won't use anymore in our website. Let's call them url A, url B and url C. To keep their SEO strength on our domain, I've though about redirecting all of them to url D. For what I understand, when 301 redirecting, about 85-90% of the link SEO juice is passed. Then, if I redirect 3 URLs to the same page... does url D receive all the link SEO juices for URLs added up? (approximately)
Intermediate & Advanced SEO | | viatrading1
e.g. future url D juice = 100% current url D juice + 85% url A juice + 85% url B juice + 85% url C juice Is this the best practice, or is there a better way? Cheers,0 -
Layered navigation and hiding nav from user agent
I am trying to deal with the duplicate content issues presented by Magento's layered navigation feature (aka faceted navigation). I installed Amasty's Improved Navigation extension (https://amasty.com/improved-layered-navigation.html) and it offers the option to hide the layered navigation from specific user agents (ie googlebot, bingbot, etc). This seems like cloaking to me and I hesitate to try it, unless hiding faceted navigation from specific user agents is known to be acceptable to Google (white hat practice). Does anyone know if this the case?
Intermediate & Advanced SEO | | Kyle_M0 -
How to handle individual page redirects on Wix?
I switched from one domain to another because I wanted a domain that had our company name so it was more brand-y. However, the old domain had better DA/PA. Originally I set up a global 301 from the old to the new, but now I'm finding that I actually need to set up individual 301's from each URL of the old site, or at least from each page. However, I am using Wix so it looks like I can't always do URL-URL 301's, although I can redirect any URL to a page on the new website. The problem is that, in some cases, the content on the new site is different (or, for example, I can only link a particular blog post on the old site back to the new site's blog's main page). How closely do URLS/pages need to resemble each other for link juice to be transferred? Also, should I try to set up all these redirects manually or bite the bullet and go back to using the old domain? The problem is that I did a lot of beginner SEO junk for the new domain, like submitting to a few higher-quality directories, and getting our website on various industry resource sites, etc. I'd need to re-do this entirely if I go back to the old page. What do you think?
Intermediate & Advanced SEO | | BohmKalish1230 -
Redirecting to a new domain... a second time
Hi all, I help run a website for a history-themed podcast and we just moved it to its second domain in 7 years. We've had very good SEO up until last week, and I'm wondering if I screwed up the way I redirected the domains. It's like this: Originally the site was hosted at "first.com", and it acquired inbound links. However, we then started to host the site on blogger, so we... Redirected the site to "second.blogspot.com". (Thus, 1 --> 2) It stayed here for about 7 years and got lots of traffic. Two weeks ago we moved it off of blogger and into Wordpress, so we 301 redirected everything to... third.com. (Thus, 1 --> 2 --> 3) The redirects worked, and when we Google individual posts, we are now seeing them in Google's index at the new URL. My question: What about the 1--> 2 redirect? There are still lots of links pointing to "first.com". Last week I went into my GoDaddy settings and changed the first redirect, so that first.com now points to third.com. (Thus 1 --> 3, and 2-->3) I was correct in doing that, right? The drop in Google traffic I've seen this past week makes me think that maybe I screwed something up. Should we have kept 1 --> 2 --> 3? (Again, now we have 1-->3 and 2-->3) Thanks for any insights on this! Tom
Intermediate & Advanced SEO | | TomNYC1 -
301 Redirect of subdomain?
Fellow Mozzers, I'm having a hard time wrapping my brain around a redirect issue and thought it was worth posing the question to the Moz community. I did a search first but couldn't find the exact answer I was looking for. How does a 301 redirect work when you redirect a sub domain example.homepage.com to www.homepage.com but you keep the sub directories of example.homepage.com/page-1 active and are trying to rank them? I'm dealing with a current project where this is happening and this doesn't make sense to me, to redirect the subdomain if you're also trying to rank/create search traffic for pages, sub directories on example.homepage.com. This also get's into the debate of if a sub domain site is viewed as it's own website and therefore has to rank itself. If this is true, it seems like we're kind of killing the authority of the site by redirecting it. Additionally, www.homepage.com has a much stronger link profile than example.homepage.com I hope this makes sense. Any thoughts are appreciated. Thanks for your time.
Intermediate & Advanced SEO | | SMG-Texas0