Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
How to safely exclude search result pages from Google's index?
-
Hello everyone,
I'm wondering what's the best way to prevent/block search result pages from being indexed by Google. The way search works on my site is that search form generates URLs like:
/index.php?blah-blah-search-results-blahI wanted to block everything of that sort, but how do I do it without blocking /index.php ?
Thanks in advance and have a great day everyone!
-
Hi Louise,
If you can ID the parameters, you can also look at blocking these in Webmaster Tools. This page explains more. As with any blocking of URLs, of course, proceed with caution.
-
I agree that can be effective. The reason I suggested the robots.txt is because Louise mentioned "blocking and preventing" as an objective. Robots.txt are particularly useful in the example where results from a search bar or something of that nature is involved. A NOINDEX, FOLLOW will not prevent bots from getting tired and dizzy, whereas the robots.txt can "block and prevent" bots from crawling certain parameters.
With all of that said, I think it is important to understand whether you need the bots to crawl and not index (in which case Spencer's answer is correct), or if you need to prevent bots from crawling the parameters altogether.
Hope that is more clear
-
I'm not sure that robots.txt is effective when url parameters are involved.
I would just add a meta robots tag to the head section of the search results template:
-
If you are able to identify a url parameter, you may excluded them using robots.txt. Here is a great resource on Robots.txt - http://a-moz.groupbuyseo.org/learn/seo/robotstxt
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Website excluded from indexing, google-selected canonical: N/A
The google search console revealed to me that none of my pages was indexed, all pages are listed in the 'excluded' section "duplicate, google chose different canonical than user".
Reporting & Analytics | | Fibigrus
But in the URL-inspection tab it shows me google-selected canonical: N/A Indexing and crawling is both allowed. Don't know how to get my pages to be indexed correctly. (by the way, they do NOT exist in other languages, so that can't be a reason why google might think they are a duplicate. There's definitively no other version of those pages available)0 -
Page Speed or Site Speed which one does Google considered a ranking signal
I've read many threads online which proves that website speed is a ranking factor. There's a friend whose website scores 44 (slow metric score) on Google Pagespeed Insights. Despite that his website is slow, he outranks me on Google search results. It confuses me that I optimized my website for speed, but my competitor's slow site outperforms me. On Six9ja.com, I did amazing work by getting my target score which is 100 (fast metric score) on Google Pagespeed Insights. Coming to my Google search console tool, they have shown that some of my pages have average scores, while some have slow scores. Google search console tool proves me wrong that none of my pages are fast. Then where did the fast metrics went? Could it be because I added three Adsense Javascript code to all my blog posts? If so, that means that Adsense code is slowing website speed performance despite having an async tag. I tested my blog post speed and I understand that my page speed reduced by 48 due to the 3 Adsense javascript codes added to it. I got 62 (Average metric score). Now, my site speed is=100, then my page speed=62 Does this mean that Google considers page speed rather than site speed as a ranking factor? Screenshots: https://imgur.com/a/YSxSwOG **Regarding: **https://six9ja.com/
Reporting & Analytics | | Kingsmart1 -
Moz reports way fewer backlinks than google search?
My site is only 11 months old but has steadily (if not slowly) been gaining backlinks. My question, is why Moz shows me at 303 backlinks and Google search console is showing at 1,237? I am more than a little suspicious that this could highlight the reason Moz shows such an unfavorable DA ranking for our site at a DA12. Other competitors that rank for similar keywords to mine are DA 42, DA 65, DA 73, etc. If the largest ranking factor is links, and they have mine reported incorrectly - is this the issue with DA as it relates to sites like mine? Any answer from someone who has experienced similar, or has a definitive answer is more than welcome to chime in! Thanks, Kevin
Reporting & Analytics | | kvncrll0 -
How to Diagnose "Crawled - Currently Not Indexed" in Google Search Console
The new Google Search Console gives a ton of information about which pages were excluded and why, but one that I'm struggling with is "crawled - currently not indexed". I have some clients that have fallen into this pit and I've identified one reason why it's occurring on some of them - they have multiple websites covering the same information (local businesses) - but others I'm completely flummoxed. Does anyone have any experience figuring this one out?
Reporting & Analytics | | brettmandoes2 -
Tasks for Google Analytics training
Hi Mozzers, I'm delivering some Google Analytics (Fundamentals level) training, and trying to make it was fun and as interesting as possible... which is quite a challenge when it comes to GA. I was just wondering if you're aware of training tasks, or interactions, I could bring into this kind of training session? The group are particularly interested in user journeys and the effectiveness of content. Thanks!
Reporting & Analytics | | A_Q0 -
Whats 'Other' in Google Analytics (in Acquisition)
When i look in GA under Channels (under Acquisition) 'other' is listed What is 'other' ? I have been told its other unidentified channels as they did not allow 3rd party cookies or surfers were in anonymous/private mode. Other is usually organic traffic that couldn't be identified for the aformentioned reasons. This data is encrypted and available but it violates Google guidelines as they are not allowed to pass personal info//data to third parties so it is automatically filtered. But they are not 'Not Provided' (since that still shows under organic) but is usually/mainly some form of organic visits. Hence Seo can take credit for much of that traffic, is this correct ? Many Thanks
Reporting & Analytics | | Dan-Lawrence0 -
Google as referring domain
Hi all, a colleague asked a question, which I could not answer (never even noticed this "problem") 😞 When we are logged into our GA account and go the referring domains section, we find Google. I always thought that these visitors came via Google Image Search, but not all of them do. Most of them come via "/imgres", but some come via "/" (always thought that "/" was the homepage?), "/url" and "//" Maybe I am just stupid, but honestly I could not explain what these strings mean... or how these visitors landed on our site... Can you help me???
Reporting & Analytics | | accessKellyOCG0 -
Google Analytics and DNS change
Our new alumni application is going be tested at domain uva.imodules.com . We are going to collect traffic data with a Google analytics account number UA-884652-XX. So going to uva.imodules.com/myPage.html would send its data to Google Analytics with that account number. Then when it is ready for production we are going to just change the domain name of the application and switch the DNS over to dardencommunity.darden.virginia.edu . So going to dardencommunity.darden.virginia.edu /myPage.html would send its data to Google Analtics with that SAME account number. Aside from having the testing domain data in the same profile are there any other issues/problems we may run into?
Reporting & Analytics | | Darden0