Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
What countries does Google crawl from? Is it only US or do they crawl from Europe and Asia, etc.?
-
Where does Google crawl the web from? Is it in the US only, or do they do it from a European base too? The reason for asking is for GeoIP redirection. For example, if a website is using GeoIP redirection to redirect all US traffic to a .com site and all EU traffic to a .co.uk site, will Google ever see the .co.uk site?
-
Hi Keith,
In my experience Google mainly crawls from the US.
You're quite right, GeoIP redirection can cause major issues with indexation - as if you're redirecting everything from the US to the .com googlebot can't see the .co.uk site.
As such, I'm not a fan. Rather than implementing a hard redirect I prefer Amazon's approach. If you visit the amazon.com site from a UK IP you get a javascript overlay that invites you to visit the .co.uk version of the site instead - they let the user decide which site to view rather than actually redirecting them.
This is a nice solution as it ensures that the search bots can crawl both versions of the site, and rankings aren't endangered.
I hope this helps,
Hannah
-
Keith, I am having the same issue and I agree with you. The fact that Google has data centers in Europe does not necessarily mean the algos are indexing from there. I also want to set Europe and US GeoIP redirection. It would be great to get Mozers opinions on this. Hopefully this post gets freshly reviewed
-
Interesting question - I'd quite like to know what happens here too.
Matt Cutts recently posted a video on cloaking (http://youtu.be/QHtnfOgp65Q) saying that as long as you don't do anything 'special' for Googlebot you're OK, but presumably if you are redirecting IP's based on location and you don't want to prevent Googlebot form accessing your site then effectively you have to do something 'special' for Googlebot (e.g. you're doing one thing for everyone else and a different thing for Googlebot).
-
Hi,
The data center locations is interesting, but it isn't what I was looking for. I need to know whether Google crawls the web from any IP other than US IPs.
To clear up the second question, let me be more specific:
Let's say Google is crawling a .co.uk site from a US IP address. The site is using GEO IP redirection to redirect all US traffic to the .com site. Therefore, when Google attempts to crawl the .co.uk from the US IP address site it will be redirected to the .com site, never seeing the .co.uk site. Can anyone confirm that this is what happens?
-
I found an article from 2008 that shows Google data centre locations: http://bit.ly/mONhf9
Your other question is a bit confusing. Why Google wouldn't see the UK site?
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Which Google does a .ie website get shown in?
Hi all, I am working on a .ie website and I was under the understanding that if you have a regional domain, like .ie you will limited to being shown in a search engine like google.ie When I go to International Targeting in Google SEarch Console it says the site is associated with: Ireland Am I limiting my ability to rank well in worldwide Google searches with this domain and if so, how can I counter this? Many thanks.
International SEO | | Bee1590 -
How to avoid duplication across multiple country domains
Here's the scenario: I have a client currently running one Shopify site (AU) They want to launch three more country domains (US, UK and EU) They want each to be a standalone site, primarily so the customers can purchase in their local currency, which is not possible from a single Shopify site The inventory is all from the same source The product desscriptions will all be the same as well Question: How do we avoid content duplication (ie. how will canonical tags work in this scenario)?
International SEO | | muzzmoz0 -
Targeting Countries in the Middle East
Hi guys, I have a client based in the Middle East using a generic top level domain (.com), and they want to target multiple countries in the GCC (UAE, Saudi Arabia, Kuwait, Qatar etc). I’m thinking that using the hreflang tag would be the best solution here, however the pages will mostly have the exact same content. There will only be slight changes on some pages in terms of using localised title tags [client service] followed by [targeted country], h1's and meta descriptions. Is this the correct approach? And if so should this be implemented side wide or can it be implemented on selected pages only? The site will be in English only.
International SEO | | Jbeetle0 -
URL Structure - Homepage, Country and State Pages
Hello, I am creating a website (or websites if best format) that will have state-specific boating license courses for every state in the US, Canada and Australia. I would like the content to be available on the website in English, French and Spanish. I want to be the global leader in providing boat test courses. For the (1) homepage, (2) country pages, and (3) state pages, what is best SEO format I should use for:
International SEO | | Monologix
(a) URL structure
(b) "href lang" code
(c) rel canonical code
(d) will meta content with non-English pages need to also be in the non-English language of that page? Also, what server company do you recommend I host my website with? I am a non-programmer and learning SEO, so any and all help will be greatly appreciated! Thank you very much in advance!!!0 -
Geographic Target set up in Google webmaster tool
Hi, When i launched my web site 3 months ago ( I'm am very new to SEO) I have set up the geographical target section in Google webmaster tool for US. Now, I'm thinking to change it to some other geo target to see if i can get more traffic. However, recently few of my prompted keywords got really well in Google US ranking. Here are my Questions: if i will change the geo settings in webmaster tool will effect the ranking i already managed to achieve in US? In the list of all the countries in Google webmaster tools what does is mean "unlisted"? Can i select more than one country to target and if I can how? Thanks!! Raviv
International SEO | | Indiatravelz0 -
SEO Company in Asia
Hi All. I have a client looking to expand their industrial services to southeast Asia (Vietnam and Indonesia specifically right now). Does anyone know of an SEO/Online Marketing firm local to that region that may be able to help them network with businesses and industries there? I've gone through the SEOmoz member database and reached out to a couple people with agencies in that area but never heard back from them. I personally thought a local firm would be more beneficial to the client but I'm also open to suggestions on ways that we might be able to help them market their services online from the US. Thanks so much! Megan
International SEO | | ILM_Marketing0 -
Non US site pages indexed in US Google search
Hi, We are having a global site wide issue with non US site pages being indexed by Google and served up in US search results. Conversley, we have US en pages showing in the Japan Google search results. We currently us IP detect to direct users to the correct regional site but it isn't effective if the users are entering through an incorrect regional page. At the top of each or our pages we have a drop down menu to allow users to manually select their preferred region. Is it possible that Google Bot is crawling these links and indexing these other regional pages as US and not detecting it due to our URL structure? Below are examples of two of our URLs for reference - one from Canada, the other from the US /ca/en/prod4130078/2500058/catalog50008/ /us/en/prod4130078/2500058/catalog20038/ If that is, in fact, what is happening, would setting the links within the drop down to 'no follow' address the problem? Thank you. Angie
International SEO | | Corel0 -
How can I see what my web site looks like from a different country?
I've tried a few proxy tools to try to see how my site looks from other global locations, but haven't found one that works very well yet -- or a list of reliable proxies around the world. I need to do this to test various geo-targetted ads and other optimizations. Can anyone make a recommendation? Thanks!
International SEO | | Dennis-529610