Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
What countries does Google crawl from? Is it only US or do they crawl from Europe and Asia, etc.?
-
Where does Google crawl the web from? Is it in the US only, or do they do it from a European base too? The reason for asking is for GeoIP redirection. For example, if a website is using GeoIP redirection to redirect all US traffic to a .com site and all EU traffic to a .co.uk site, will Google ever see the .co.uk site?
-
Hi Keith,
In my experience Google mainly crawls from the US.
You're quite right, GeoIP redirection can cause major issues with indexation - as if you're redirecting everything from the US to the .com googlebot can't see the .co.uk site.
As such, I'm not a fan. Rather than implementing a hard redirect I prefer Amazon's approach. If you visit the amazon.com site from a UK IP you get a javascript overlay that invites you to visit the .co.uk version of the site instead - they let the user decide which site to view rather than actually redirecting them.
This is a nice solution as it ensures that the search bots can crawl both versions of the site, and rankings aren't endangered.
I hope this helps,
Hannah
-
Keith, I am having the same issue and I agree with you. The fact that Google has data centers in Europe does not necessarily mean the algos are indexing from there. I also want to set Europe and US GeoIP redirection. It would be great to get Mozers opinions on this. Hopefully this post gets freshly reviewed
-
Interesting question - I'd quite like to know what happens here too.
Matt Cutts recently posted a video on cloaking (http://youtu.be/QHtnfOgp65Q) saying that as long as you don't do anything 'special' for Googlebot you're OK, but presumably if you are redirecting IP's based on location and you don't want to prevent Googlebot form accessing your site then effectively you have to do something 'special' for Googlebot (e.g. you're doing one thing for everyone else and a different thing for Googlebot).
-
Hi,
The data center locations is interesting, but it isn't what I was looking for. I need to know whether Google crawls the web from any IP other than US IPs.
To clear up the second question, let me be more specific:
Let's say Google is crawling a .co.uk site from a US IP address. The site is using GEO IP redirection to redirect all US traffic to the .com site. Therefore, when Google attempts to crawl the .co.uk from the US IP address site it will be redirected to the .com site, never seeing the .co.uk site. Can anyone confirm that this is what happens?
-
I found an article from 2008 that shows Google data centre locations: http://bit.ly/mONhf9
Your other question is a bit confusing. Why Google wouldn't see the UK site?
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Has any one seen negative SEO effects from using Google Translate API
We have a site currently in development that is using the Google Translate API and I am having a massive issue getting screaming frog to crawl and all of our non-native English speaking employees have read through the translated copy in their native language and the general consensus is it reads at a 5th grade level at best. My questions to the community is, has anyone implemented this API on a site and has it a) helped with gaining traffic from other languages/countires and b) has it hurt there site from an SEO standpoint.
International SEO | | VERBInteractive0 -
How can I restrict the domains country by country?
Hello, I have Two Domains one is xyz.co.uk and other is xyz.com Now, my main target for .com is United States, and I don't want to open that .com domain in any other country especially India. The same with the .co.uk, I dont want to open .co.uk in other countries. I did it with some developer help but it gave me redirected error in Google Webmaster. Can anyone please guide me how I can do this the proper way ? And Other issues is, how can I implement ,if any user in United States open xyz.co.uk than he should redirect to the .com version. Thank you.
International SEO | | AmitTulsiyani0 -
Can you target the same site with multiple country HREFlang entries?
Hi, I have a question regarding the country targeting aspect of HREFLANG. Can the same site be targeted with multiple country HREFlang entries? Example: A global company has an English South African site (geotargeted in webmaster tools to South Africa), with a hreflang entry targeted to "en-za", to signify English language and South Africa as the country. Could you add entries to the same site to target other English speaking South African countries? Entries would look something like this: (cd = Congo, a completely random example) etc... Since you can only geo-target a site to one country in WMT would this be a viable option? Thanks in advance for any help! Vince
International SEO | | SimonByrneIFS0 -
E-Commerce - Country Domains versus 1 Domain?
Hi, Just wanted to get some feedback and opinions re the idea of segmenting our ecommerce site languages under various domains, like .jp for japanese, .it for italian etc... I do understand the geolocation benefits that this could bring us, but on the flipside, it would mean that we would need to grow our domain authority, link buiding per country domain, which is quite a bit of work. Has anyone ever considered or implemented this and any thoughts? Thanks!
International SEO | | bjs20100 -
URL Structure - Homepage, Country and State Pages
Hello, I am creating a website (or websites if best format) that will have state-specific boating license courses for every state in the US, Canada and Australia. I would like the content to be available on the website in English, French and Spanish. I want to be the global leader in providing boat test courses. For the (1) homepage, (2) country pages, and (3) state pages, what is best SEO format I should use for:
International SEO | | Monologix
(a) URL structure
(b) "href lang" code
(c) rel canonical code
(d) will meta content with non-English pages need to also be in the non-English language of that page? Also, what server company do you recommend I host my website with? I am a non-programmer and learning SEO, so any and all help will be greatly appreciated! Thank you very much in advance!!!0 -
Website Target in Europe
Hi, I am planning a site to target in Europe and I expect to translate my site into ten different languages namely English, French, Spanish, Italian, German, Russian, Greek, Portuguese, Dutch and Swedish. I am doing some study of this case in targeting different countries for SEO, most of the advise are the following: a. Build 10 different websites and target different geographical location in Google Webmaster b. Get 10 different country specific domains for 10 different websites I would like to hear any suggestion if there is anything better than this ? I had all the materials and translation ready but building 10 different websites or getting 10 different domains are very time consuming and costly. I would be appreciated if any one had any advise for me to make the website more management friendly. Thank you. Tom
International SEO | | Stevejobs20110 -
Country specific domains pointing to a .com site
Hello, I am new to seo so please be easy if this happens to be a "silly" question. My company has a .com site. We are expanding into global markets, focusing on specific countries right now. General question: Would I be penalized for duplicate content if I purchased country-specific domains and pointed them to the .com site? Thanks, Jim
International SEO | | jimmer0 -
Country specific landing pages
I have a client who wants to put a re-direct on his landing pages based on the visitors IP address. The landing page will be a sub domain relevant to the country their IP is located in. I am a little concerned this will effect the SEO. Appreciate any advice. Dylan 🙂
International SEO | | gomyseo0