Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Where does Googlebot crawl from?

Tags:

gis

googlebot

I have been looking high and low for the locations from which Googlebot and its brothers and sisters crawl from other than "CA" and am having no luck finding locations specific to crawling. I have found a map of data canters from 2008 (http://bit.ly/mONhf9), I have found Googles own self professed brilliance in their data centers, complete with lovely photos (http://www.google.com/about/datacenters/).

I do know from experience, at least 3 years ago, that the ole' bot would also come from VA... but past that... Nothing.

Can anyone help with this riddle? I'm gathering Google doesnt want ME to know.

like image 904
Jason Avatar asked Oct 29 '25 16:10

Jason


2 Answers

Your mileage will vary, but in the case of the most popular website I have access to, the crawls emanate from:

  • Illinois (46.35%)
  • Arizona (15.68%)
  • Wisconsin (14.95%)
  • Virginia (11.85%)
  • California (11.17%)

The web servers that have been subject to these crawls are located on Amazon, in Virginia.

Here it is on a map: http://www.chrisbrunner.com/2014/04/01/where-does-googlebot-crawl-from/ (I couldn't post it directly, due to this being my first post.)

Again, others will have different experiences, but this is what we observed over the course of March 2014.

like image 196
user2113156 Avatar answered Oct 31 '25 11:10

user2113156


I've collected information about that topic and I've concluded that GoogleBot crawls from 6 different locations. I haven't been able to precisely put them on the map but seems that 4 of them are in different parts of the States and one of them in Europe.

You have all the details in this post: http://blog.domenech.org/2013/09/where-is-googlebot-crawler-located.html

like image 41
Juan Domenech Avatar answered Oct 31 '25 12:10

Juan Domenech



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!