Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How fast does Google take to crawl new page, and can we influence Google's crawler?

Tags:

search

seo

I want to submit my site to Google. How much time does it take to crawl a new post on the website?

Also, is there a way to feed this post to Google crawler as soon as a post is created?

like image 898
Usama Khalil Avatar asked Jan 26 '12 03:01

Usama Khalil


1 Answers

Google has three modes of entering a website into its results - discover, crawl, index.

In order to 'discover' your site, it must be made aware of it's existence - normally through back-links. If you're site is brand new you can use the submit URL form - but this isn't really a trusted method. You're better off signing up for a Google Webmaster Tools account and submitting your site. An additional step is to submit an XML sitemap of your site. If you are publishing to your site in a blogging/posting way - you can always consider PubSubHubbub.

From there on, crawl frequency is normally based on site popularity (as measured by ye olde PageRank). Depth of crawl (crawl-budget) is also determined by PR.

like image 130
Mike Hudson Avatar answered Oct 03 '22 03:10

Mike Hudson