Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

When does Googlebot execute javascript?

I have a few single page web apps on multiple domains that heavily rely on javascript/ajax to fetch and show content. Based on logs and search results I can tell that googlebot runs javascript on some of the domains but not on others. On some it indexes everything thats only available with js on others it doesn't even seem to run js at all.

Can anybody tell me how googlebot decides what js to run and if I can to anything to get it to run js on my other domains?

PS: I know that normally I should use something like serverside rendering for this, but I'm not at all depended on search results and rankings, so its not really worth the effort. I'm just curious how googlebot decides whether it should run js or not and if there's anything easy I can do to change that on my other domains.

like image 691
Moto Avatar asked Mar 23 '14 08:03

Moto


People also ask

Does Googlebot execute JavaScript?

Googlebot queues all pages for rendering, unless a robots meta tag or header tells Google not to index the page. The page may stay on this queue for a few seconds, but it can take longer than that. Once Google's resources allow, a headless Chromium renders the page and executes the JavaScript.

What is Googlebot and how does it work?

Googlebot is the general name for two different types of crawlers: a desktop crawler that simulates a user on desktop, and a mobile crawler that simulates a user on a mobile device. Your website will probably be crawled by both Googlebot Desktop and Googlebot Smartphone.

Can bots execute JavaScript?

Using the Open action of the JavaScript package, bot builders can establish a session that can be called using the Run JavaScript action.

Can search engines crawl JavaScript?

While Google are generally able to crawl and index most JavaScript content, they still advise using server-side rendering or pre-rendering, rather than relying on a client-side approach as its 'difficult to process JavaScript, and not all search engine crawlers are able to process it successfully or immediately'.


2 Answers

You can learn more about how Google render ajax based website and a list of best practice directly from Google developer website here:

https://webmasters.googleblog.com/2014/10/updating-our-technical-webmaster.html https://developers.google.com/webmasters/ajax-crawling/

Regarding your specific problem as first thing, I suggest you to analyse each domain using Google Webmaster tool with functionality "Fetch as Google" and go trough every technical aspects mentioned in Google guide.

https://support.google.com/webmasters/answer/158587?hl=en

like image 136
GibboK Avatar answered Nov 15 '22 14:11

GibboK


I think Google Updated Research on the Subject http://searchengineland.com/tested-googlebot-crawls-javascript-heres-learned-220157

like image 23
OBender Avatar answered Nov 15 '22 16:11

OBender