Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to detect bots for open graph with user-agent?

So I am doing some server-side logic for my single page app and I need to detect various open graph crawlers so that I can serve open graph data instead of the standard index page.

As far as I can tell, user agents with the string facebot twitterbot and facebookexternalhit should be included. But has somebody created a regex (or npm package) for this?

I think there are other places on the web that might want to use open-graph data, and I'd like to be able to serve the open-graph data (I dunno, like pinterest or something) to them.

like image 276
RoccoB Avatar asked Oct 31 '17 00:10

RoccoB


1 Answers

You can use https://github.com/monperrus/crawler-user-agents.

The exact list is here https://github.com/monperrus/crawler-user-agents/blob/master/crawler-user-agents.json and it is updated quite often.

like image 161
rkouye Avatar answered Nov 12 '22 15:11

rkouye