Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

What is the alternative of Selenium for executing Js in Python-Scrapy?

I want to execute JavaScript's function in Python-Scrapy, before, i used Selenium, but Selenium is too much slow for scraping the big sites.

I would like to know what's the best alternative for selenium for executing js in Scrapy?

like image 218
parik Avatar asked Dec 05 '22 16:12

parik


1 Answers

Splash - A javascript rendering service

Splash is a javascript rendering service with an HTTP API. It's a lightweight browser with an HTTP API, implemented in Python using Twisted and QT.

It's fast, lightweight and state-less which makes it easy to distribute.

Bonus points for it being developed by the same guys who are developing Scrapy.

like image 165
Granitosaurus Avatar answered May 17 '23 23:05

Granitosaurus