Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Exporting Scrapy requests in Curl format

enter image description here

I use Postman for API testing , which has the ability to import curl commands.

https://www.getpostman.com/docs/importing_curl

Can scrapy export in this format (or any other format) ? This may help in debugging requests.

like image 659
user1592380 Avatar asked Jun 08 '16 16:06

user1592380


2 Answers

Not exactly answering the question, but I hope may help as well.

I remember going to browser developer tools, using the "Copy as cURL" functionality, then pasting it to the textbox here and getting a Python+Requests code, which should be easier to transform into Scrapy.Request() call.

like image 105
alecxe Avatar answered Oct 09 '22 16:10

alecxe


You could use minreq to ouptut a Scrapy Request from curl. It even only outputs the necessary data to send to repeat the same request successfully.

like image 33
eLRuLL Avatar answered Oct 09 '22 16:10

eLRuLL