I use Postman for API testing , which has the ability to import curl commands.
https://www.getpostman.com/docs/importing_curl
Can scrapy export in this format (or any other format) ? This may help in debugging requests.
Not exactly answering the question, but I hope may help as well.
I remember going to browser developer tools, using the "Copy as cURL" functionality, then pasting it to the textbox here and getting a Python+Requests code, which should be easier to transform into Scrapy.Request()
call.
You could use minreq
to ouptut a Scrapy Request from curl. It even only outputs the necessary data to send to repeat the same request successfully.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With