Menu
I need to get the HTML source of pinnaclesports.com. The problem is it detects whether cookies and JS are enabled and if not, it just returns some page saying
This site requires JavaScript and Cookies to be enabled. Please change your browser settings or upgrade your browser.
Is there any way how to spoof JS support when using cURL?
EDIT: I can use a headless browser that runs either as a Perl/Ruby module or is written in PHP
645
Wget and curl don't handle javascript. Your question makes no sense. The web uses HTTP as a universal protocol for client-server web transactions. The server and the client are independent objects/ entities that can exchange data because of HTTP.
To make a GET request using Curl, run the curl command followed by the target URL. Curl automatically selects the HTTP GET request method unless you use the -X, --request, or -d command-line option.
From Chrome and Edge On the line of the specific resource you are interested in, you right-click with the mouse and you select "Copy as cURL" and it will generate a command line for you in your clipboard. Paste that in a shell to get a curl command line that makes the transfer.
'cURL' is a command-line tool that lets you transmit HTTP requests and receive responses from the command line or a shell script.
Other sugestion is set the user agent, this solution works for me on parser of the Google Groups:
curl -L -v "https://groups.google.com/d/forum/<GROUP-NAME>" -A "Mozilla/5.0 (compatible; MSIE 7.01; Windows NT 5.0)"
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With