I've written a shell script that has a function like this:
function getpage {
echo $1
curl -O "http://www.example.com/$1" -b cookie.txt -s
}
The problem is if the website times out then that page will be skipped, I need it to re-try if it times out (I will also be putting in a 60 second timeout).
How do I do this?
How to set a timeout for a Curl command? To set a timeout for a Curl command, you can use the --connect-timeout parameter to set the maximum time in seconds that you allow Curl to connect to the server, or the --max-time (or -m) parameter for the total time in seconds that you authorize the whole operation.
The best method of achieving this is by using the --connect-timeout option. You can specify the timeout in seconds (e.g., 5), in milliseconds (e.g. 0.001), or as a combination of seconds and milliseconds (e.g., 4.20), and curl will use that time as the maximum time for a response until a connection is dropped.
You can use --retry <num>
for forced retries. An alternate way is to add -w http_code
to see what the return code is...if it's not 200 then try again.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With