Menu
I have .sh script that, among other things, wgets currency exchange rates from Google as follows:
printf 'Bash: Going to get exchange rates'
echo
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=aud" | sed '/res/!d;s/<[^>]*>//g' > exrates
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=jpy" | sed '/res/!d;s/<[^>]*>//g' >> exrates
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=hkd" | sed '/res/!d;s/<[^>]*>//g' >> exrates
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=nzd" | sed '/res/!d;s/<[^>]*>//g' >> exrates
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=eur" | sed '/res/!d;s/<[^>]*>//g' >> exrates
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=gbp" | sed '/res/!d;s/<[^>]*>//g' >> exrates
mv /home/stan/perl/2014/scripts/exrates /home/stan/perl/2014/exrates/exrates
printf 'Bash: Got exchange rates'
echo
Occasionally script hangs here, however. I don't mind not updating these rates every time it runs, if it hangs I'd like to skip this step altogether, but how?
What should I put in "if" statement to check if wget can fetch data promptly or will take forever? A little more verbosity in wget execution wouldn't hurt either.
Btw, I don't know why wget hangs. Browser opens those pages okay, and same commands run from terminal line by line work, too.
637
wget responds with a return code of 8 when it receives a response code that corresponds to an error from the server, and thus 16 is the total after 2 errors. Stopping when failures only occur twice in a row can be done by resetting the threshold whenever wget succeeds, i.e. when the return code is 0 Show activity on this post.
If you use brace expansion with wget, you can fetch sequentially-numbered images with ease: It fetches the first 10 files numbered 90.jpg to 99.jpg just fine, but 100.jpg and onward return a 404: File not found error (I only have 100 images stored on the server).
it seems wget isn't able to connect to your ip on port 80. if the server does not respond to ping, it is probably down or unreachable for your source net. test with traceroute tool to see where the connection fails. Show activity on this post. Fix for slow DNS resolution in PHP scripts using CURL library.
wget responds with a return code of 8 when it receives a response code that corresponds to an error from the server, and thus 16 is the total after 2 errors. Stopping when failures only occur twice in a row can be done by resetting the threshold whenever wget succeeds, i.e. when the return code is 0
I assume that it hangs because you have a number of HTTP requests being sent to a single host in a script. The host in question doesn't like that too much and it starts to block requests from your IP address.
A simple workaround would be to put a sleep in between the requests. You could also make use of a function:
getExchangeRates() {
wget -qO- "http://www.google.com/finance/converter?a=1&from=usd&to=$1" | sed '/res/!d;s/<[^>]*>//g' >> exrates
sleep 10 # Adding a 10 second sleep
}
and invoke it by passing a parameter to the function:
getExchangeRates aud
The function could also be invoked in a loop for various currencies:
for currency in aud jpy hkd nzd eur gpb; do
getExchangeRates $currency
done
use timeout in wget statement only
wget --timeout 10 <URL>
timeout is in seconds and put some sleep in between two wgets
34
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
