Menu
I'm wondering if there is a way to check ahead of time the size of a file I might download via wget? I know that using the --spider option tells me if a file exists or not, but I'm interested in finding the size of that file as well.
845
If so, use wget -c as suggested by @dtmland. Finally, wget does have an option to limit file size but it is not set by default. One possibility is that your sysadmin has set a limit by making wget an alias to something like wget --max-filesize N . To check if wget is an alias run alias wget .
The wget tool is essentially a spider that scrapes / leeches web pages but some web hosts may block these spiders with the robots. txt files. Also, wget will not follow links on web pages that use the rel=nofollow attribute. You can however force wget to ignore the robots.
Downloading a file In order to download a file using Wget, type wget followed by the URL of the file that you wish to download. Wget will download the file in the given URL and save it in the current directory.
curl --head URL Look for "Content-Length:" in the output.
And thanks to Henning Makholm's comment:
wget --spider URL and look for "Length:" in the output.
21
Hmm.. for me --spider does display the size:
$ wget --spider http://henning.makholm.net/ Spider mode enabled. Check if remote file exists. --2011-08-08 19:39:48-- http://henning.makholm.net/ Resolving henning.makholm.net (henning.makholm.net)... 85.81.19.235 Connecting to henning.makholm.net (henning.makholm.net)|85.81.19.235|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 9535 (9.3K) [text/html] <------------------------- Remote file exists and could contain further links, but recursion is disabled -- not retrieving. $ (But beware that not all web servers will inform clients of the length of the data except by closing the connection when it's all been sent.)
If you're concerned about wget changing the format it reports the length in, you might use wget --spider --server-response and look for a Content-Length header in the output.
190
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
