Wget with input-file and output-document

Question

I have a list of URLs which I would like to feed into wget using --input-file.

However I can't work out how to control the --output-document value at the same time, which is simple if you issue the commands one by one. I would like to save each document as the MD5 of its URL.

 cat url-list.txt | xargs -P 4 wget

And xargs is there because I also want to make use of the max-procs features for parallel downloads.

Answer 1

Don't use cat. You can have xargs read from a file. From the man page:

       --arg-file=file
       -a file
              Read items from file instead of standard input.  If you use this
              option, stdin remains unchanged when commands are  run.   Other‐
              wise, stdin is redirected from /dev/null.

Answer 2

how about using a loop?

while read -r line
do
   md5=$(echo "$line"|md5sum)
   wget ... $line ... --output-document $md5 ......
done < url-list.txt