How do I curl multiple resources in one command?

Question

Say I am trying to download a set of 50 lecture notes efficiently. These notes are inside the prof subdirectory of a university website. The 45th lecture note is inside the lect45 subdirectory as a pdf entitled lect45.pdf. I get my first pdf as follows:

curl -O http://www.university.edu/~prof/lect1/lect1.pdf

How do I get all my 50 notes efficiently using cURL and bash? I'm trying to do this from the command line, not through a Python / Ruby / Perl script. I know something like the below will generate a lot of 404s:

curl -O http://www.university.edu/~prof/lect{1..50}/lect{1..50}.pdf

so what will work better? I would prefer an elegant one-liner over a loop.

Answer 1

Do it in several processes:

for i in {1..50}
do
    curl -O http://www.university.edu/~prof/lect$i/lect$i.pdf &
done

or as a one-liner (just a different formatting):

for i in {1..50}; do curl -O http://www.university.edu/~prof/lect$i/lect$i.pdf & done

The & makes all processes run in parallel.

Don't be scared by the output; the shell tells you that 50 processes have been started, that's a lot of spam. Later it will tell you for each of these that they terminated. A lot of output again.

You probably don't want to run all 50 in parallel ;-)

EDIT:

Your example using {1..50} twice makes a matrix of the numbers. See for example echo {1..3}/{1..3} to see what I mean. And I guess that this way you create a lot of 404s.