Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Downloading all the files in a directory with cURL

I am using cURL to try to download all files in a certain directory.

here's what my list of files looks like:

enter image description here

I have tried to do in bash script: iiumlabs.[].csv.pgp and iiumlabs* and I guess curl is not big on wildcards.

curl -u login:pass ftp.myftpsite.com/iiumlabs* -O

question: how do i download this directory of files using cURL?

like image 834
Alex Gordon Avatar asked Aug 02 '12 18:08

Alex Gordon


People also ask

How do I download all files with curl?

Grab file with curl run: $ curl https://your-domain/file.pdf. Get file using ftp or sftp protocol: $ curl ftp://ftp-your-domain-name/file.tar.gz. You can set the output file name while downloading file with the curl, execute: $ curl -o file. pdf https://your-domain-name/long-file-name.pdf.

How do I download multiple files with curl?

Download multiple files simultaneously To download multiple files at the same time, use –O followed by the URL to the file that you wish to download. The above command will download both files. The above Curl command will download all the URLs specified in the files. txt file.

Can curl download files?

curl lets you quickly download files from a remote system. curl supports many different protocols and can also make more complex web requests, including interacting with remote APIs to send and receive data.


1 Answers

If you're not bound to curl, you might want to use wget in recursive mode but restricting it to one level of recursion, try the following;

wget --no-verbose --no-parent --recursive --level=1\
--no-directories --user=login --password=pass ftp://ftp.myftpsite.com/
  • --no-parent : Do not ever ascend to the parent directory when retrieving recursively.
  • --level=depth : Specify recursion maximum depth level depth. The default maximum depth is five layers.
  • --no-directories : Do not create a hierarchy of directories when retrieving recursively.
like image 67
kkeller Avatar answered Oct 20 '22 02:10

kkeller