Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

download files using bash script using wget

Tags:

bash

loops

wget

cat

I've been trying to create a simple script that will take a list of file to be downloaded from a .txt file, then using a loop it will read the .txt what files needs to be downloaded with the help of the other separated .txt file where in the address of the files where it will be downloaded. But my problem is I don't know how to do this. I've tried many times but I always failed.

file.txt
1.jpg
2.jpg
3.jpg
4.mp3
5.mp4

=====================================

url.txt
url = https://google.com.ph/

=====================================

download.sh
#!/bin/sh
url=$(awk -F = '{print $2}' url.txt)
for i in $(cat file.txt);
do 
wget $url
done

Your help is greatly appreciated.

like image 967
user3534255 Avatar asked May 05 '14 02:05

user3534255


People also ask

How do I download files using wget?

In order to download a file using Wget, type wget followed by the URL of the file that you wish to download. Wget will download the file in the given URL and save it in the current directory.

How do I download a file using shell script?

Before writing a shell script, we will see how to download a file directly using commands, then we will extend it to a script. The FTP commands for downloading files are “get” and “mget” which are used for downloading single or multiple files respectively. These commands should be entered inside an FTP prompt.


1 Answers

Other than the obvious issue that R Sahu pointed out in his answer, you can avoid:

  • Using awk to parse your url.txt file.
  • Using for $(cat file.txt) to iterate through file.txt file.

Here is what you can do:

#!/bin/bash

# Create an array files that contains list of filenames
files=($(< file.txt))

# Read through the url.txt file and execute wget command for every filename
while IFS='=| ' read -r param uri; do 
    for file in "${files[@]}"; do 
        wget "${uri}${file}"
    done
done < url.txt
like image 193
jaypal singh Avatar answered Sep 27 '22 18:09

jaypal singh