Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to pass a URL to Wget

Tags:

grep

pipe

wget

If I have a document with many links and I want to download especially one picture with the name www.website.de/picture/example_2015-06-15.jpeg, how can I write a command that downloads me automatically exactly this one I extracted out of my document?

My idea would be this, but I'll get a failure message like "wget: URL is missing":

grep -E 'www.website.de/picture/example_2015-06-15.jpeg' document | wget
like image 435
fragant Avatar asked Jun 15 '15 21:06

fragant


People also ask

How do I download a website using wget?

To download a single HTML page (or a handful of them, all specified on the command-line or in a -i URL input file) and its (or their) requisites, simply leave off -r and -l: wget -p http://<site>/1.html Note that Wget will behave as if -r had been specified, but only that single page and its requisites will be ...

How do I transfer files using wget?

Download file from destination server. From receiving server, navigate to the directory you'd like to download it to. Type wget https://domain.com/backup.tar.gz – to download external file to your current directory. (Of course, your file path and filename may vary.) Wait for it to finish transferring.

What is wget spider?

The wget tool is essentially a spider that scrapes / leeches web pages but some web hosts may block these spiders with the robots. txt files. Also, wget will not follow links on web pages that use the rel=nofollow attribute. You can however force wget to ignore the robots.

What does wget command do?

Wget is a computer tool created by the GNU Project. You can use it to retrieve content and files from various web servers. The name is a combination of World Wide Web and the word get. It supports downloads via FTP, SFTP, HTTP, and HTTPS.


2 Answers

Use xargs:

grep etc... | xargs wget

It takes its stdin (grep's output), and passes that text as command line arguments to whatever application you tell it to.

For example,

echo hello | xargs echo 'from xargs '

produces:

from xargs  hello
like image 70
Marc B Avatar answered Oct 05 '22 22:10

Marc B


Using back ticks would be the easiest way of doing it:

wget `grep -E 'www.website.de/picture/example_2015-06-15.jpeg' document`
like image 36
r3mainer Avatar answered Oct 05 '22 21:10

r3mainer