Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to download a page with wget but ignore 404 error messages if the page does not exist?

Tags:

wget

Is there any way to have wget ignore HTTP error response codes when downloading a URL or spidering a webpage?

like image 754
bsuire Avatar asked Aug 19 '15 12:08

bsuire


People also ask

How do I download a website using wget?

Let's start with something simple. Copy the URL for a file you'd like to download in your browser. Now head back to the Terminal and type wget followed by the pasted URL. The file will download, and you'll see progress in realtime as it does.

What is the most common message on error 404?

A 404 error is a standard HTTP error message code that means the website you were trying to reach couldn't be found on the server. It's a client-side error, meaning either the webpage was removed or moved and the URL wasn't changed accordingly, or the person just typed in the URL incorrectly.


1 Answers

Assuming I understood what you mean by "ignoring errors", you can try the --content-on-error argument. According to wget manual, it will force wget to skip status error codes\.

like image 128
koohyar Avatar answered Oct 11 '22 17:10

koohyar