Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Wget output document and headers to STDOUT

Tags:

http

wget

Try the following, no extra headers

wget -qO- www.google.com

Note the trailing -. This is part of the normal command argument for -O to cat out to a file, but since we don't use > to direct to a file, it goes out to the shell. You can use -qO- or -qO -.


wget -S -O - http://google.com works as expected for me, but with a caveat: the headers are considered debugging information and as such they are sent to the standard error rather than the standard output. If you are redirecting the standard output to a file or another process, you will only get the document contents.

You can try redirecting the standard error to the standard output as a possible solution. For example, in bash:

$ wget -q -S -O - 2>&1 | grep ...

or

$ wget -q -S -O - 1>wget.txt 2>&1

The -q option suppresses the progress bar and some other annoyingly chatty parts of the wget output.


It works here:

    $ wget -S -O - http://google.com
HTTP request sent, awaiting response... 
  HTTP/1.1 301 Moved Permanently
  Location: http://www.google.com/
  Content-Type: text/html; charset=UTF-8
  Date: Sat, 25 Aug 2012 10:15:38 GMT
  Expires: Mon, 24 Sep 2012 10:15:38 GMT
  Cache-Control: public, max-age=2592000
  Server: gws
  Content-Length: 219
  X-XSS-Protection: 1; mode=block
  X-Frame-Options: SAMEORIGIN
Location: http://www.google.com/ [following]
--2012-08-25 12:20:29--  http://www.google.com/
Resolving www.google.com (www.google.com)... 173.194.69.99, 173.194.69.104, 173.194.69.106, ...

  ...skipped a few more redirections ...

    [<=>                                                                                                                                     ] 0           --.-K/s              
<!doctype html><html itemscope="itemscope" itemtype="http://schema.org/WebPage"><head><meta itemprop="image" content="/images/google_favicon_128.png"><ti 

... skipped ...

perhaps you need to update your wget (~$ wget --version GNU Wget 1.14 built on linux-gnu.)


This worked for me for printing response with header:

wget --server-response http://www.example.com/

This will not work:

wget -q -S -O - google.com 1>wget.txt 2>&1

since redirects are evaluated right to left, this sends html to wget.txt and the header to STDOUT:

wget -q -S -O - google.com 2>&1 1>wget.txt