Menu
I trying to set header in wget. When I run the following command in terminal it works wget -d --header="User-Agent: Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11"' http://website.com -O index
but once i put the same in variables and try to run a bash script it's not working.
what i've tried
header='-d --header="User-Agent: Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11"'
wget "$header" http://google.com -O index
error
wget: invalid option -- ' '
wget: invalid option -- '-'
wget: invalid option -- '-'
Usage: wget [OPTION]... [URL]...
534
wget allows you to send an HTTP request with custom HTTP headers. To supply custom HTTP headers, use "--header" option. You can use "--header" option as many time as you want in a single run. If you would like to permanently set the default HTTP request header you want to use with wget, you can use ~/.
wget with –output-document The wget command outputs the document content in a separate file by default. However, we can use the –output-document (-O) option to redirect the content to a file of our choice. As a particular use case, if we use – as the file, it directs the content to stdout. Great!
The wget command is very popular amongst Linux users as it supports a lot of different protocols like HTTP, HTTPS, FTP, etc. As a command-line tool, the wget is called from the bash and when we try to run the wget command we may get the “bash: /usr/bin/wget: No such file or directory” error.
The wget package is pre-installed on most Linux distributions today. To check whether the Wget package is installed on your system, open up your console, type wget , and press enter. If you have wget installed, the system will print wget: missing URL .
You have to use double quotes when using the variable. Otherwise it will be expanded into multiple words. On the other hand, there is no need to quote the variable value twice. The following should work:
header='--header=User-Agent: Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11'
wget "$header" http://website.com -O index
Edit: If you want to use variables for several arguments, use arrays:
args=(-d '--header=User-Agent: Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11')
wget "${args[@]}" http://website.com -O index
Instead of this
wget $header http://website.com -O index
try this,
wget "$header" http://website.com -O index
The spaces in the header text is breaking up when you are assigning it the header variable. To do away with the problem you must surround a variable with quotes "".
After comments: Try this -
someheader="-d --header='User-Agent: Mozilla/5.0 (Windows NT 6.0) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11'"
wget "$someheader" http://website.com -O index
The name header seems conflicting with your --header. Or, may be the quotes, instead of copying type them and retry. Still, Weird!
42
COMMENT: 'nosid's trick about the bash array worked for me. In my case the relevant code is:
WGET_OPTS="-r -N -nd -np -nH --timeout=120 --tries=3"
WGET_OPTS_ARRAY=(${WGET_OPTS// / })
wget "${WGET_OPTS_ARRAY[@]}" -A "$FILE_PAT" -P "$TO_DIR" "$FROM_URL"
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
