Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Passing a URL with brackets to curl

Tags:

url

curl

If I try to pass a URL to curl that contains brackets, it fails with an error:

$ curl 'http://www.google.com/?TEST[]=1' curl: (3) [globbing] illegal character in range specification at pos 29 

However, if I escape both brackets, it appears to work:

$ curl 'http://www.google.com/?TEST\[\]=1' 

Interestingly, I use a backslash to escape only the first bracket it fails silently with error code 20497:

$ curl 'http://www.google.com/?TEST\[]=1' $ echo $! 20497 

My question is how to fix this for general cases? Is there an argument that will escape URLs automatically, or a description of the characters that need to be escaped before passing to curl?

like image 305
chaimp Avatar asked Nov 30 '11 22:11

chaimp


People also ask

Can URL contain curly braces?

Curly brackets are unsafe in URLs. cURL (unlike Google Chrome) tries to do you a favor and automatically encodes the URL. In other words, it transforms { to %7B and } to &7D . To prevent that behavior, you can pass the query string parameters using -d instead.

What is curl Globbing?

curl offers "globbing" as a way to specify many URLs like that easily. The globbing uses the reserved symbols [] and {} for this, symbols that normally cannot be part of a legal URL (except for numerical IPv6 addresses but curl handles them fine anyway). If the globbing gets in your way, disable it with -g, --globoff .

Are square brackets allowed in URLs?

According to the URL specification, the square brackets are not valid URL characters.


1 Answers

Never mind, I found it in the docs:

-g/--globoff               This  option  switches  off  the "URL globbing parser". When you set this option, you can               specify URLs that contain the letters {}[] without having them being interpreted by  curl               itself.  Note  that  these  letters  are not normal legal URL contents but they should be               encoded according to the URI standard. 
like image 64
chaimp Avatar answered Sep 19 '22 21:09

chaimp