How to urlencode data into a URL, with bash or curl

Question

How can a string be urlencoded and embedded into the URL? Please note that I am not trying to GET or POST data, so the -G and --data and --data-urlencode options of curl don't seem to do the job.

For example, if you used

curl -G http://example.com/foo --data-urlencode "bar=spaced data"

that would be functionally equivalent to

curl http://example.com/foo?bar=spaced%20data"

which is not desired.

I have a string foo/bar which must be urlencoded foo%2fbar and embedded into the URL.

curl http://example.com/api/projects/foo%2fbar/events

One hypothetical solution (if I could find something like this) would be to preprocess the data in bash, if there exists some kind of urlencode function.

DATA=foo/bar
ENCODED=`urlencode $DATA`
curl http://example.com/api/projects/${ENCODED}/events

Another hypothetical solution (if I could find something like this) would be some switch in curl, similar to this:

curl http://example.com/api/projects/{0}/events --string-urlencode "0=foo/bar"

The specific reason I'm looking for an answer to this question is the Gitlab API. For example, gitlab get single project NAMESPACE/PROJECT_NAME is URL-encoded, eg. /api/v3/projects/diaspora%2Fdiaspora (where / is represented by %2F). Further to this, you can request individual properties in the project, so you end up with a URL such as http://example.com/projects/diaspora%2Fdiaspora/events

Although this question is gitlab-specific, I imagine it's generally applicable to REST API's in general, and I'm surprised I can't find a pre-existing answer on stackoverflow or internet search.

Answer 1

The urlencode function you propose is easy enough to implement:

urlencode() {
  python -c 'import urllib, sys; print urllib.quote(sys.argv[1], sys.argv[2])' \
    "$1" "$urlencode_safe"
}

...used as:

data=foo/bar
encoded=$(urlencode "$data")
curl "http://example.com/api/projects/${encoded}/events"

If you want to have some characters which are passed through literally -- in many use cases, this is desired for /s -- instead use:

encoded=$(urlencode_safe='/' urlencode "$data")