Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to properly handle a gzipped page when using curl?

Tags:

bash

curl

gzip

People also ask

Does curl support gzip?

In most distributions, PHP Curl is compiled with gzip , deflat , and br (Brotli). However, it is also possible to add support for zstd by compiling libcurl with zstd support, and recompiling PHP with the new libcurl header files.

How do I know if API response is Gzipped?

You can tell using Developer Tools (F12). Go to the Network tab, select the file you want to examine and then look at the Headers tab on the right. If you are gzipped, then you will see that in the Content-Encoding.

What does curl compressed do?

compressed to a generated curl command so that curl decompresses the response body.

How do I curl a file?

Grab file with curl run: $ curl https://your-domain/file.pdf. Get file using ftp or sftp protocol: $ curl ftp://ftp-your-domain-name/file.tar.gz. You can set the output file name while downloading file with the curl, execute: $ curl -o file. pdf https://your-domain-name/long-file-name.pdf.


curl will automatically decompress the response if you set the --compressed flag:

curl --compressed "http://example.com"

--compressed (HTTP) Request a compressed response using one of the algorithms libcurl supports, and save the uncompressed document. If this option is used and the server sends an unsupported encoding, curl will report an error.

gzip is most likely supported, but you can check this by running curl -V and looking for libz somewhere in the "Features" line:

$ curl -V
...
Protocols: ...
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz 

Note that it's really the website in question that is at fault here. If curl did not pass an Accept-Encoding: gzip request header, the server should not have sent a compressed response.