Curl error while unencoding

Question

Hi i am sending a simple get request with Curl, it works perfectly for all sites i have had to use it on except this last one. When i get the response i get a weird unencoding error. This is what i get with verbose.

* About to connect() to www.shufersal.co.il port 80 (#0)
*   Trying 212.199.57.42...
* connected
* Connected to www.shufersal.co.il (212.199.57.42) port 80 (#0)
> GET /Pages/Catalog.aspx HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.5 (KHTML, like Gecko) C
hrome/19.0.1084.52 Safari/536.5
Host: www.shufersal.co.il
Accept: */*
Accept-Encoding: gzip,deflate,sdch
Referer:
Cookie: SMC=; TSeb84b7=6c5fc932ce03e1ab1ca864bae7ec71c8d659417300b51c794fce0178

< HTTP/1.1 200 OK
< Cache-Control: private
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=utf-8
< Content-Encoding: gzip
< Vary: Accept-Encoding
< SPRequestGuid: 217857fb-493b-4542-a5b4-575023c5767a
< X-SharePointHealthScore: 0
* Added cookie ASP.NET_SessionId="5lwlq2ify1glmxirhndgdm45" for domain www.shufe
rsal.co.il, path /, expire 0
< Set-Cookie: ASP.NET_SessionId=5lwlq2ify1glmxirhndgdm45; path=/; HttpOnly
< X-AspNet-Version: 2.0.50727
< X-UA-Compatible: IE=8
< Date: Tue, 05 Jun 2012 12:58:24 GMT
* Added cookie BIGipServerB2C_Commerce="1533875210.20480.0000" for domain www.sh
ufersal.co.il, path /, expire 0
< Set-Cookie: BIGipServerB2C_Commerce=1533875210.20480.0000; path=/
* Replaced cookie TSeb84b7="6c5fc932ce03e1ab1ca864bae7ec71c8d659417300b51c794fce
0178" for domain www.shufersal.co.il, path /, expire 1338902005
< Set-Cookie: TSeb84b7=6c5fc932ce03e1ab1ca864bae7ec71c8d659417300b51c794fce0178;
 Max-Age=900; Path=/
<
* Error while processing content unencoding: invalid block type
* Failed writing data
* Closing connection #0
PHP Fatal error:  Call to a member function find() on a non-object in C:\wamp\ww
w\adwords\grocery_scraper.php on line 86
PHP Stack trace:
PHP   1. {main}() C:\wamp\www\adwords\grocery_scraper.php:0

Fatal error: Call to a member function find() on a non-object in C:\wamp\www\adw
ords\grocery_scraper.php on line 86

Call Stack:
    0.0011     367424   1. {main}() C:\wamp\www\adwords\grocery_scraper.php:0


c:\wamp\www\adwords>php grocery_scraper.php
* About to connect() to www.shufersal.co.il port 80 (#0)
*   Trying 212.143.91.230...
* connected
* Connected to www.shufersal.co.il (212.143.91.230) port 80 (#0)
> GET /Pages/Catalog.aspx HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.5 (KHTML, like Gecko) C
hrome/19.0.1084.52 Safari/536.5
Host: www.shufersal.co.il
Accept: */*
Accept-Encoding: gzip,deflate,sdch
Referer:

< HTTP/1.1 200 OK
< Cache-Control: private
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=utf-8
< Content-Encoding: gzip
< Vary: Accept-Encoding
< SPRequestGuid: 8adff954-20d7-4d0a-9cab-193bc6cd6e72
< X-SharePointHealthScore: 0
< X-AspNet-Version: 2.0.50727
* Added cookie ASP.NET_SessionId="ji00xpqq5z1ufa454t1c0c45" for domain www.shufe
rsal.co.il, path /, expire 0
< Set-Cookie: ASP.NET_SessionId=ji00xpqq5z1ufa454t1c0c45; path=/; HttpOnly
* Added cookie SMC="" for domain www.shufersal.co.il, path /, expire 1370437143
< Set-Cookie: SMC=; expires=Wed, 05-Jun-2013 12:59:03 GMT; path=/; HttpOnly
< X-UA-Compatible: IE=8
< Date: Tue, 05 Jun 2012 12:59:02 GMT
* Added cookie BIGipServerB2C_Commerce="1533875210.20480.0000" for domain www.sh
ufersal.co.il, path /, expire 0
< Set-Cookie: BIGipServerB2C_Commerce=1533875210.20480.0000; path=/
* Added cookie TSeb84b7="c054d61ec8ab44efb7ca8f8680023cf1889b86d8962f57f14fce029
7" for domain www.shufersal.co.il, path /, expire 1338902044
< Set-Cookie: TSeb84b7=c054d61ec8ab44efb7ca8f8680023cf1889b86d8962f57f14fce0297;
 Max-Age=900; Path=/
<
* Error while processing content unencoding: invalid stored block lengths
* Failed writing data
* Closing connection #0

Any ideas whats going wrong? or how to fix it? Maybe there is a parameter to skip unencoding or something and do it on php.

Answer 1

The error message Error while processing content unencoding comes from libcurl and means that it detected a problem in the gzip stream which prevented it from uncompressing the full contents.

This indicates a problem either in the site or in zlib/curl, but I would rather suspect the site...

As a work-around, trying asking for the page without compression.