Making sense of Apache HttpClient wire log from response

Question

My application is using Apache HTTPClient 4.3.5 that sends HTTP request and gets response.

I am trying to figure out what response the app received.

Below is the log snippet -

[Jan 04 2015 05:38:14.109 AM] DEBUG wire:72 - http-outgoing-2 >> "POST /app HTTP/1.1[\r][\n]"
[Jan 04 2015 05:38:14.109 AM] DEBUG wire:72 - http-outgoing-2 >> "User-Agent: Mozilla/5.0[\r][\n]"
[Jan 04 2015 05:38:14.109 AM] DEBUG wire:72 - http-outgoing-2 >> "Content-Length: 47[\r][\n]"
[Jan 04 2015 05:38:14.109 AM] DEBUG wire:72 - http-outgoing-2 >> "Content-Type: application/x-www-form-urlencoded[\r][\n]"
[Jan 04 2015 05:38:14.109 AM] DEBUG wire:72 - http-outgoing-2 >> "Host: test.com[\r][\n]"
[Jan 04 2015 05:38:14.109 AM] DEBUG wire:72 - http-outgoing-2 >> "Connection: Keep-Alive[\r][\n]"
[Jan 04 2015 05:38:14.110 AM] DEBUG wire:72 - http-outgoing-2 >> "Accept-Encoding: gzip,deflate[\r][\n]"
[Jan 04 2015 05:38:14.110 AM] DEBUG wire:72 - http-outgoing-2 >> "[\r][\n]"
[Jan 04 2015 05:38:14.489 AM] DEBUG wire:72 - http-outgoing-2 << "HTTP/1.1 200 OK[\r][\n]"
[Jan 04 2015 05:38:14.489 AM] DEBUG wire:72 - http-outgoing-2 << "Server: nginx[\r][\n]"
[Jan 04 2015 05:38:14.490 AM] DEBUG wire:72 - http-outgoing-2 << "Date: Sun, 04 Jan 2015 10:37:31 GMT[\r][\n]"
[Jan 04 2015 05:38:14.490 AM] DEBUG wire:72 - http-outgoing-2 << "Content-Type: text/html;charset=UTF-8[\r][\n]"
[Jan 04 2015 05:38:14.490 AM] DEBUG wire:72 - http-outgoing-2 << "Transfer-Encoding: chunked[\r][\n]"
[Jan 04 2015 05:38:14.490 AM] DEBUG wire:72 - http-outgoing-2 << "Connection: keep-alive[\r][\n]"
[Jan 04 2015 05:38:14.490 AM] DEBUG wire:72 - http-outgoing-2 << "Vary: Accept-Encoding[\r][\n]"
[Jan 04 2015 05:38:14.490 AM] DEBUG wire:72 - http-outgoing-2 << "Pragma: no-cache[\r][\n]"
[Jan 04 2015 05:38:14.490 AM] DEBUG wire:72 - http-outgoing-2 << "Expires: Thu, 01 Jan 1970 00:00:00 GMT[\r][\n]"
[Jan 04 2015 05:38:14.491 AM] DEBUG wire:72 - http-outgoing-2 << "Cache-Control: no-cache[\r][\n]"
[Jan 04 2015 05:38:14.491 AM] DEBUG wire:72 - http-outgoing-2 << "Cache-Control: no-store[\r][\n]"
[Jan 04 2015 05:38:14.491 AM] DEBUG wire:72 - http-outgoing-2 << "Set-Cookie: JSESSIONID=0C1444D7E0FA6B0F3CCE20AFBA28237E; Path=/; HttpOnly[\r][\n]"
[Jan 04 2015 05:38:14.491 AM] DEBUG wire:72 - http-outgoing-2 << "Front-End-Https: on[\r][\n]"
[Jan 04 2015 05:38:14.491 AM] DEBUG wire:72 - http-outgoing-2 << "Content-Encoding: gzip[\r][\n]"
[Jan 04 2015 05:38:14.491 AM] DEBUG wire:72 - http-outgoing-2 << "[\r][\n]"
[Jan 04 2015 05:38:14.491 AM] DEBUG wire:72 - http-outgoing-2 << "37[\r][\n]"
[Jan 04 2015 05:38:14.492 AM] DEBUG wire:72 - http-outgoing-2 << "[0x1f][0x8b][0x8][0x0][0x0][0x0][0x0][0x0][0x0][0x3][0xb3])J-.[0xc8][0xcf]+N[0xb5][0xe3][0xb2][0x81]1[0xe3][0x93][0xf3]S[0x80][0xfc][0xe0][0xd2][0xe4][0xe4][0xd4][0xe2]b.[0x1b]}4[0x9][0x84][0x80][0x1d][0x17][0x0]<Xn]@[0x0][0x0][0x0][\r][\n]"
[Jan 04 2015 05:38:14.492 AM] DEBUG wire:72 - http-outgoing-2 << "0[\r][\n]"
[Jan 04 2015 05:38:14.492 AM] DEBUG wire:72 - http-outgoing-2 << "[\r][\n]"

At first, I thought "[0x1f][0x8b][0x8][0x0][0x0]..." is hex string but there are non-hex string such as ")J-." and "+N." and each debug line is ending with "[\r][\n]". How would I be able to determine what response the app received.

I have spent quite few hours trying to find the answer but was unsuccessful.

Does it have to do anything with gzip compression or chunked transfer encoding?

Your help would be greatly appreciated if you could answer it.

Thanks,
James

Answer 1

HttpClient wire log replaces character 10 with [\n], character 13 with [\r], characters < 32 and > 127 (non-printable) with [0xN] where N is char's hexadecimal representation.

The reason you are seeing a lot of non-printable chars is indeed GZIP content encoding.