Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

how to handle / read the response with "Transfer-Encoding:chunked"?

May i know how to handle / read the response with "Transfer-Encoding:chunked"?

Currently im using common-httpclient.3.1

My current coding as followings (can handle response with content-length in header only) :-

            httppost = new PostMethod(localurl);
            httppost.setRequestHeader("Content-Type", "application/xml; charset=utf-8");
            RequestEntity entity = new StringRequestEntity(in, "application/xml", "UTF-8");
            httppost.setRequestHeader("Content-length", entity.getContentLength()+"");
            httppost.setRequestEntity(entity);
            for (int i=0; i<retryAttempt; i++) {
                try {
                    httpclient.executeMethod(httppost);
                    if (httppost.getStatusCode() == 200) {

                        br = new BufferedReader(new InputStreamReader(httppost.getResponseBodyAsStream(), httppost.getResponseCharSet()));
                        String reply = null;

                        long len = httppost.getResponseContentLength();

                        if(len != 0) {
                            char[] cbuf = new char[Integer.parseInt(len+"")];
                            if (br.read(cbuf, 0, Integer.parseInt(len+"")) != -1 ) {
                                repOut = String.valueOf(cbuf);
                            }
                        }else{
                            while ((reply = br.readLine()) != null) {
                                if (!reply.equals("")) repOut += reply+" \n ";
                            }
}

Response get from URL (via httppost):

Thread-9, READ: TLSv1 Application Data, length = 443
Padded plaintext after DECRYPTION:  len = 443
0000: 48 54 54 50 2F 31 2E 31   20 32 30 30 20 4F 4B 0D  HTTP/1.1 200 OK.
0010: 0A 53 65 72 76 65 72 3A   20 53 54 52 4F 4E 47 48  .Server: STRONGH
0020: 4F 4C 44 2F 33 2E 30 2F   4F 70 65 6E 42 53 44 0D  OLD/3.0/OpenBSD.
0030: 0A 44 61 74 65 3A 20 53   75 6E 2C 20 32 32 20 4D  .Date: Sun, 22 M
0040: 61 79 20 32 30 31 31 20   31 34 3A 30 31 3A 30 38  ay 2011 14:01:08
0050: 20 47 4D 54 0D 0A 43 6F   6E 74 65 6E 74 2D 74 79   GMT..Content-ty
0060: 70 65 3A 20 61 70 70 6C   69 63 61 74 69 6F 6E 2F  pe: application/
0070: 78 6D 6C 3B 63 68 61 72   73 65 74 3D 55 54 46 2D  xml;charset=UTF-
0080: 38 0D 0A 53 65 74 2D 63   6F 6F 6B 69 65 3A 20 4A  8..Set-cookie: J
0090: 53 45 53 53 49 4F 4E 49   44 3D 34 34 42 35 33 45  SESSIONID=44B53E
00A0: 35 46 38 30 39 41 35 35   32 34 32 41 44 36 34 38  5F809A55242AD648
00B0: 33 41 39 33 32 42 30 46   44 36 3B 20 50 61 74 68  3A932B0FD6; Path
00C0: 3D 2F 3B 20 53 65 63 75   72 65 0D 0A 54 72 61 6E  =/; Secure..Tran
00D0: 73 66 65 72 2D 65 6E 63   6F 64 69 6E 67 3A 20 63  sfer-encoding: c
00E0: 68 75 6E 6B 65 64 0D 0A   0D 0A 62 62 0D 0A 3C 3F  hunked....bb..<?
00F0: 78 6D 6C 20 76 65 72 73   69 6F 6E 3D 22 31 2E 30  xml version="1.0
0100: 22 20 65 6E 63 6F 64 69   6E 67 3D 22 55 54 46 2D  " encoding="UTF-
0110: 38 22 3F 3E 0A 3C 54 68   72 65 65 44 53 65 63 75  8"?>.<ThreeDSecu
0120: 72 65 3E 3C 4D 65 73 73   61 67 65 20 69 64 3D 22  re><Message id="
0130: 50 41 54 72 61 6E 73 52   65 71 53 70 65 63 34 31  PATransReqSpec41
0140: 37 39 32 30 31 31 30 35   32 32 32 32 30 30 30 32  7920110522220002
0150: 36 30 31 37 32 39 33 33   37 36 38 36 35 22 3E 3C  6017293376865"><
0160: 50 41 54 72 61 6E 73 52   65 73 3E 3C 76 65 72 73  PATransRes><vers
0170: 69 6F 6E 3E 31 2E 30 2E   32 3C 2F 76 65 72 73 69  ion>1.0.2</versi
0180: 6F 6E 3E 3C 2F 50 41 54   72 61 6E 73 52 65 73 3E  on></PATransRes>
0190: 3C 2F 4D 65 73 73 61 67   65 3E 3C 2F 54 68 72 65  </Message></Thre
01A0: 65 44 53 65 63 75 72 65   3E 0D 0A 76 67 28 7C B3  eDSecure>..vg(..
01B0: C8 F9 7C A1 C9 77 2F F6   E8 8A 42                 .....w/...B
[Raw read]: length = 5
like image 658
Jacelyn Avatar asked May 25 '11 12:05

Jacelyn


People also ask

How does transfer encoding chunked work?

In chunked transfer encoding, the data stream is divided into a series of non-overlapping "chunks". The chunks are sent out and received independently of one another. No knowledge of the data stream outside the currently-being-processed chunk is necessary for both the sender and the receiver at any given time.

How do you stop transfer encoding chunked?

Try adding "&headers=false" to your request. That should shorten it up and cause the response to be less likely to be chunked. Also, are you sending a HTTP/1.1 or HTTP/1.0 request? Try sending a HTTP/1.0 if your device cannot handle a HTTP/1.1 request.

How do you use transfer encoding?

Transfer-Encoding is a hop-by-hop header, that is applied to a message between two nodes, not to a resource itself. Each segment of a multi-node connection can use different Transfer-Encoding values. If you want to compress data over the whole connection, use the end-to-end Content-Encoding header instead.

How do I enable chunked transfer encoding?

To enable chunked transfer encoding, you need to set the value of AspEnableChunkedEncoding to "True" in the metabase of the site, server, or virtual directory for which chunked transfer encoding is enabled. By default this value is set to "True", you can try to change the value to "False" to disable it.


1 Answers

commons-httpclient will de-chunk your server's response out of the box, there's no need for you to handle chunking. However, you may be running into problems because - due to the nature of chunking - you don't know the length of the content until you've read it all. (Requesting the content length will return -1.) You therefore cannot allocate a fixed-size buffer to hold the entire content.

If you change your test of the getResponseContentLength return value to "if(len > 0)", I think you'll see that you fall into the readLine case when you encounter a response that is chunked, and that it should read the entire response line-by-line.

like image 107
Edward Thomson Avatar answered Oct 08 '22 12:10

Edward Thomson