How to get JSON data in chunks to report on progress? [duplicate]

Question

I need to download contact data via a REST API, which I get in JSON format. Issue is, it might be many many contacts, so I want to observe the progress (how many contacts have already been downloaded) and report back to the user (with a progress bar, the code below runs in a thread).

However, it seems that the line client.execute(getRequest); establishes the connection and downloads the whole content in one go, i.e. my InputStream reader (to fetch the content in chunks) seems to be useless. Now I'm wondering how to make it work in chunks, so I can report on the progress?

/** prepare HTML get request */
HttpGet getRequest = new HttpGet(url[0]);
getRequest.addHeader("Authorization", "OAuth " + myTokens.get_access_token());

/** execute HTML request */
DefaultHttpClient client = new DefaultHttpClient();
JSONArray records = null;
HttpResponse response = client.execute(getRequest);

/** init response handlers for input stream */
InputStream input = new BufferedInputStream(response.getEntity().getContent());

byte data[] = new byte[MAX_BUFFER_SIZE];

long totalContactsCount = -1;
int readContactsCount = 0;
int currentByteReadCount = 0;

/** read response from inpus stream */
while ((currentByteReadCount = input.read(data)) != -1) {
    String readData = new String(data, 0, currentByteReadCount);
    dataString.append(readData);

    // then +1 progress on every ...},{... (JSON object separator)
    if (readData.indexOf("},{") >= 0) {
        readContactsCount++;
    }

    // publishing the progress....
    if (totalContactsCount > 0) {
        publishProgress((int)(readContactsCount * 100 / totalContactsCount));
    }
}
input.close();

/** transform response into JSONArray */
String result = dataString.toString();

//... convert into JSONArray

Answer 1

Unfortunatelly, as it seems from the documentation on DefaultHttpClient, the execute method returns a HTTP response, already parsed, and hence, already read from the network, so you cannot plug in the input stream. That stream is prepared for you to be read, but once it has been retreived, as you have noticed. You'd have to use another strategy. For example, trying yourself the connection using the sockets API.