How to avoid parsing when the server returns the same response using Retrofit?

Question

I used to avoid parsing the server response over and over if it did not change by calculating the hash of the response:

public class HttpClient {

    protected OkHttpClient mClient = new OkHttpClient();

    public String get(final URL url, final String[] responseHash)
        throws IOException {
        HttpURLConnection connection = new OkUrlFactory(mClient).open(url);
        InputStream inputStream = null;
        MessageDigest messageDigest = null;
        try {
            messageDigest = MessageDigest.getInstance("MD5");
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        }
        assert messageDigest != null;
        try {
            // Read the response.
            inputStream = connection.getInputStream();
            byte[] response = readFully(inputStream);
            final byte[] digest = messageDigest.digest(response);
            responseHash[0] = Base64.encodeToString(digest, Base64.DEFAULT);
            return new String(response, Util.UTF_8);
        } finally {
            if (inputStream != null) {
                inputStream.close();
            }
        }
    }

    private byte[] readFully(InputStream in) throws IOException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        byte[] buffer = new byte[1024];
        for (int count; (count = in.read(buffer)) != -1; ) {
            out.write(buffer, 0, count);
        }
        return out.toByteArray();
    }

}

This is the response header:

HTTP/1.1 200 OK
Server: Apache/2.4.10 (Linux/SUSE)
X-Powered-By: PHP/5.4.20
X-UA-Compatible: IE=edge
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Expires: Thu, 08 Oct 2015 16:15:09 +0000
X-Frame-Options: SAMEORIGIN
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Date: Wed, 07 Oct 2015 16:15:09 GMT
X-Varnish: 505284843
Age: 0
Via: 1.1 varnish
Connection: keep-alive

Now that I switched to Retrofit I wonder what is an elegant way to avoid parsing the same response? Are interceptors the way to go? I am not in charge of the server backend nor can I modify it.

Answer 1

Update

You could use the Expires header for cache control so you can avoid unneeded downloads. I don't think it´s a good approach but in this case since you don't have control over the server side, it´s the only way I could think of right now.

The expiration time of an entity MAY be specified by the origin server using the Expires header (see section 14.21). Alternatively, it MAY be specified using the max-age directive in a response. When the max-age cache-control directive is present in a cached response, the response is stale if its current age is greater than the age value given (in seconds) at the time of a new request for that resource. The max-age directive on a response implies that the response is cacheable (i.e., "public") unless some other, more restrictive cache directive is also present.

If a response includes both an Expires header and a max-age directive, the max-age directive overrides the Expires header, even if the Expires header is more restrictive. This rule allows an origin server to provide, for a given response, a longer expiration time to an HTTP/1.1 (or later) cache than to an HTTP/1.0 cache. This might be useful if certain HTTP/1.0 caches improperly calculate ages or expiration times, perhaps due to desynchronized clocks.

Many HTTP/1.0 cache implementations will treat an Expires value that is less than or equal to the response Date value as being equivalent to the Cache-Control response directive "no-cache". If an HTTP/1.1 cache receives such a response, and the response does not include a Cache-Control header field, it SHOULD consider the response to be non-cacheable in order to retain compatibility with HTTP/1.0 servers.

Note: An origin server might wish to use a relatively new HTTP cache control feature, such as the "private" directive, on a network including older caches that do not understand that feature. The origin server will need to combine the new feature with an Expires field whose value is less than or equal to the Date value. This will prevent older caches from improperly caching the response.

---

There're different approaches. I use this one:

• On the server response we get the Etag header and save it on SharedPreferences.
• Every server call goes with the "If-None-Match" header with the Etag value.
• The server, compares the Etag values and returns 304 - Not Modified or the result of the request itself if something changed and the content needs to be updated.

You can use a RequestInterceptor to do this as you pointed out:

public class HeaderRequestInterceptor implements RequestInterceptor {

    private final static String TAG = 
        HeaderRequestInterceptor.class.getSimpleName();

    private SharedPreferences mPreferences;

    public HeaderRequestInterceptor() {
        mPreferences = PreferenceManager.getDefaultSharedPreferences(
            DaoApplication.getAppContext());
    }

    @Override
    public void intercept(RequestFacade request) {
        String etagValue = mPreferences.getString(EtagConfig.MY_ETAG_VALUE, "");
        request.addHeader("If-None-Match", etagValue);
    }
}

Sample output:

Retrofit  D  ---> HTTP GET https://url.irontec.com/rest/schedule
    D  If-None-Match:
    D  Authorization: MyToken M2JiOGQwZGNjNWJiNWNiOTA1Yjc3YTA0YTAyMzEwYWY6OjIwMTUtMTAtMDhUMTM6MDc6MDMrMDA6MDA=
    D  Connection: close

Retrofit  D  <--- HTTP 200 https://url.irontec.com/rest/schedule (559ms)
    D  : HTTP/1.1 200 OK
    D  Access-Control-Allow-Credentials: true
    D  Access-Control-Allow-Headers: Authorization, Origin, Content-Type, X-CSRF-Token
    D  Access-Control-Allow-Methods: GET, PUT, POST, OPTIONS, DELETE
    D  Access-Control-Allow-Origin: *
    D  Connection: close
    D  Content-Type: application/json; charset=UTF-8;
    D  Date: Thu, 08 Oct 2015 13:07:07 GMT
    D  Etag: a3145c3f85f2dca1c78f87107331c766
    D  Server: Apache
    D  Transfer-Encoding: chunked
    D  X-Android-Received-Millis: 1444309624169
    D  X-Android-Response-Source: NETWORK 200
    D  X-Android-Sent-Millis: 1444309623870
    D  X-Content-Type-Options: nosniff
    D  X-Frame-Options: sameorigin

Now when refreshing the content:

Retrofit  D  ---> HTTP GET https://url.irontec.com/rest/schedule
    D  If-None-Match: a3145c3f85f2dca1c78f87107331c766
    D  Authorization: MyToken MGQ1OWM4YjViYTMxZWM3OGRmMDBlYTZjNmFjNDY3MmI6OjIwMTUtMTAtMDhUMTM6MTA6MDkrMDA6MDA=
    D  Connection: close
    D  ---> END HTTP (no body)

Retrofit  D  <--- HTTP 304 https://url.irontec.com/rest/schedule (299ms)
    D  : HTTP/1.1 304 Not Modified
    D  Connection: close
    D  Date: Thu, 08 Oct 2015 13:10:12 GMT
    D  Server: Apache
    D  X-Android-Received-Millis: 1444309809335
    D  X-Android-Response-Source: NETWORK 304
    D  X-Android-Sent-Millis: 1444309809163
    D  <--- END HTTP (0-byte body)

Answer 2

Yes, you can use an interceptor. Keep in mind it will run on all your requests, so need to account for that. Here is a sample interceptor to avoid parsing if the data this the same as a given hash. First, it uses headers to communicate the expected and computed hash values. Since headers have character restrictions, I used hex encoding instead of base64 for the hash. If the expected hash is null, it lets the request handle as normal and does no hash checking. This is to account for requests that you may not want to hash. If the expected hash is non-null and not equal to the computed hash, Retrofit parsing occurs as normal, except we add a header to the response so the caller can store the returned hash. If the expected and computed hash are equal, then the response is converted to a 204 (No content) with no body, which will prevent parsing.

public class HashingInterceptor implements Interceptor {
    public static final String HASH_HEADER = "content-hash";

    final protected static char[] hexArray = "0123456789abcdef".toCharArray();

    public static String bytesToHex(byte[] bytes) {
        char[] hexChars = new char[bytes.length * 2];
        for (int j = 0; j < bytes.length; j++) {
            int v = bytes[j] & 0xFF;
            hexChars[j * 2] = hexArray[v >>> 4];
            hexChars[j * 2 + 1] = hexArray[v & 0x0F];
        }
        return new String(hexChars);
    }

    @Override
    public com.squareup.okhttp.Response intercept(Chain chain) throws IOException {
        Request request = chain.request();
        String expectedHash = request.header(HASH_HEADER);
        if (expectedHash != null) {
            com.squareup.okhttp.Response response = chain.proceed(request);
            byte[] bytes = response.body().bytes();
            try {
                MessageDigest messageDigest = MessageDigest.getInstance("MD5");
                final byte[] digest = messageDigest.digest(bytes);
                String responseHash = bytesToHex(digest);
                if (responseHash.equals(expectedHash)) {
                    return response.newBuilder()
                            .code(204).build();
                } else {
                    return response.newBuilder()
                        .body(ResponseBody.create(
                            response.body().contentType(), bytes))
                        .addHeader(HASH_HEADER, responseHash)
                        .build();
                }
            } catch (NoSuchAlgorithmException e) {
                throw new IOException(e);
            }

        } else {
            // Header was not set, just proceed as usual
            return chain.proceed(request);
        }
    }
}

To use, create an interface. Note: I am assuming you are using Retrofit 2 here.

public interface GitHubService {

    @GET("/users/{user}")
    Call<User> users(
        @Path("user") String user, 
        @Header(HashingInterceptor.HASH_HEADER) String hash);

}

create and use --

OkHttpClient okHttpClient = new OkHttpClient();
okHttpClient.interceptors()
        .add(new HashingInterceptor());
Retrofit retrofit = new Retrofit.Builder()
        .baseUrl("https://api.github.com/")
        .addConverterFactory(GsonConverterFactory.create())
        .client(okHttpClient)
        .build();
final GitHubService gitHubService = retrofit.create(GitHubService.class);
// Note: make sure hashValue is non-null on the first request to 
// make sure the hash is computed
Call<User> users = gitHubService.users("octocat", hashValue);
users.enqueue(new Callback<User>() {
    @Override
    public void onResponse(Response<User> response, Retrofit retrofit) {
        // 200 = updated date, 204 = same data, not parsed
        Log.d("response", "code = " + response.code());  
        Log.d("response", "returned hash = " +  
            response.headers().get(HashingInterceptor.HASH_HEADER));
    }

    @Override
    public void onFailure(Throwable t) {
        t.printStackTrace();
    }
});