How to handle pagination in retrofit

Question

I'm building an app using retrofit. Everything's working swimmingly, but I'm worried about the size of my API requests and would like to split them up using pagination.

What would be the best strategy to page through the API automatically with Retrofit, so that all available data is downloaded by default?

Answer 1

So I ended up solving my problem like this:

I am using Grape on my server, so I installed the Grape-kaminari gem to handle the pagination server-side. Grape-kaminari provides a page query on your urls, and adds handy paging information to the header response.

I wrote a small class to allow me to automatically recurse through pages until I've consumed all the data on the API:

package com.farmgeek.agricountantdemo.app.helpers;

import android.util.Log;

import retrofit.client.Header;
import retrofit.client.Response;

public class APIHelper {
    public static PaginationData getPaginationData(Response response) {
        int currentPage = 1;
        int totalPages = 1;
        for (Header header : response.getHeaders()) {
            try {
                    if (header.getName().equals("X-Page")) {
                        currentPage = Integer.parseInt(header.getValue());
                    } else if (header.getName().equals("X-Total-Pages")) {
                        totalPages = Integer.parseInt(header.getValue());
                    }
            } catch (NullPointerException e) {
                // We don't care about header items
                // with empty names, so just skip over
                // them.
                Log.w("APIHelper -> getPaginationData", "Skipped over header: " + e.getLocalizedMessage());
            }
        }
        return new PaginationData(currentPage, totalPages);
    }

    public static class PaginationData {
        public final int page;
        public final int total;

        public PaginationData(int currentPage, int totalPages) {
            this.page = currentPage;
            this.total = totalPages;
        }
    }
}

I'd then use it in my API call like so:

public void getStuff(int page) {
    final RestAdapter restAdapter = buildRestAdapter();

    // Tell the sync adapter something's been added to the queue
    ApiService apiService = restAdapter.create(ApiService.class);
    apiService.getStuff(page, new Callback<List<Stuff>>() {
        @Override
        public void success(final List<Stuff> stuffList, Response response) {
            final APIHelper.PaginationData pagination = APIHelper.getPaginationData(response);

            for (final Stuff stuff : stuffList) {
                handleRecord(stuff);
            }

            if (pagination.page == pagination.total) {
                App.getEventBus().postSticky(new StuffSyncEvent());
                App.getEventBus().post(new SuccessfulSyncEvent(Stuff.class));
            } else {
                // Otherwise pull down the next page
                new StuffSyncRequestAdapter().getStuff(pagination.page+1);
            }

        }

        @Override
        public void failure(RetrofitError error) {
            String errorMessage = error.getCause().getMessage();
            App.getEventBus().post(new UnsuccessfulSyncEvent(Stuff.class, errorMessage));
        }
    });
}

Answer 2

First, the pagination would need to be supported by the backend service that you're using. Second if you're looking to get an example of how this can be implemented from the client side using retrofit I would recommend you take a look at the u2020 project from @JakeWharton. The GalleryService retrofit interface implements such a mechanism in a very simple manner. Here's a link to the interface itself.

Here's a light example based on the u2020 project

// See how it uses a pagination index.
public interface GalleryService {
  @GET("/gallery/{page}") //
  Gallery listGallery(@Path("page") int page);
}

By tracking the total amount of items already downloaded from your rest service and a predefined maximum of items per page you can compute the page index necessary to call your rest service for your next set of items to download.

You can then call you rest api like this.

int nextPage = totalItemsAlreadyDownloaded / ITEMS_PER_PAGE + 1;    
restApi.listGallery(nextPage);

This is a very light example based on the u2020 project but hopefully it gives you an idea of how to attack this.