RxJava and cursor based RESTful pagination

Question

I'm working with the Spotify API and am hoping to chain a few paginated results using RxJava. Spotify uses cursor based pagination, so solutions like the one from @lopar will not work.

The response is from this call and looks something like this (imagine there are 50 items):

{
  "artists" : {
    "items" : [ {
      "id" : "6liAMWkVf5LH7YR9yfFy1Y",
      "name" : "Portishead",
      "type" : "artist"
    }],
    "next" : "https://api.spotify.com/v1/me/following?type=artist&after=6liAMWkVf5LH7YR9yfFy1Y&limit=50",
    "total" : 119,
    "cursors" : {
      "after" : "6liAMWkVf5LH7YR9yfFy1Y"
    },
    "limit" : 50,
    "href" : "https://api.spotify.com/v1/me/following?type=artist&limit=50"
  }
}

Right now, I'm getting the first 50 results like this, using retrofit:

public class CursorPager<T> {
    public String href;
    public List<T> items;
    public int limit;
    public String next;
    public Cursor cursors;
    public int total;

    public CursorPager() {
    }
}

public class ArtistsCursorPager {
    public CursorPager<Artist> artists;

    public ArtistsCursorPager() {
    }
}

then

public interface SpotifyService  {

    @GET("/me/following?type=artist")
    Observable<ArtistsCursorPager> getFollowedArtists(@Query("limit") int limit);

    @GET("/me/following?type=artist")
    Observable<ArtistsCursorPager> getFollowedArtists(@Query("limit") int limit, @Query("after") String spotifyId);

}

and

mSpotifyService.getFollowedArtists(50)
        .flatMap(result -> Observable.from(result.artists.items))
        .flatMap(this::responseToArtist)
        .sorted()
        .toList()
        .subscribe(new Subscriber<List<Artist>>() {
            @Override
            public void onNext(List<Artist> artists) {
                callback.onSuccess(artists);
            }
            // ...
        });

I'd like to return all (in this case 119) artists in callback.success(List<Artist>). I'm new to RxJava, so I'm unsure if there is a smart way to do this.

Answer 1

The only problem with the recursive solution is the stack over flow problem. A way to do it without recursion is

Observable<ArtistsCursorPager> allPages = Observable.defer(() ->
{
    BehaviorSubject<Object> pagecontrol = BehaviorSubject.create("start");
    Observable<ArtistsCursorPager> ret = pageControl.asObservable().concatMap(aKey ->
    {
        if (aKey != null && aKey.equals("start")) {
            return Observable.getFollowedArtists(50).doOnNext(page -> pagecontrol.onNext(page.cursors.after));
        } else if (aKey != null && !aKey.equals("")) {
            return Observable.getFollowedArtists(50,aKey).doOnNext(page -> pagecontrol.onNext(page.cursors.after));
        } else {
            return Observable.<ArtistsCursorPager>empty().doOnCompleted(()->pagecontrol.onCompleted());
        }        
    });
    return ret;
});

See the solutions to this question.