How to paginate Trello api response?

Question

I am using Trello's API to pull the boards and their cards.

I've noticed one of the boards I am pulling is approaching thousand cards at the moment, so I wonder how to paginate the results?

I tried using the since attribute and feed last card's update time into it, but afterwards I get zero results, so I must be doing somethign wrong.

The api documentation doesn't say much and the API response I've tried is very confusing:

/boards/xyz/cards?limit=1&before=2015-1-1

returns card which activity is after specified date.

"dateLastActivity"=>"2015-02-03T17:51:23.253Z",

and there is no other date field I could use to continue pagination.

Answer 1

So since the API really desn't work as you'd expect it to, I am documenting this here at least.

I created a testing board with cards A, B, C, ... , M so I can test on it.

First, grabbing all cards works as expected.

api.find(:boards, '56af532ae7a74a5fcac32e66').cards.map { |c| [c.name, c.id] }

=> [
 ["A", "56af53367f15b163c0e345fd"],
 ["B", "56af533783d4fccc1cce3941"],
 ["C", "56af533831727db409451c9e"],
 ["D", "56af53397bf672bb56c7c536"],
 ["E", "56af5339f5e67cf6bcc7323b"],
 ["F", "56af533ac0aa02de54ab881a"],
 ["G", "56af533b212b095266ebb506"],
 ["H", "56af533e11992f20f375fca1"],
 ["I", "56af533fc1775129aaa7028b"],
 ["J", "56af5345f81227a7a1cc3a35"],
 ["K", "56af5345e36b3d3c45b16967"],
 ["L", "56af534d964b6c4aa93d4aca"],
 ["M", "56af534e768610ff67c781ce"]
]

Note: the order of the cards isn't by create time, its by their appearance in the board from left column to right column and from top to bottom.

Limit

Okay, time to start paginating. Let's get only the first card, shall we?

board.cards(limit: 1).map { |c| [c.name, c.id] }

=> [["M", "56af534e768610ff67c781ce"]]

Ooops, API returns the last element. Definitely unexpected. Lets play with the after/before next.

Before

Let's grab card before the last one.

board.cards(limit: 1, before: '56af534e768610ff67c781ce').map { |c| [c.name, c.id] }

=> [["L", "56af534d964b6c4aa93d4aca"]]

That works well, L is before M.

After

What happens if we grab a card after the last one, response should be empty, right?

board.cards(limit: 1, after: '56af534e768610ff67c781ce').map { |c| [c.name, c.id] }

=> [["M", "56af534e768610ff67c781ce"]]

Mmmm, what? What happens if I grab two card then?

board.cards(limit: 2, after: '56af534e768610ff67c781ce').map { |c| [c.name, c.id] }

=> [
  ["L", "56af534d964b6c4aa93d4aca"], 
  ["M", "56af534e768610ff67c781ce"]
]

This defies any logic. Definitely can't go through the list from start to finish then, because there is no end to the card list.

Back to pagination

We'll have to go through the cards in reverse order in that case. Just specifying limit works, since by default we're going in reverse order. (which makes total sense, right?)

api.find(:boards, '56af532ae7a74a5fcac32e66').cards(limit: 5).map { |c| [c.name, c.id] }

=> [
 ["I", "56af533fc1775129aaa7028b"], 
 ["J", "56af5345f81227a7a1cc3a35"], 
 ["K", "56af5345e36b3d3c45b16967"], 
 ["L", "56af534d964b6c4aa93d4aca"], 
 ["M", "56af534e768610ff67c781ce"]
]

Don't let the order of these items fool you! Its still the order how they appear in the board, they're not sorted by the time of creation. If I move M card in its column so that it is right over L, the API will return them in reverse order.

The question is now, how to grab the older card from the list? As the other answer says, the IDs are timestamp, so we just sort by the ID and pick the lowest one and continue paginating.

Note: If you're fast, you can create multiple cards during the same second, so the IDs may not in the right order as they just use the number of seconds, not microseconds. That doesn't prevent us from paginating though.

board.cards(limit: 5, before: '56af533fc1775129aaa7028b').map { |c| [c.name, c.id] }

=> [
  ["D", "56af53397bf672bb56c7c536"], 
  ["E", "56af5339f5e67cf6bcc7323b"], 
  ["F", "56af533ac0aa02de54ab881a"], 
  ["G", "56af533b212b095266ebb506"], 
  ["H", "56af533e11992f20f375fca1"]
]

Awesome, we're getting the next page, let's get the last one.

board.cards(limit: 5, before: '56af53397bf672bb56c7c536').map { |c| [c.name, c.id] }

=> [
  ["A", "56af53367f15b163c0e345fd"],
  ["B", "56af533783d4fccc1cce3941"],
  ["C", "56af533831727db409451c9e"]
]

Yesss, so the last page contains less cards, than the limit, so we know its the last page. Horray! (if it was equal to page size, we'd find the next response is empty)

Answer 2

​You are right in your assessment. The before and since parameters operate on card creation date. Because we use standard Mongo IDs, you can actually derive the Card creation date from the Card ID. (Here is one such tool: https://steveridout.github.io/mongo-object-time/)

​ ​As a convenience, Trello also allows you to pass in a Card ID to the before and since parameters and we will use the date for you. The recommended way of paging through a large set of cards is to repeat the same query, and use these parameters to move forward or backward.