Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why is it a common practice to encode pagination cursors or id values as string?

For example the Facebook Graph API: why are after and before base64 encoded numbers?

{
  "data": [
     ... Endpoint data is here
  ],
  "paging": {
    "cursors": {
      "after": "MTAxNTExOTQ1MjAwNzI5NDE=",
      "before": "NDMyNzQyODI3OTQw"
    },
    "previous": "https://graph.facebook.com/me/albums?limit=25&before=NDMyNzQyODI3OTQw"
    "next": "https://graph.facebook.com/me/albums?limit=25&after=MTAxNTExOTQ1MjAwNzI5NDE="
  }
}

What benefits could it possibly bring in contrast to just plain numbers?

As the following python log shows, the benefits can not be shorter representation of the data or the data containing unsafe characters:

>>> base64.b64decode("MTAxNTExOTQ1MjAwNzI5NDE=")
'10151194520072941'
>>> len('10151194520072941')
17
>>> len("MTAxNTExOTQ1MjAwNzI5NDE=")
24
like image 621
stefreak Avatar asked Feb 08 '15 02:02

stefreak


People also ask

What is a cursor pagination?

Cursor-based pagination works by returning a pointer to a specific item in the dataset. On subsequent requests, the server returns results after the given pointer.

Is pagination linked list?

Cursor ImplementationIf offset pagination is an array, then cursor pagination is a linked list. Where offset grabs records based on where they are located in the table like an index, cursors use a pointer that points to a specific record and grabs the records following this specific record.


1 Answers

Highest possible number in JavaScript is 9007199254740992 according to question asked in StackOverflow What is JavaScript's highest integer value that a Number can go to without losing precision?

If you compare these values

9007199254740992    // the JS maximum
10151194520072941   // the Base64 encoded number

If certainly looks like Facebook is internally - for reasons we do not know - storing values which are too big for JavaScript number precision to handle.

Thus, it seems to me that they had no other possibility than to handle numbers as strings.

Of course they could just use "10151194520072941" as number in string format but some programmers might confuse this to be a number. Even though this is rarely happening, they probably thought that Base64 encoding the number avoids the problem of someone converting the string to integer.

Furthermore, because this is Public API function, it is not used by their own engineers, so the risk is even higher, because persons using the API come from different educational backgrounds. They could accidentally use for example parseInt or similar to the number resulting in unnecessary customer service requests.

EDIT: Using very big numbers might also serve another purpose: detecting purposeful abuse of the API. If they would be using for example random UUID values or consecutive numeric values, any close-by value could be potentially legal. If it is an UUID they first must make the request to see if it is a legal entry. Having a big number base it could be that only every 1000th is legal or they follow some other mathematical rule which can be detected by a single server, without requests to other server, sorting out clients which are purposefully crafting requests with illegal values becomes much more effective and perhaps can be filtered out before they reach the databases.

like image 87
Tero Tolonen Avatar answered Sep 28 '22 07:09

Tero Tolonen