What if RESTful Web API default GET would return way too many results?

Question

Environment: ASP.NET MVC 4 Web API

Given that the RESTful standard for GET is:

http://www.example.com/api/entity/ returns a collection of all entity http://www.example.com/api/entity/{id} returns entity with ID of {id}

...what should be done if there are so many "entity" that making that first call would return a ridiculous amount of result data?

I could just disallow that call altogether and return an HTTP 500 or something, but I'd rather conform to the standard and implement some sort of range option, arbitrary though it may be.

I've been searching around for HTTP headers that would allow for what equates to pagination and came across Accept-Ranges, but couldn't find an applicable example (if that's even correct to begin with). Am I on the right track? Are there any resources out there that might help?

Answer 1

Accept-Ranges is a response header, so that doesn't help you send pagination requests from your client to the server. The spec lets you return any value for this header (although the only standardised value is bytes), so you could use Accept-Ranges as a way for your server to inform clients that your API supports pagination, but TBH that probably isn't that useful.

The matching request header is Range. The value of this header is a ranges-specifier, but unfortunately the only valid value for this according to the spec is a byte-ranges-specifier such as The first 500 bytes (byte offsets 0-499, inclusive): bytes=0-499 which isn't useful for the type pagination we want.

Since there is no standard, you will just have to make up you own request headers or query params for pagination. Here is what we do for a RESTful API I am working on:

GET /users/?offset=10&limit=50

{
    "users": [...],
    "offset": 10,
    "limit": 50,
    "total": 10000
}