What to do about huge resources in REST API

Question

I am bolting a REST interface on to an existing application and I'm curious about what the most appropriate solution is to deal with resources that would return an exorbitant amount of data if they were to be retrieved.

The application is an existing timesheet system and one of the resources is a set of a user's "Time Slots". An example URI for these resources is:

/users/44/timeslots/

I have read a lot of questions that relate to how to provide the filtering for this resource to retrieve a subset and I already have a solution for that.

I want to know how (or if) I should deal with the situation that issuing a GET on the URI above would return megabytes of data from tens or hundreds of thousands of rows and would take a fair amount of server resource to actually respond in the first place.

• Is there an HTTP response that is used by convention in these situations?
  I found HTTP code 413 which relates to a Request entity that is too large, but not one that would be appropriate for when the Response entity would be too large
• Is there an alternative convention for limiting the response or telling the client that this is a silly request?
• Should I simply let the server comply with this massive request?

EDIT: To be clear, I have filtering and splitting of the resource implemented and have considered pagination on other large collection resources. I want to respond appropriately to requests which don't make sense (and have obviously been requested by a client constructing a URI).

Answer 1

You are free to design your URIs as you want encoding any concept.

So, depending on your users (humans/machines) you can use that as a split on a conceptual level based on your problem space or domain. As you mentioned you probably have something like this:

/users/44/timeslots/afternoon
/users/44/timeslots/offshift
/users/44/timeslots/hours/1
/users/44/timeslots/hours/1
/users/44/timeslots/UTC1624

Once can also limit by the ideas/concepts as above. You filter more by adding queries /users/44/timeslots?day=weekdays&dow=mon

Making use or concept and filters like this will naturally limit the response size. But you need to try design your API not go get into that situation. If your client misbehaves, give it a 400 Bad Request. If something goes wrong on your server side use a 5XX code.

Make use of one of the tools of REST - hypermedia and links (See also HATEOAS) Link to the next part of your hypermedia, make use of "chunk like concepts" that your domain understands (pages, time-slots). No need to download megabytes which also not good for caching which impacts scalability/speed.

Answer 2

timeslots is a collection resource, why won't you simply enable pagination on that resource

see here: Pagination in a REST web application

calling get on the collection without page information simply returns the first page (with a default page size)

Should I simply let the server comply with this massive request? I think you shouldn't, but that's up to you to decide, can the server handle big volumes? do you find it a valid usecase?