Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Best practice for implementing long-running searches with REST

Tags:

rest

As part of a REST Service, I need to implement a search call. The basic idea is that the user can POST a new search, and GET the results:

POST http://localhost/api/search
GET http://localhost/api/search?id=123

However, my search may run a few minutes, and return partial results until it is done. That is, the GET-Request would return something like:

status: running
results: a, b, c.

while the next GET-Request might return

status: completed
results: a, b, c, d, e.

This contradicts the semantics of a RESTful GET request. The request should always return the same result when called several times. For example, when the user uses a caching proxy, the full results might never be delivered to the user.

Question: Is there any way to provide a truly RESTful implementation for long running searches with partial results?

like image 623
fabstab Avatar asked Oct 11 '11 17:10

fabstab


People also ask

What is a good practice that should be followed for a REST API?

REST API Best Practices: Prioritize Nouns over Verbs in URI Since REST API is mostly developed for resources like services, it is essential to use Nouns and not verbs. So it is better to use only Nouns to represent an entity in REST endpoint paths. This is because the HTTP request method already consists of verbs.

What should REST API return?

The API should always return sensible HTTP status codes. API errors typically break down into 2 types: 400 series status codes for client issues & 500 series status codes for server issues. At a minimum, the API should standardize that all 400 series errors come with consumable JSON error representation.


3 Answers

While the search is executing, you could set the appropriate response headers (e.g. Expires or max-age) to indicate that the response should not be cached (HTTP/1.1 14.9.3, 13.4).

Once the search result is complete, you could then send a more appropriate Expires / max-age header to allow or extend the cacheability of the result.

The burden would be on the client to re-query the resource until its search status is complete. The client could maybe use the value of the Expires header to determine when it should re-query for updated results.

Alongside of this, you could also use a custom 2XX status code to indicate that the result is not yet complete. Maybe a HTTP/1.1 299 In Progress, or whatever makes sense. The spec indicates that HTTP status codes are extensible.


For the record, your statement:

This contradicts the semantics of a RESTful GET request. The request should always return the same result when called several times.

is not true for GET requests - resources can change. That GET requests are idempotent only means that "...the side-effects of N > 0 identical requests is the same as for a single request". [spec]

like image 99
Rob Hruska Avatar answered Oct 17 '22 23:10

Rob Hruska


A few days ago I happend to stumble upon a blog post over at reddit that deals with your problem. You might want to check it out: Bill Higgin's RESTy long-ops.

Happy reading.

like image 23
aefxx Avatar answered Oct 17 '22 23:10

aefxx


It's not a problem if the first GET request returns partial results, and the second GET request returns the full results. That's because the first GET request doesn't cause the result of the second request to change: That request would have returned the full results, even if the first GET hadn't been issued. "idempotent" doesn't mean identical results. It means that the first GET doesn't affect the second GET.

It would be a problem if the first GET request returned partial results, and the second GET would return the remaining results (first GET returns A, B, C; second GET returns D, E, F). Here the first GET changes the second result, so it's not RESTful.

like image 3
gnasher729 Avatar answered Oct 17 '22 23:10

gnasher729