REST APIs and messaging

Question

I have a system that exposes a REST API with a rich set of CRUD endpoints to manage different resources. The REST API is used also by a front-end application that executes calls by using Ajax.

I would like to make some of these calls asynchronous and add reliability.

The obvious choice seems a message broker (ActiveMQ, RabbitMQ, etc...).

Never used message brokers before and I am wondering if they can be "put in front of" the REST API without having to rewrite them.

I do not want to access the REST API only through the messaging system: for some endpoints, a call must always be synchronous and the reliability is less important (mainly because in case of error the user receives an immediate feedback).

Would a full ESB be a better option for this use case?

Answer 1

If I understand your question, you would like to "register" an API endpoint as a subscriber so that it could receive the messages sent to a given queue.

I do not think that a message broker can be configured to do this.

For example, if you want to use a message broker, both your producers and subscribers need to use the JMS API.

I do not know if a solution can be to implement a subscriber that will execute the corresponding API call. In this case, the reliability is compromised because the message will be dequeued before the API call is executed. It can make sense if the subscriber is running in the same process of the API, but in this case it is not clear why you should use a REST API instead of a library.

Answer 2

IMO @EligioEleuterioFontana you have a misunderstanding of the roles of:

• an RESTful Api
• a message broker

These are two different subsystems which provide different services.

Now, let's explain their roles with respect to your requirements:

• You have clients (desktop browsers, mobile phone browsers or apps) which need to get/push data to your system. (Assumption from the REST API mention).
• Requests from the clients are using HTTP/HTTPS (that's the REST API part of your requirement).
• Any data that is pushed, you wish to make this more responsive, quicker, reliable.

If I've gotten that right, then I would answer it as:

• All clients need to push requests to a REST API because this does just more than simple CRUD. The Api also handles things like security (authentication and authorization), caching, possibly even request throttling, etc.

• REST API should always been the front end to clients as this also 'hides' the subsystems that the API uses. Users should never see/know about any of your subsystem choices (eg. what DB you are using. Are you caching? if so, with what? etc).

• Message Brokers are great for offloading the work that was requested now and handling the work later. There's heaps of ways this can be done (queues or pub/sub, etc) but the point here is this is a decision the clients should never see or know about.

• MB's are also great for resilience (as you noted). If something fails, the message on a queue would be re-attempted after 'x' time ... etc. (no, I'm not going to mention poison queues, dead letter queue, etc).

You can have some endpoints of the Api that are synchronous. Sure! Then have others that leverage some eventual consistency (i.e. for that request, I'll deal with it later (even if later in 5 secs later) and just return the response to the client saying "thanks! got it! I'll do it soon"). This is the asynchronous workflow you are after.

The API endpoints needs to be simple, concise and hopefully pretty stable. What you do behind the scenes as you change things hopefully will be hidden away from the clients. This includes the use of message brokers.

---

Anyway, that my take on how I see REST APIs and Message Brokers and how they related to each other.

Answer 3

It might be worth looking into the Google Cloud sub/pub? - https://cloud.google.com/pubsub/docs/overview