Best practices for API hooks/callbacks?

Question

Lets say I have web applicatons/services:

• API
• Set of Applications

API is used for managing some resources (simple CRUD operations). Now what I need is to subscribe Applications for changes of different API resources. Applications would do some background work on a change.

---

I came up to idea of callbacks. So that Applications can oauthorise and post to the API a callback config.

I think that this config should look like this:

{
  'callback_url': 'http://3rdpartyservice.com/callback',
  'resources':    ['foo1', 'foo2'],
  'ref_data':     { 'token': 'abcd1234' }
}

• resources is array of the resources that 3rd party service is interested in
• ref_data is custom json for 3rd party usage (e.g. for auth)

This way on specified resource change the API would send a request to callback_url. This request would contain resource data, action(create/update/delete) and ref_data.

The intention here is to make this generic enough to allow 3rd party clients configure such callbacks.

---

So the question are:

1. Are there any best practices?
2. What about security potential issues?
3. Are there any real world examples on the web?

Tx

Answer 1

Sounds very similar as WebHooks or Service Hooks.

Check out the Web Hooks on GitHub, to get a good idea what they are and how they work. See also last alinea Service Hooks, as it explains how github handles these WebHooks. This would be similar for your application. The OAuth explains why and how it is done.

See also Webhooks, REST and the Open Web, from API User Experience.

There is even RestHooks.

Answer 2

The general solution to this requirement is usually called "publish/subscribe". There are dozens of solutions to this - google "publish subscribe REST" for some examples. You can also read "Enterprise Integration Patterns".

They key challenge in this kind of solution is "real-time versus queue".

For instance, if you have an API with a million clients, who are all interested in the same event, you cannot guarantee that in real time you can reach all of those clients within whatever timeframe their application demands. You also have to worry about the network going away, or clients being temporarily down. In this case, you application might define an event queue, and clients look in that queue for events they're interested in. Once you go down that route, you're probably going to use some off-the-shelf software rather than building your own. Apache Camel is a good open source implementation.

In your example, for instance, what happens if you cannot reach 3rdpartyservice.com? Or if http://3rdpartyservice.com/callback throws an error when posting an update to foo1, but not to foo2? Or if http://3rdpartyservice.com/ uses a different flavour of OAuth than you're used to? How do you guarantee http://3rdpartyservice.com/ that it's you who is posting an update, not a hacker?

Your choices really tend to come down to your non-functional requirements, rather than the functional ones - things like uptime, guarantee of notification, guarantee of delivery, etc. are more important than the specifics of how you pass across the parameters, and whether it's "resource-based" or some other protocol.