Callbacks in Thrift Asynchronous Functions?

Question

In Thrift it is possible to use the oneway modifier to specify a call as asynchronous.

Apparently, it's not possible to define a callback, though, to be executed when the execution of the function is completed.

It seems that the only possibility I have is to give my Thrift client (PHP) some "server" capabilities, so that, when the heavy computation is completed on the server side, I can send a notification to it. This means that I should have a new .thrift file, with new definitions, new services and all the rest and that I should generate php-server side code with Thrift.

Even if this is feasible, it looks like an overkill to me and I'm wondering if there's a more clever way to implement the callback.

Looking forward for some feedback from you, guys.

Answer 1

Roberto, unfortunately the Thrift framework has no such built-in functionality. There may be a number of alternatives, though, depending on what you want your PHP client session to do during the time you would normally have waited for the computationally-intensive Thrift server to answer (had you not used oneway.)

I can only imagine, for now, that you are in a situation where, having coded a web application where a user (or several users in parallel) can each trigger a computationally-intensive task, you would like to provide some feedback to said users while said tasks churn along.

From the start, you are absolutely right in trying to avoid the solution that you are trying to avoid. Your PHP client sessions cannot service a callback interface without blocking (unless you dig your hole even deeper by trying to use pcntl_fork or some other PHP threading band-aid.)

The simplest and IMHO best way out of this is two switch from an event-driven model (I want to be notified when the server is done) to a polling model (I will periodically inquire with the server whether or not it is done.) There are several ways of implementing a polling model, with multiple implementation options on the server as well as on the client sides, such as:

1. during the invocation phase:

   • the PHP client session allocates a unique job_id value; the session then makes the asynchronous oneway call void compute(..., job_id) to the computationally-intensive Thrift server,

   -- or --

   • the PHP client session makes a synchronous call job_id start_compute(...) to the computationally-intensive Thrift server; the server allocates the unique job_id value, then spawns the actual computationally-intensive task in a separate thread/process, returning right away to the PHP client session with the allocated job_id

2. during the computation phase:

   • the PHP client session proceeds to periodically check the status of the computationally-intensive job via a synchronous status get_status(job_id) call to the computationally-intensive Thrift server,

   -- or --

   • the PHP client session terminates right away in order to free up precious resources, after passing on the job_id to the browser and also instructing the browser to periodically check the status of the computationally-intensive job job_id (e.g. via META REFRESH, or via an XHR (AJAX) request from Javascript, etc.); the browser check spawns a brief PHP client session which performs the synchronous status get_status(job_id) call to the computationally-intensive Thrift server, terminating immediately after forwarding the status (whichever it may be) on to the browser

Answer 2

I've received an answer on a channel different from Stack Overflow. Since the author gave me permission to post here his answer, I thought it could be useful to someone else in the community.

Hey Robert,

Yeah, this has come up on the Apache lists before. There is no elegant way to do what you're asking for with Thrift. It's fundamentally not designed for bidirectional messaging.

There are hacks around this, such as: - client-side polling - invoking send_method(), waiting on the client side, then recv_method(), instead of just method() - making the client also implement a Thrift server

But obviously none of these are true bi-directional messaging. We've tried to keep the Thrift interfaces as simple as possible and focused on the core RPC use case, which has meant leaving some stuff like this out.

Probably not the answer you were hoping for.

Cheers, mcslee