Actor-based webservice - How to do it properly?

Question

In the past few months, me and my colleagues have successfully built a server-side system for dispatching push notifications to iPhone devices. Basically, a user registers for these notifications via a RESTful webservice (Spray-Server, recently updated to use Spray-can as the HTTP layer), and the logic schedules one or multiple messages for dispatch in the future, using Akka's scheduler.

This system, as we built it, simply works: it can handle hundreds, maybe even thousands of HTTP requests a second, and can send out notifications at a rate of 23,000 per second - possibly even more if we reduce log output, add multiple notification sender actors (and thus more connections with Apple), and there might be some optimization to be done in the Java library we use (java-apns).

This question is about how to do it Right(tm). My colleague, much more knowledgeable about Scala and actor-based systems in general, noted how the application isn't a 'pure' actor-based system - and he's right. What I'm wondering now is how to do it Right.

At the moment, we have a single Spray HttpService actor, not subclassed, that is initialized with a set of directives that outlines our HTTP service logic. Currently, very much simplified, we have directives like this:

post {
  content(as[SomeBusinessObject]) { businessObject => request =>
    // store the business object in a MongoDB back-end and wait for the ID to be
    // returned; we want to send this back to the user.
    val businessObjectId = persister !! new PersistSchedule(businessObject)
    request.complete("/businessObject/%s".format(businessObjectId))
  }
}

Now, if I get this right, 'waiting for a response' from an actor is a no-no in actor-based programming (plus the !! is deprecated). What I believe is the 'correct' way to do it is to pass the request object over to the persister actor in a message, and have it call request.complete as soon as it's received a generated ID from the back-end.

I have rewritten one of the routes in my application to do just this; in the message that is sent to the actor, the request object / reference is also sent. This seems to work like it's supposed to:

  content(as[SomeBusinessObject]) { businessObject => request =>
    persister ! new PersistSchedule(request, businessObject)
  }

My main concern here is that we seem to pass the request object to the 'business logic', in this case the persister. The persister now gets additional responsibility, i.e. call request.complete, and knowledge about what system it runs in, i.e. that it's part of a webservice.

What would be the correct way to handle a situation like this, so that the persister actor becomes unaware of it being part of a http service, and doesn't need to know how to output the generated ID?

I'm thinking that the request should still be passed to the persister actor, but instead of the persister actor calling request.complete, it sends a message back to the HttpService actor (a SchedulePersisted(request, businessObjectId) message), which simply calls request.complete("/businessObject/%s".format(businessObjectId)). Basically:

def receive = {
  case SchedulePersisted(request, businessObjectId) =>
    request.complete("/businessObject/%s".format(businessObjectId))
}

val directives = post {
  content(as[SomeBusinessObject]) { businessObject => request =>
    persister ! new PersistSchedule(request, businessObject)
  }
}

Am I on the right track with this approach?

A smaller secondary spray-server specific question, is it okay to subclass HttpService and override the receive method, or will I break things that way? (I have no clue about subclassing actors, or how to pass unrecognized messages to the 'parent' actor)

Final question, is passing the request object / reference around in actor messages that may pass throughout the entire application an okay approach, or is there a better way to 'remember' what request should be sent a response after flowing the request through the application?

Answer 1

In regards to your first question, yes, you are on the right track. (Although I would also like to see some alternative ways to handle this sort of issue).

One suggestion I have is to insulate the persister actor from knowing about requests at all. You can pass the request as an Any type. Your matcher in your service code can automagically cast the cookie back into a Request.

case class SchedulePersisted(businessObjectId: String, cookie: Any)

// in your actor
override def receive = super.receive orElse {
  case SchedulePersisted(businessObjectId, request: Request) =>
    request.complete("/businessObject/%s".format(businessObjectId))
}

In regards to your second question, actor classes are really no different than regular classes. But you do need to make sure you call the superclass's receive method, so that it can handle its own messages. I had some other ways of doing this in my original answer, but I think I prefer chaining partial functions like this:

class SpecialHttpService extends HttpService {
  override def receive = super.receive orElse {
    case SpecialMessage(x) =>
      // handle special message
  }
}