Retry / replay of failed messages in AKKA

Question

I'm using AKKA.NET in my current .NET project.

My question is this: How are experienced AKKA-developers implementing the replay-message-on-failure pattern using the latest AKKA libraries for either Java or .NET?

Here are some more details.

I want to ensure that a failed message (i.e. a message received by an actor leading to an exception) is replayed / retried a number of times with a time interval between each. Normally the actor is restarted by the failed message is thrown away.

I have written my own small helper method like this to solve it:

   public void WithRetries(int noSeconds, IUntypedActorContext context, IActorRef receiver, IActorRef sender, Object message, Action action)
    {
            try
            {
                action();
            }
            catch (Exception e)
            {
                context.System.Scheduler.ScheduleTellOnce(new TimeSpan(0, 0, noSeconds), receiver, message, sender);
                throw;
            }
        }
    }

Now my actors typically look like this:

 Receive<SomeMessage>(msg =>
        {

            ActorHelper.Instance.WithRetries(-1, Context, Self, Sender, msg, () =>     {
              ...here comes the actual message processing  
            });
        });

I like the above solution because it is straightforward. However, I don't like that it adds yet another layer of indirection in my code, and the code gets a bit more messy if I use this helper method in many places. Furthermore it has some limitations. First of all, the number of retries is not governed by the helper method. It is governed by the supervision strategy of the supervisor, which I believe is messy. Furthermore, the time interval is fixed whereas I would in some cases like a time interval that increases for each retry.

I would prefer something that can be configured using HOCON. Or something that can be applied as a cross-concern.

I can see various suggestions for either AKKA for Scala, AKKA for Java and AKKA.NET. I have seen examples with routers, examples with Circuit Breaker (e.g. http://getakka.net/docs/CircuitBreaker#examples) and so forth. I have also seen some examples using the same idea as above. But I have a feeling that it should be even simpler. Perhaps it involves some usage of AKKA Persistence and events.

So to repeat my question: How are experienced AKKA-developers implementing the replay-message-on-failure pattern using the latest AKKA libraries for either Java or .NET?

Answer 1

I looked into this last year sometime - I'm away from my dev machine so cannot check,so this is all coming from memory:

I seem to remember the solution to this was a combination of stashing and supervision strategies and lifecycle hooks :)

I think you can wrap your child actor code in a try-catch, then in the case of error, stash the message and re-throw the exception so it is handled by the supervisor and all the usual supervision strategies come into play. I think you would resume rather than restart. Then in the appropriate lifecycle message (onresume?!) unstash messages which should mean the failed message is processed again.

Now this isn't all that different from what you've already posted above, so hopefully someone has a better solution :)