Blue/Green Deployments with Azure ServiceFabric

Question

I'm currently building an application using the ReliableActors framework on Azure ServiceFabric. As we scale up, I'm looking at doing blue/green deployments. I can see how to do this using a stateless system. Is there's a way to do this using statefull actors?

Answer 1

Service Fabric is all about rolling upgrades, rather than deployment swaps, like a VIP swap. Both stateless and stateful services are upgraded the same way, but there are a few additional nuances to stateful that I'll mention later.

By rolling upgrades, I mean upgrades to an application are done in place, one upgrade domain at a time, so that there is no downtime and no sudden switch. A rolling upgrade in Service Fabric can be done in a safe "managed" mode where the platform will perform health checks before moving on to the next upgrade domain, and will automatically roll back if health checks fail.

OK, that all sounds nice. But how do you do blue/green deployments when upgrades are always rolling upgrades?

This where application types and version come in. Instead of having two "environments" that can hold two running applications, Service Fabric has this concept of versioned application types from which application instances can be created. Here's an example of how this works:

Let's say I want to make an application called Foo. My Foo application is defined as an application type, call it FooType. This is similar to defining a class in C#. And like class in C#, I can create instances of my type. Each instance has a unique name, similar to how each object instance of a class has a unique variable name. But unlike classes in C#, my FooType has a version number. Then I can "register" the application type and version in my cluster:

FooType 1.0

With that registered, I can create an instance of that application:

"fabric:/FooApp" of FooType 1.0

Now, let's say I develop version 2.0 of my application. So I register version 2.0 of my FooType in the cluster:

FooType 1.0
FooType 2.0

Now I have both versions of FooType registered, and I still have an instance of 1.0 running:

"fabric:/FooApp" of FooType 1.0

Here's where it gets fun. I can do some interesting things:

I can take "fabric:/FooApp" - an instance of FooType 1.0 - and upgrade it to FooType 2.0. This will be a rolling upgrade of that running application.

Or.. I can leave "fabric:/FooApp" alone, and create a new instance of my version 2.0 application:

"fabric:/FooApp" of FooType 1.0
"fabric:/FooAppv2Test" of FooType 2.0

Now I have two applications, running side-by-side, in the same cluster. One is an instance of 1.0, and the other is an instance of 2.0. With some configuring of ports and application endpoints, I can ensure users are still going to the 1.0 instance while I test out the 2.0 instance.

Great, so all my tests pass against the 2.0 instance, so now I can safely take the 1.0 instance and upgrade it to 2.0 of FooType. Again, this is a rolling upgrade of that instance (fabric:/FooApp), it's not migrating users to the new instance (fabric:/FooAppv2Test). Later I'll go and delete fabric:/FooAppv2Test because that was just for testing.

One of the benefits of blue/green though is being able to swap back to the other deployment if the new one fails. Well, you still have both 1.0 and 2.0 of FooType registered. So if your application started misbehaving after the upgrade from 1.0 to 2.0, you can just "upgrade" it back to 1.0! In fact, you can "upgrade" an application instance between as many different versions of its application type as you want! And you don't need to have instances of all your application versions running like you do in a swapping environment, you just have the different versions registered and a single application instance that can "upgrade" between versions.

I mentioned caveats with stateful services. The big thing to remember with stateful services is that the application state - your users' data - is contained in the application instance (fabric:/FooApp), so for your users to see their data you need to keep them on that instance. That's why we do rolling upgrades instead of deployment swaps.

This is just the basic idea. There are other ways you can play around with application types, versions, and instances depending on what your goals are and how your application works, but that's for another time.