How to implement a limited feature rollout (language agnostic) to your users?

Question

I would like to know some common or best practices of rolling out a new website feature to a select group of the userbase.

The users could be, for example, based solely on a percentage of your overall user base (10%). The rollout should be customizable (configurable) and support any number of features. It would also be useful to associate rollouts to specific user roles or privileges (ACL).

So, in essence, what is an architecture that would scale reasonably well?

As for the language agnostic portion, you can either provide pseudo-code, general concepts or ideas, or snippets from your preferred language to get your point across.

Links to any examples or tutorials are welcome.

Answer 1

My recommendation would be that, for the people who are getting the new feature, the site they are on should be as close as possible to the site that everyone will be on once the feature is public.

In other words, I wouldn't have, for example, conditional logic on a page to determine whether a button should be visible or not, if that condition would only exist during this beta period. Rather, I would determine at login time whether this person is a beta participant or not (that determination might be random, might reference their role, etc). If they are a participant, they would be redirected to a separately deployed site that is deployed from a version control branch.

Once the rollout is complete, the branch is merged in and everyone sees the new feature.

This way, you don't have to worry that the codebase the public beta users are seeing is different (in some way that could break something) from the eventual release codebase.

Answer 2

At my last job we accomplished this using the load balancer and a current revision cookie.

The load balancer set a cookie identifying the revision number of the instance that the user was using. If that cookie was already present, the load balancer would just send that request to a running instance with the corresponding revision. When we deployed a new revision, the load balancer continued to send traffic with an existing revision cookie to their original revision until that revision was no longer running or the user closed their browser. New traffic would get sent to the newly deployed revision. This allowed us to test out changes for a bit while our existing users were continuing to run on the old revision. We could also manually set that cookie to test out the new rev internally on the production environment before turning new traffic onto it. When we were comfortable that the new revision had no major problems we could bring down the old revision and all traffic would start going to the latest revision.

That setup does not support backwards incompatible database changes though. There's pretty much no way to do it where you can have part of your users on one db schema and part on another, and be able to take writes to both and then somehow merge those writes together after you've decided the new rev is ok. I mean, it's possible but it really depends on what the schema changes are exactly and how they effect your app, so you can't do it in a reliable, agnostic way on deployment. For us, we just generally tried not to do backwards incompatible schema changes. If we really did need to, we'd try to postpone the destructive part (dropping a column or table) to a later revision where we could migrate the schema and have both versions running with no adverse effect on current users.