How to do simple AB testing in iOS

Question

I am looking to split up my user base to 10 group and show 10 different UI and see how they feel about it.

so each user group will have single type of UI always.

i.e Let's say I have 10k users and when I roll out my next release when user install I will be showing for 1000 user 1 UI and for another 1000 user 1 UI like all 10K users.

I know this can be done with the help of AB testing framework.

Basically I want to call one API at the launch of app and it has to return value between 1 to 10 then I can store it in my keychain and next time when app is launched I will see if it's already there in keychain and I will not call the API.

So basically the API will know how many requests has come and it'll divide and send right values back

so based on value in keychain I will show different , different UI and here AB testing framework's job would be giving me value 1 to 10 the API part.

There are so many AB testing framework available online.But I couldn't find any framework that suits my needs.

any help is appreciated !

Answer 1

I would like to attribute the credit of this answer to this post as he has pointed out FireBase Remote Config and A/B testing.

As questioner asked I will explain the steps involved in that to achieve it.

Configuration on server

• Visit https://console.firebase.google.com/ and sign in with your google account.

• Choose Create project and Click iOS

• Key in app id and nick name and click register app

• It'll show a link to GoogleService-Info.plist download then drag & drop it in the project

• Choose Next

• It'll show you Run your app to verify installation you can choose skip this step

• Choose remote config from the landing page

• Choose Add variable and enter a variable name of your choice but I enter ABTestVariationType and leave value empty and choose Publish changes

• Choose A/B testing from the side bar then click Create Experiment then Choose Remote config

• In the upcoming pop up Enter the name of your choice I enter as A/B test POC enter some description about it and that's optional anyway

• In the the target users choose your app id and in the percentage of target users choose 100% and click Next then it'll show the variants section

• In the variants section there will be a general category named Control group with 50% loaded by default and a variant box with 50% filled in and empty box and you can enter any name in that but I would enter variant 2.Now click add a parameter 8 times now you can see each variant has 10% and name all the variants and I would name variant 3,variant 4 to variant 10.

• In the same variants section click Add Parameter from Remote config
  Now you can see the a box appearing besides each variation parameter.You can enter unique value to identify each flavour.I would enter value 1 for the first variant and 2 for the second variant like that I will finish up with value 10 for the last variant and click Next

• Then goal section appears you can choose one of it but I would choose Retention(15+) days and click Review and click start experiment and in prompt that's appearing choose start again

Integrating in the app

1. Add the following pods in your project

   pod 'Firebase/Core'
   
   pod 'Firebase/RemoteConfig'
   

2. Drag and drop the GoogleService-Info.plist that was downloaded during the server configuration

3. Initiate the firebase with following boiler-plate code

     @import Firebase;
   
     -(BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions(NSDictionary *)launchOptions 
    {
   
      [FIRApp configure];
   
      return YES;
   
    }
   

4.Have the class RcValues which is another boiler-plate code in your project

     #import "RcValues.h"  

     @import Firebase;

     @implementation RcValues

     +(RcValues *)sharedInstance 
     {

      static RcValues *sharedInstance = nil;

      static dispatch_once_t onceToken;

      dispatch_once(&onceToken, ^{

        sharedInstance = [[RcValues alloc] init];
      });

       return sharedInstance; 

     }

   -(id)init{

     self=[super init];

     if(self)
     {

      [self AcivateDebugMode];

      [self LoadDefaultValues];

      [self FetchCloudValues];

     }

     return self;

    }

   -(void)LoadDefaultValues    
   { 

    [FIRRemoteConfig.remoteConfigsetDefaults:
    @{@"appPrimaryColor":@"#FBB03B"}]; 

   }

   -(void)FetchCloudValues
   {

     NSTimeInterval fetchInterval=0;

    [FIRRemoteConfig.remoteConfigfetchWithExpirationDuration:
    fetchInterval completionHandler:^(FIRRemoteConfigFetchStatus 
    status, NSError *_Nullable error) 

     {

       NSLog(@"error:%@",error);

      [FIRRemoteConfig.remoteConfig activateFetched];

      }]; 
    }

  -(void)AcivateDebugMode{ //    

   FIRRemoteConfig.remoteConfig.configSettings=debugSettings;

   FIRRemoteConfigSettings *config = [[FIRRemoteConfigSettings alloc] initWithDeveloperModeEnabled:YES];

   FIRRemoteConfig.remoteConfig.configSettings=config;
  }

  @end

5.Invoke the class in appdelegate didFinishinglaunchoptions

    RcValues *Obj=[RcValues sharedInstance]; 

This will download the keyvalue for ABtesting

6.Use the below code to get the AB testing key from firebase to your app

    self.flavourNumber.text=[FIRRemoteConfig.remoteConfig
    configValueForKey:@"ABTestVariationType"].stringValue;

Based on the key value you can show different UI as you wish.

Firebase will take care of sending the right value and you don't have to worry about divide the users into groups by yourself.

P.S

Please follow the below tutorials for more detailed info this is just a summary and I will try to summarise or add more pictures when I have free time to make it for easier understand if possible I will try to add sample project in github and link it here

firebase-tutorial-ios-ab-testing

firebase-remote-config-tutorial-for-ios