Fabric Beta versus Apple TestFlight for iOS beta app distribution? [closed]

Question

What are the advantages/disadvantages of using Fabric's Beta kit versus Apple's TestFlight service for beta iOS app distribution? When and why would you choose one over the other?

Answer 1

I've spent some time working with both services and here's what I've found:

Summary

• Use Fabric Beta for distributing your alpha version to a small group of testers.
• Use Apple TestFlight for distributing your beta version to a larger group of testers.

Fabric Beta Advantages

Distribution of your app to existing testers is quick and easy. You can release more often and get changes/fixes into your testers' hands sooner. You don't need to upload your app to iTunes, submit it to Beta Review, and wait for approval.

If you're already using Fabric's Answers Kit to track events, adding the kit is simple.

Fabric Beta Disadvantages

Onboarding a new tester is painful. Here is the general flow:

1. Add a new tester via the Fabric Mac App using their email.
2. The new tester receives an email inviting them to test the app.
3. The new tester accepts the invitation.
4. You get a notification email that they have accepted.
5. The new tester is prompted to install the Crashlytics iOS App.
6. After they install, you get a notification email with their device info (UUID).
7. You download the device info and upload it to iTunes Connect.
8. You open xCode and re-download your provisioning profile with the new device info.
9. You re-archive your app.
10. You re-distribute your app to this tester.

If you're recruiting more than just a few testers, this gets out of hand quickly.

You end up with a bunch of builds with the same build number. The code is the same, just the provisioning profile contains different device info. Tracking which testers are on which build becomes tedious.

The 2-step process creates a road block. The tester has to wait to download your app after they accept the invitation. You need to hold their hand a bit more and make sure they understand the process.

Installing the Crashlytics App can be a road block. It opens settings and asks them for permission to access their device info. Testers that you don't personally know may be wary of this and not make it past step 4. Then you may find yourself in a vortex of emails trying to convince them to trust you, that it's Crashlytics asking for permission and not you, that Crashlytics is trustworthy, and on and on...

Apple TestFlight Advantages

Onboarding a new user is pretty simple. The process is as follows:

1. Add a new tester via iTunes Connect using their email.
2. The new tester receives an email inviting them to test the app. This email prompts them to download the TestFlight app and gives them a redeem code to get your app.
3. The new tester downloads the TestFlight app.
4. The new user enters the redeem code.
5. The new tester downloads your app.

As you can see, there aren't any significant roadblocks between the invitation email and the tester downloading your app.

If a previous version of your app has been approved by Beta Review, a subsequent release that does not contain major changes does not need to be reviewed.

Apple TestFlight Disadvantages

You have to upload an archive of your app to iTunes Connect before releasing it. After uploading, your build enters the 'processing' state. Processing can take anywhere from a few minutes to forever.

Your app must be submitted to Beta Review and approved before you can distribute it. The review process can take anywhere from a couple hours to a week (according to anecdotes I've read).

Conclusion

Fabric Beta seems like the right choice if you're working with a small number of testers who you know personally and will be actively engaged in the testing process. After the painful onboarding process, you can push new versions and even new app projects to them very easily. It's great for getting your alpha version into the hands of a few.

Apple TestFlight is superior when you want to take your testing to the next level. If you're recruiting a large pool of people to test your beta app, TestFlight's more friendly onboarding process makes it the better choice. At this point your app should be almost fully functional, so getting it through Beta Review shouldn't be a problem.

Regardless of which service makes the most sense for you, don't forget to look at Fabric Answers (or a similar service) for tracking events. Direct feedback from testers is incredibly helpful, but collecting data on their in-app behaviors can definitely shed light on other important things.