Testing for the masses with only one phone and emulator

Question

I have a new android app I put on the marketplace a few days ago. I did quite a bit of testing on my moto droid before publishing it and tried to be very thorough. Well, I got some negative comments back on how it didnt run right on someone's EVO 2.2 or X10 (didnt even know what that was at first). Well, it runs perfect on my phone.

Any advice on how I'm supposed to fix or support an app that runs on like 100 different phones?

How are other developers approaching this? (without actually buying every phone out there). Thanks.

Answer 1

I don't think there is a silver bullet for avoiding these kind of problems. A couple of guidelines/suggestions:

Preventing problems:

• Use only publicly documented APIs. If you depend on implementation details, things are likely to fail on different phones.
• Follow the guidelines for supporting multiple screens, and make sure to test each of the combinations of screen sizes and densities in the emulator.
• If you are using lots of openGL, research which extensions are supported by which phones. This is where a largest number of the problems tend to be.
• Recruit your friends to be beta testers. You can send them an apk to load before publishing on the market.

Diagnosing problems:

• Starting with Android 2.2, you will be able to get error reports back from users, which should help in quickly diagnosing problems: Android Application Error Reports
• You can also implement something like remote stack trace to get error messages back in the meantime.

Answer 2

I think a good approach is making sure your logging all your error's and receiving them from the users remotely. This post has a good solution : How do I obtain crash-data from my Android application?

Answer 3

One option that I am considering is some kind of public beta. Provide a free version on the market, clearly marked as a beta, with instructions to e-mail details of the device used and any problems encountered. If you intend to charge for the final version you'll want to make the beta version time-limited or restricted in some other way. You can delete the beta version from the market, and with it any negative feedback, when you release the full version.

An alternative to doing this on the official market is to upload to the pre-release area of AndAppStore.

As for the X10, in my limited experience this phone seems to crop up quite frequently when it comes to complaints of apps not working, particularly if it relates to sound. Sony-Ericsson have released an X10 add-on for the Android SDK, so that you can make your emulator look like an X10, though I'm not sure how much difference it makes to its behaviour.

Answer 4

Or use a testing community, so people with different devices can give actually test your app, and provide feedback.