How does a solo/small group of developers test on multiple devices?

Question

First off, sorry if this is too subjective, I just didnt know how else/where to ask.

Anyway, in the light of all my recent questions, I'm getting ready to release an Android app soon, and most of the testing has been done on my phone, the Droid. I really dont have the money to test on "multiple" devices, nor do I know anyone with an older phone that I could ask for help that would possibly get any kind of bug. Not to mention, when I do get a bug report, how would I go about fixing it for that particular phone without having to buy it to make sure it actually gets fixed, or that the person didnt just came across a one-time freakish accident of a glitch?

How do you guys solve these kinds of issues?

Answer 1

You can test the vast majority of issues via the emulator:

Check out this data on platform versions and screen sizes to get an idea of what configurations you should test for.

Based on that data, I'd test at least one configuration with API versions 1.5, 1.6 and 2.1, and versions with medium and high density resolutions.

If you wanted to test physical devices, I would guess that the G1 and the Droid would be the top two... G1 would give you the lower API versions, and Droid would give you the 2.1.

Depending on your application that may be sufficient. Applications that make heavy use of OpenGL extensions might need to test further, since that is the area where there is the most difference from device to device. I don't think that the emulator is sufficient for that. See this thread on the differences.

Other than that, I would just send out a demo version of the app to a few friends or an appropriate forum. If you find problems once you launch, collecting log data from users having problems can be very helpful. I wouldn't worry too much about device specific problems though, I don't think they are that common.

Answer 2

Disclaimer: I'm a Motorola employee in our developer services team. I don't speak for other OEMs.

Cover the range of devices that are enumerated in the "supports-screen" manifest element. Also, take into consideration when compatibility is mode on and off. Screen sizes and market filters seem to me to be the biggest things that trip developers up. Some of this you can test with the emulator and others you need real hardware.

OEMs provide SDK "addons" that allow you to run emulator images with the skin and screen size/density of their devices. Download addons from the OEM's developer sites. Motorola's addons are available at developer.motorola.com. HTC and Samsung do the same.

A commercial alternative is Mob4Hire. They have real people on real networks who can test your app for you.

Good luck

Answer 3

I have few friends wich have different android devices. Before app publishing I give it to them for tests. Sometimes any users submit bugreports to market, sometimes sends it to you by email. There is impossible to have all android devices and test own app on it. This is ok.

Answer 4

It might be worth having a look somewhere specialist such as http://www.xda-developers.com/

They've got a sizeable community there of reasonably knowledgeable people and its not uncommon to see people posting betas of apps there for consumption and feedback. There are also dedicated subforums for each phone which may assist when attempting to resolve problems on certain handsets.