How to identify factors that will help reproduce web rendering bugs for android web browsers?

Question

Recently I've been working on a number of tickets for web rendering bugs that are only seen on a subset of Android devices (or even a single Android device). Reproducing bugs in this category seems to be really difficult -- sometimes I find that I can't do it at all, and worse, sometimes I can on some devices but not on others and I have no idea why.

As an example: one ticket was recently reported by QA as affecting "Galaxy Tab/ Android QA build 4.1." I tried

• running an emulator (both ADT and Genymotion) with 4.1 (couldn't reproduce there)
• two different Galaxy Tab devices running 4.1 (unable to reproduce there)
• Browserstack's Galaxy Tab 2 10.1 emulator (running Android 4.0)... and I saw exactly what the screenshot QA had attached to the ticket showed, successully reproducing the problem.

It's great that I eventually solved that specific bug, but on the other hand, it would have been easy to give up after the first two attempts and never have tried Browserstack. It's half dumb luck that I found a way to reproduce it. I'd like to rely less on that and learn to better (a) identify likely places I can reproduce problems (b) grill QA to give me all the details I need.

What are the relevant variables? I'm thinking of things like:

• Android version
• Browser version
• Maybe kernel version?
• Stock vs custom OS (how big a role can carrier/manufacturer variations play)?
• Screen dimensions/resolution
• Other hardware variations?

But I don't know much about how these interact or what other things may be in play.

Another way of asking this question -- if someone comes to me and says "Hi, I'm seeing a problem on your website when browsing with my Samsum DroidTab?" (or other imprecisely specified device) what exactly do I need to find out in order to find another device/emulator on which I can reliably reproduce what they're seeing?

Answer 1

There is a bunch of what I would call environmental variables. As environmental variables I would define those factors that affects the way an application or device behaves, but are not depending on a device itself.

For instance, the type of Internet connection is an environmental variable. I have been dealing a couple of times with problems regarding Internet connectivity. I did request full information of the device and app, but I was not able to reproduce it. It was not until I decided to try with different connectivities until I reach the conclusion that the user was under a WiFi network that required authorization, and it was thus unable to perform some background operations. When a device is connected to a WiFi network Android deactivates the 3G connection even if the WiFi network does not have connectivity, so I had to solve it with a hack.

Another factor that might have some impact in how an application behaves are device situational issues, such as battery level (Google Glasses does not allow to use voice commands under low battery, or some devices does not allow to take pictures).

Stock vs custom OS implementations are definitely in the list (i.e., Date Picker implementation varies in Samsung Layer).