JQM grades Android 2.1-2.3, 3.2, 4.0, and 4.1 at an A level. But, don't the devices have a huge impact on that? I have my application working well with iPhone 4 hardware and higher (any OS 4 and higher). But, as I start to get my hands on some Android devices I'm seeing alot of things not working. HTC (Verizon on Wifi - Sense 2.1, Android 2.3.4)
The click delay doesn't seem to work so scrolling the window activates everything (I give the device a 6 out of 10 on scrolling capability while testing on the JQM demo site)
With a list inside of a collapsible, I have to click about a full thumb width above the list item or it clicks both of them (alot of times I can get the ui-active class added but it doesn't actually fire the anchor - like the click doesn't delegate correctly unless it's from the inner most element or something)
The rounded corners and box shadows look atrocious
Sometimes when an input has focus it doesn't actually popup the keyboard - I have to click it again
Samsung Galaxy S3, Android 4.0.4
When expanding collapsibles, everything below the collapsible heading and body doesn't get rendered correctly. Reminds me of uncleared floats. Tapping anywhere on the screen repaints and fixes.
Popups don't work for me at all (thought they work on the jqm demo site) so it must have something to do with the fact that I've CSS centered dialogs rather than letting JQM deal with it. The popups are there because it lets me interact with them they are just invisible!
There are a bunch more things here and there that I could nitpick. And my real questions are: 1. How do you really test a decent variety of Android devices? 2. Should some of these really have A grades considering that they barely work?
I guess I would say it really depends on what you are trying to do. You have to keep in mind they most likely test using their documentation. I doubt they do any customization in their testing, since it is just a JQM test/grade.
Also, I would agree devices do matter. It's really hard to test everything... so the OS of the device does make sense as a testing point. But yes, you are correct here.
As for the S3, I have done testing on this device and have not seen any of the issues you are talking about. There might be something wrong on your end.
In the end, again it really depends on what you are doing with JQM. It's really hard to grade older devices, and I don't really know JQM criteria for 'A' grade since I have seen some issues with devices in the 'A' range.
The collapsible thing is probably reproducible. But, it only happens when you have an inset listview inside of a collapsible. I'll do a bit more on this one and see if I can figure it out.
The popup is working on the test page I created even with a fixed position footer. I'm going to have to dig into this one for awhile. I thought it might be related to some of my reset css or CSS specific to popups. But, I removed all of that and still the same.
For the main question, I guess it's really open ended as to how people certify that their applications work well across alot of Android devices. It's hard to say something is an A grade when you can barely navigate from page to page because the scrolling and transitions are so choppy. You obviously can't please everyone.
I guess in general I'll just shoot to make my applications work as well as the demo pages. ;)