As mobile grows and enters the enterprises, banks, social and other area’s, the users are expecting on top of the performance, and usability which are unique in mobile to not see crashes, or receive error messages such as “This application is not supported on your device”, “This application does not support this current OS version” etc.
To assure that your users will not run into such kind of issues it is a must to implement a mobile testing strategy in your organization, and leverage existing mobile testing tools (Preferable choice would be to chose tools which integrates to your existing ALM tools and provide automation capabilities which is a MUST), cloud based solutions and so forth.
Testing just few devices, which are not representing any mobile family, or are not key “players” in the market is one of the root causes to not detecting relevant bugs during the life cycle. Other problems we often see is the fact that the product management does not clearly define the supported mobile platforms and mobile OS versions (Minimum version to support is critical – e.g. Android 2.3.3 as initial supported platform).
On top of the above – It is important to define the handsets to be tested per the current version in a meaningful way (we in PerfectoMobile recommend to define 3 sets of devices or categories – “Must” devices (4-8) , “Major” Devices (8-16) and “Market” Devices (30-40) –> The market devices usually can be covered using cloud based solutions )
See an example to Bloomberg mobile application which recently launched an update to the App Store due to crashes in older iPhone devices (3G, 3GS) and iPod devices (If these were not to be supported, the application while being installed should have notified the users, but definitely not crash.
Another good example below shows how a similar web page is differently shown on 2 Android devices
You can see in both images BTW that the page does not cover the entire screen size, which means that the portability to these devices is wrong.
Last, look into the 2 illustrations below which shows the % of crashes reported across the various iOS and Android versions (Showing the fragmentation in the mobile market).
For questions about mobile testing methodologies, tools and E2E ALM mobile solutions, feel free to contact me