Over the past few years, the mobile world seems to have gone app wild. The credit, of course goes to Apple with its groundbreaking iPhone and its successful App Store, dominating the market with over 10 million downloads every day. However, over the past year or so, the mobile market is becoming increasingly competitive and complex. Operating systems like Android are gaining popularity enabling 3rd party handset manufacturers such as HTC and Motorola to create Android based devices and compete for market share. And that is before even mentioning tablets…
These extreme dynamics characterizing the mobile market create significant challenges for any company going mobile. The huge diversity of devices, operating systems, OS versions, carriers etc. makes it virtually impossible to sustain reasonable quality standards for their mobile applications and websites across platforms. As Mobile web usage statistics continue to sky rocket and more people start favoring their smartphones for surfing the web, inter-platform compatibility is becoming imperative and carries serious business Implications.
What is mobile application testing really about? How do we verify that an application is ready for launch? The complexity starts with the multitude of handsets having different screen sizes, hardware configurations and image rending capabilities. In addition, the proliferation of operating platforms (iPhone, Android, Blackberry, Windows etc.) and the hundreds of carriers worldwide working on diverse local network standards (GSM, CDMA etc.) is putting further strain on development teams. For instance, the performance of a GPS application may be influenced by bandwidth limitations and loss of data. Similarly, an SMS feature that works in Canada will not necessarily work in France or Asia.
So… testing is becoming more complicated, sustaining quality is becoming impossible and your mobile QA is becoming increasingly expensive…
Common Mobile Testing Practices
The traditional form of testing is manual using a physical device. It is widely accepted that this strategy has serious time to market implications since it is both time-consuming and error prone, often leading to late bug discovery and significant delays in the release of applications.
For small budget testing this is especially problematic since it is very difficult for a test manager to provide an accurate estimation of time, cost and resources for the project. Moreover, creating a testing lab with physical devices can be very expensive, especially when new devices are regularly launched and there is a need to test applications in a multitude of live networks worldwide which means testers have to physical be present in the geography they are testing . Moreover, developers also need to consider data charges (SMS, phone call charges), depending on the nature of the app.
The use of emulators and simulators can provide yet another kind of testing platform. This form of testing is useful but there are some major drawbacks and serious time-to-market implications. Emulators and simulators are merely approximations of devices that can give a false sense that applications support all test requirements like performance, security and fault-tolerance in different geographies and networks. The software version they emulate is not always up-to-date and does not follow regularly the real life firmware versions. In addition they do not offer advanced testing capabilities like automation that can significantly speed up the testing process and save valuable resources.
Remote Device Access (i.e. The Handset Cloud) – Enabling Enterprises to Respond to Mobile Market Dynamics
Today, we are in the midst of a new mobile testing era. Leading enterprises and web companies are shifting away from tedious and time consuming manual testing and emulator based testing methodologies to the use of a handset cloud, an environment to remotely test, troubleshoot and monitor applications on real devices over the Internet, in real time across the major mobile networks in the world. The main benefits are seen in significantly lower testing and development costs as well as a very flexible and agile application development environment that enables companies to securely and quickly respond to business needs and the extremely dynamic mobile market.
With cloud-based platforms, like the Perfecto Mobile Handset Cloud, QA teams no longer have to maintain a large bank of mobile devices for testing their widgets, applications and mobile content. Real devices are connected to the web and can be used to make phone calls, send text messages, down load applications and more. The test team simply chooses what device they want from a list of hundreds of devices physically located around the world and remotely run functional test exercises like regression testing, functional testing, data driven testing, security testing monitoring and more.
Solutions of this kind are designed to meet the needs of globally distributed organizations enabling easy collaboration between distributed teams. For instance, the people writing the test script can reside in England, those running the application tests can be located in the US and the person in charge of debugging the application in Asia. There is no need to transfer devices from one region to another and no threat of devices getting lost in the process.
Delivering Efficiency via Automation
Automation testing tools are without a doubt “the holy grail” of contemporary software testing; allowing testers to automate tedious repetitive test scenarios and generate reports. Add that to the ability to access and operate real mobile devices via the web and you get yourself a powerful mobile testing machine. Scripting specific usage and test scenarios for data driven regression and functional testing, scheduling them, running them in loops, running them across multiple devices, all of these allow a given QA team to accomplish much more with much less resources. Tasks that previously took manual testers days to complete can be replaced by unattended execution of comprehensive automated test scripts on multiple devices.
Using a cloud-based remote device access platform with strong automation capabilities, such as the Perfecto Mobile Handset Cloud, enables organizations to address the ongoing problems of testing costs and complexities in the impossibly dynamic mobile market.