All companies making the transition to digital share one common goal: release high-quality mobile apps more quickly. Whether they’re introducing new app functionality, fixing a production issue or supporting a new operating system, time (or the lack of it) is always a factor. If you’re relying on manual testing over test automation, get ready to lose the race against time.
The consequences can be rough if you don’t deliver a good working app in a timely manner. Think: disgruntled users complaining about your buggy app on app stores and social media. Upper management viewing developer and QA teams as incompetent or, worse, expendable.
It’s no secret that manual testing is a major factor in slow releases. Teams that are still testing their mobile apps manually are faced with the following limitations:
1) Less test coverage
2) Less platform coverage
3) An error-prone process
4) Inefficient releases due to slow feedback
All of the above slow down the time it takes app delivery teams to respond to market changes.
With that in mind, it’s important to note that even though test automation is a critical component of any mobile app SDLC, there will still be tests that should not be automated. Such tests are related to the unique user experience (validating the colors, look and feel of an app), usability tests or corner case tests (tests for relevant but infrequently-used features).
There are many advantages of test automation, but let’s focus on three that will most benefit the organizations implementing them.
Fast feedback loop and quicker time to market
When you consider that many organizations are moving to an Agile workflow, you can see how test automation allows teams to work more closely and efficiently. When test teams can execute many functional tests on multiple devices in just hours, the feedback to developers flows fast. Developers can then fix bugs and return code quickly to the testing team.
Test automation done right will be integrated into a CI (continuous integration) workflow that can be triggered upon each code commit. The alternative is a developer making a code change, building the mobile app and sending it to the testing team. They then do manual testing on multiple devices. This takes “ages” in today’s agile world.
Lastly, manual testers may interpret a test flow differently or fail to understand the test’s setup or dependencies. This will result in the wrong test execution and false positives.
Fast feedback must be part of the SDLC.
The only way to address market changes such as this month’s iOS 10/iPhone 7 announcement and others like it is through faster release cycles. The enabler for that acceleration is test automation that can quickly validate changes and asses the stability of the new code.
Increase coverage with test automation
Manual testing allows each tester to test on one platform at a time and only do one single test. On the other hand, test automation allows teams to scale up their suite and run tests in parallel on multiple devices and operating systems.
Test automation also operates like a machine, executing tests at scale. For example, there are tests that perform 100 login scenarios one after the other or in parallel on 10 devices. It would make no sense to assign this to a manual tester — it’s simply too time-consuming. Automation also enhances test coverage by executing more and different tests — such as functional tests, performance and stress tests, unit tests and others — on real devices.
From a platform coverage perspective, test automation allows a single test suite to be launched against various devices and OS versions at once, increasing the coverage as well as your team’s confidence in the quality of the app. For companies that have customers in various geographies and have to test on different sets of devices/OSes, this is a huge advantage and time saver.
Keep up with the market
The mobile market is always being redefined by new mobile devices, operating systems and APIs. All of these changes have a direct impact on existing tests being executed on real devices. A test running on Android Marshmallow (6.0) will be different on Android Nougat (7.0). Nougat has new features like Split Window and Doze Mode, causing the tester to check that the app works with these new features.
With a good automation test suite in place, re-directing tests against new devices or enhancing a few lines in the test code will aid support for new platforms and features, compared to having manual testers deal with new changes one by one.
Bottom line: Increase automation now
While not every test can or should be automated, digital teams should not rely on manual testing for quality. They should make the shift to test automation with the goal of reaching 70-80% automation coverage. There are a variety of commercial and open-source tools that are up-to-date with mobile platforms. Teams need to take advantage of these tools and grow into them. It’s also worth noting that automation is not a one-off activity. It requires continuous monitoring, maintenance and debugging to keep running against every build, code commit and regression cycle.