Anyone involved in software testing and software test automation should know the test automation pyramid introduced by Mike Cohn.
I will expand on this subject in this post, which contains excerpts from my book “Hands-On Mobile App Testing” published with Pearson Education.
The typical testing pyramid consists of three layers, as you can see in the image below. At the bottom, there’s the automated unit-testing layer; in the middle sits the automated integration testing layer, and at the top there’s the automated end-to-end testing layer (including the user interface tests). Each layer has a different size, indicating the number of tests that should be written within each stage. Manual testing is not part of the test pyramid, hence it is shown as a cloud for additional testing work.
But this pyramid is not applicable to mobile apps and mobile test automation.
Mobile testing requires a totally different set of testing activities like movement, sensors, and different devices and networks, as compared to other software like desktop or web applications. Lots of manual testing is required to make sure that a mobile app is working as expected in different and challenging usage scenarios.
Mobile test automation tools are currently not as mature as their counterparts for web and desktop apps, which leads to a flipped test automation pyramid (see the image below). As the tools become increasingly mature, this pyramid is likely to flip back to normal because the default test automation pyramid is based on a more stable foundation. So for now, the default pyramid can’t be used as an indicator for test automation and manual testing in the mobile world.
The flipped testing pyramid looks like this:
In this version of the pyramid, the automated unit testing layer is the smallest one. This is because not every unit or method of mobile apps can be tested in an isolated manner. In some cases, different APIs, layers or systems may need to be faked or mocked in order to get the small unit to work. This is also the case for every other software application, but mocking or faking other systems for mobile apps is much more complex. This is not efficient from a technical or economic point of view. However, it’s no excuse for not writing mobile unit tests at all. The business logic of an app must be tested at unit level.
The next stage is the end-to-end test automation layer. Within this layer, the whole app is tested from a user perspective. It will be tested to make sure the whole system is working, from the app’s user interface through to the backend system via a wireless network, including integration testing with different libraries or APIs. The integration testing layer is therefore part of the end-to-end layer.
The biggest change in this pyramid is that manual testing is part of it. Mobile testing requires lots of manual testing, and this can’t be replaced by test automation or any other tools yet.
Nevertheless, mobile test automation is an extremely important topic and every mobile tester should be able to write automated regression tests that provide fast feedback about the current quality state of an app. Furthermore, test automation helps the team build a reliable and robust mobile app that makes the customers happy.
The Mobile Test Pyramid
But the problem with the flipped testing pyramid is it has no stable foundation and mobile testing requires lots of manual testing. This is why I created my own mobile test pyramid consisting of four layers including manual and automated steps (see below). The biggest layer of the pyramid is manual testing and forms the strong foundation for every mobile app project, followed by end-to-end testing, beta testing and a top layer comprising unit testing.
The grey parts of the pyramid indicate the automated steps and the white parts are the manual testing steps. The beta-testing layer is new to the pyramid but essential to every mobile app project. Keeping the high expectations of mobile users in mind requires this layer to be part of every mobile project to get early feedback from your mobile customers. You can either use a crowd testing approach for your beta testing or you can ask your colleagues to beta test early versions of your app to provide important feedback.
Note: The End-2-End and Unit tests layers can also be swapped. The amount of End-2-End tests and Unit tests can differ from project to project and from app to app.
I used this mobile test pyramid in several projects and it helped me set up a reliable, effective and valuable testing process.
If you want to read more about mobile testing, please have a look at my book “Hands-On Mobile App Testing” which is available as a printed copy as well as an eBook.
Daniel Knott is a well-known blogger and speaker, and the author of the book “Hands-On Mobile App Testing.” He has been working in the field of software development and software testing since 2003.