Running XCUITest in Parallel with Quantum on Perfecto Cloud
If you're an iOS developer, you're probably familiar with Apple XCTest and XCUITest automation framework, which came with Xcode to allow you to easily develop UI tests that reflect the users’ interface with the application.
Since Xcode 8, Apple has deprecated UIAutomation in favor of the XCUITest framework. XCTest has been part of Xcode for a few years now, but it is finally catching up and more developers are getting on the bandwagon. XCTest and XCUITest provide feature-rich capabilities for iOS developers and test automation engineers that aim to implement different levels of tests using Xcode features and supported programming languages, like Objective-C and Swift.
One clear benefit of using XCTest/XCUITest is that this framework bundles the tests on the client side (the device under test), therefore the tests run faster. Additional advantages of this method are reduction of network traffic and greater control over the classes and controller while other open-source frameworks like Appium has multiple layers that need to be dealt with.
Of course XCUITest cases can only be executed on a single device at a time, so if you want to run these tests in parallel, you will not be able to do this out of the box.
In this post, I’ll reveal a way that will allow you to overcome that obstacle!
We will be able to set up XCUITest to run on multiple devices and in parallel with the help of an open-source Quantum Framework on the Perfecto cloud devices.
Before I explain how easily you can do this, let me explain the nature of the Quantum automation Framework (QAF). QAF is an open-source test automation framework for web and mobile test authoring, execution and analysis. The framework enables its users to easily set up their environment to access the Perfecto Lab in the cloud, with user authentication and driver management. It allows you to create test scripts in BDD that are Gherkin-based or written in Java, and it will generate Perfecto DigitalZoom™ reports at the end of the test execution for quick drill down into the root cause of the failure.
This QAF Starter-Kit is available on GitHub.
The following steps describe a high-level how-to that, once followed, can enable XCUITest scripts to run on multiple devices in parallel using testing in a Quantum project.
First Step is to Create IPA Test Files Using Xcode for Testing
- There are plenty of online instructions from developer.apple.com on how to set up and generate XCUITest files.
- In this example, I am using a project called Document_Manager, and once XCUITest script is created, I build the ipa files and go through the code signing and profile provisioning. The two generated files are below.
- You can download the project from Gitlab Document_ManagerProject
- The two ipa files are:
- Once the ipa files are built, next thing is to upload these files onto your Perfecto-hosted cloud repository.
Second Step is Preparing the Quantum Starter-Kit Project
- Download and install QAF start project.
- In my example, I am using IntelliJ Community Edition as my IDE for Quantum Project, and I have installed Perfecto Continuous Quality Lab plugin for my IDE tool.
- Once installation is completed, you will first need to make edits to the application.properties file under resources folder, and change the following to your user credential and hosted-cloud domain name:
- Next, you will need to create a custom steps file, and place it under the steps folder (src/main/java/com.quantum/steps)
Right-click on steps folder -> New -> File, and then name it and save it as XCUITestStepsDefs.java
Then add the following code as shown below:
- Make sure you are referencing to the test ipa files that you had created from the above instructions
- Now, we need to create a scenario file and place it under scenario folder (src/main/resources/scenarios)
Right-click on scenarios folder -> New -> File, and then name it and save it as xcuitest.feature
Then add the following code as shown below:
- The final file we need is a TestNG configuration file to allow us to run multiple devices in parallel. Therefore, we need to create this file and place it under config folder (src/main/resources/config)
Right-click on config folder -> New -> File, and then name it and save it as testng_XCUITestDemo.xml (or whatever name make sense)
Alternatively, you can copy testng_appium.xml file and then make edits to your devices needs.
I will be setting up to run three different iPhone devices in parallel. The selection of device is set to be any available iPhone S-type phone, however, if you want to select a specific device then you need to change the parameter capabilities from “model” to “deviceName” and set it with the device UDID number.
In my example, I will be using the wildcard for iPhone.* phones, and each of my test suites are unique. (i.e., iPhone Simple Document Manager XCUITest Test1)
The Final Step is Running XCUITest on Multiple Devices in Parallel
1. Right-click on testng_EspressoDemo.xml file -> and select “Run…” from menu item.
- Viewing the all devices running XCUITest script
- Click on Perfecto Dashboard pane to view devices testing in progress.
In summary, if you’re using Xcode today to build and execute XCUITest scripts on a single device, and want to increase your testing coverage as well as your overall velocity, and move to parallel execution, it is recommended to follow the above instructions. Start by using the Quantum starter-kit project, which includes all the open-source components and vast capabilities required to accomplish these goals.