It's a given that more than 70% of enterprise organizations are using open-source test automation frameworks such as Selenium, Appium and Espresso .
With that great statistic, it is also known that many of these organizations are struggling with the following:
Maintaining the test automation suite
Running the test unattended
Dealingwith existing organizational skillset
Extending test automation coverage
What we've learned is that not all open-source test automation frameworks are built the same, nor do they serve all organizational requirements.
What is clear from the dashboard above is that in order to choose the "RIGHT" test automation framework, a few things need to be clear:
Realize that for your specific need, one framework might not be enough
There are tradeoffs that need to be acknowledged by the teams beforehand - see the No's above
Decision making needs to be based on test scalability, as well as existingtool suites and IDE's
Acknowledge your lab setup, environment, and IT requirements when picking your solution - at the end of day, it's your team that will need to maintain it
Tests are aimed to run in an unattended fashion via a trigger, CI, or other pre-defined events throughout the SDLC - a test framework that is flaky or does not fit your goals, will interfere and delay your schedule
So, what is the golden path?
There are three pillars to guide you - Your lab, your automation requirements and the analysis post-execution. If you can get these all under one lab in a managed way, you're on the right path to success.
Lab - At the heart of an unattended automation lies the lab that assures devices, browsers, and other digital platforms are always connected, available and in a ready state to execute your scripts. If the lab is also accessible from the IDE environments, that's even better.
With a reliable lab, comes the following benefits:
Flexibility as market shifts
Autonomy across teams
Complete E2E coverage
Unattended testing - to sustain CI, and move faster
Zero setup of environments
2. Automation - As mentioned above, not every automation framework can fulfill the entire set of requirements. In today's reality, multiple teams and personas are leveraging test automation for various goals; Dev builds validation, CI, quality goals and more. If the automation framework lacks certain capabilities, the entire project is at risk.
If the automation framework that connects to the lab can support the various capabilities and personas, the end-result is greater test coverage and faster Dev-Build-Test cycles. Having an automation solution that is open and integrated enables flexibility and can match various skills across teams.
Some examples of common test automation requirements that are often a challenge within open-source tools are:
Complete system level control
Same day support for new OS versions
Capabilities like network condition simulation, GPS etc.
3. Analysis - Having a lab and test automation framework that match the team's requirements is great, however, at the end of each test execution, the team needs an efficient way to drill down into failures, identify the root cause and deliver fast feedback to the developers to address and fix.
That's why having a digital reporting capability is necessary for teams to take action and fix issues faster and earlier.
In order to choose the right open source frameworks and maximize value, teams need to assess their Lab at the core of their daily operation, the test automation capabilities and finally the reports that framework can generate - especially when it evaluates hundreds of test cases across multiple platforms.
To learn more about open source frameworks check out : The Essential Guide To Selenium Test Automation.
Tags: Mobile Testing, open source, Test Automation
Eran Kinsbruner is the Mobile Technical Evangelist at Perfecto. He is formerly the CTO for mobile testing and Texas Instruments project manager at Matrix, Eran has been in testing since 1999 with experience that includes managing teams at Qulicke & Soffa, Sun Microsystems, General Electric, and NeuStar. You can find Eran on Facebook, Twitter @ek121268, LinkedIn, and on his professional mobile testing blog at https://mobiletestingblog.com.
Love to learn about creating top notch digital experiences?
Get the latest news, tips and articles delivered right to your inbox.