Self-healing test automation – a step towards test lifecycle automation

We’ve come a long way in the software industry, but there’s always room for further innovation. The advent of agile methodologies and the adoption of DevOps practices have become a game-changer over the last decade. These transformations are paving the way for the rise of various practices like continuous integration, deployment, monitoring, and, more importantly, the rise of continuous testing. Test automation has made this pivot possible. However, test automation comes with a unique set of challenges. Self-healing test automation drives the next level of innovation by ensuring reduced development cycle time and test flakiness.

How test automation changed the industry

The software industry was poised to find an alternative to laborious manual testing, and so, test automation was born. For various reasons, test automation has increasingly become the preferred choice of testing over manual testing and one of the principal drivers of change and the de-facto test methodology. It promises unparalleled efficiency and faster “go-to-market” to those deploying it.

For one, test automation can be conducted through a plethora of technological choices both on the commercial off-the-shelf (COTS) side and open-source as well. When it comes to GUI automation, irrespective of the tool used, the main design philosophy used for test automation is to capture and store the UI element properties in an object repository and then use them in automation framework to mimic user actions on them. So, the test script can click a UI button, enter values into a text box, select values in a list box, etc.

The second reason test automation has become popular is because of the integration of teams. The development and test teams, which used to be separate entities working on a project, are now all part of the same feature teams as SDEs and SDETS working in unison. 

However, integrated teams and emphasis on regular build generation and testing do bring about some challenges. The teams are under a lot of pressure to deliver quality products on a regular basis.

The problem is further exacerbated when the applications being tested are constantly evolving and changing. The automated test scripts depend on a valid UI object property to run, which frequently changes, thus leading to fragile and flaky tests. A test script breaks when the UI object properties change, and an automation engineer must stop new script development to troubleshoot and fix the broken one. This issue gets amplified when there are hundreds of automated tests, and it decreases the amount of work the automation team can produce during a sprint.

Currently, there’s a manual process to resolve this issue. First, the automation engineers manually debug the script failures and then they inspect the UI element and find new properties, which are used to update the script or object repository. Lastly, they perform test re-execution.

Over the last few years, an idea has been gaining traction in the testing industry. While the tests themselves are automated, the ancillary activities which go into the entire test lifecycle, like script maintenance, impact analysis, defect triaging, etc., are still extremely manual. Relying on manual processes tends to slow the software development process. Hence, we now see a paradigm shift in the philosophy to encompass “test lifecycle automation” and not just “test automation.” The rise of AI/ML techniques paired with programming capabilities goes a long way to help this developing philosophy.

Self-healing test automation is an important cog in this endeavor.

What is self-healing test automation?

For web applications, there are multiple locators which can be used to identify a UI element – ID, name, Xpath, CSS locator, class, tag, link text, or partial link text. Depending on the situation, there can be a change in either one or multiple locators for a UI element, and the algorithms developed should be capable of handling these changes.

As shown in the diagram above, self-healing test automation works to handle these changes. During script execution, if one of the object properties used fails due to a change in the application under test, the automation framework will scrape the new application under test (DOM extractor) at runtime and try to find an alternate element for the execution to proceed.

The self-healing algorithm uses information stored in a database that maintains all of the historical object properties based on historical test runs. It then uses similarity scoring algorithms to evaluate how alike each property is between the old objects and the available objects on the page. Each property associated with that object is then given a confidence score and depending on the threshold defined by the automation engineer, the self-healing engine will decide to proceed with self-healing or not. At the end of test suite execution, the reports generated will also reflect this and inform the test engineers about the self-healed test cases.

If required, this self-healing model can further be augmented with technologies like natural language processing (NLP) and machine learning (ML) techniques to dynamically adapt testing to changes in an application’s user interface (UI) or environments.

What are the benefits of self-healing test automation?

There’s no denying the benefits that self-healing automation can offer. These techniques will change the overall approach to test automation. With reduced test flakiness, automation engineers can now spend their time on new script development instead of being bogged down by script maintenance.

Self-automation also reduces the number of failed tests that automation engineers have to deal with, therefore reducing the time and efforts on the part of the automation team to debug those failures. More importantly, there are fewer breaks in the constant cycle of continuous integration and testing due to application change.

With the added automation to self-heal tests, the “test lifecycle automation” is positioned to be one of the main disruptive innovations changing the way testing is performed in the coming decade.