Automation – A word we are hearing commonly in the mainstream media now. Hate it or love it, it’s here to stay. In the testing world, automation has been around for a long time. It is one of the fields where automation is most prevalent. While we will continue to see the usage of automation evolving and increasing, it is still very evident that there is a need for effective non-automated tests. Exploratory testing, leveraging judgement and creative thinking, may be the answer to developing quality solutions in the absence of automation.
Exploratory testing does have immense potential to change the future. Here are 5 ways how exploratory testing can make the difference:
Chartering is essentially setting a clear mission for the session to be tested. As a tester, the job is to be an investigator and ask questions about the user story, as well as define the scope of the mission.
One example of a charter is “Explore the login field with security hacks to discover vulnerability.” The session can be established using the Pomodoro technique – take the test and break into time boxes. Focus on the test for 90 minutes. Turn off your email, social media and everything else and just focus on the session. At the end of the session the testers should take five minutes to reflect. If the team works in pairs while performing session based testing it is found to be more effective.
Here is a simple charter template to follow:
- Explore – mention the target
- With resources
- To discover — fill information
The “target” is what you are exploring. It could be a feature, requirement or module.
Resources can be a dataset, technique, or independent feature.
Information – What are you planning to find? Are you looking for any violation of standard or are you characterizing performance, security, functionality, etc.?
Another key to perform exploratory testing is to Identify the “never” and “always” conditions. Work with your stakeholders to make a list of things that should never happen and should always happen.
Debriefing is the key to exploratory testing: How did it go? What were the risks? Then, you can log defects appropriately.
- Catching hidden bugs
Exploratory testing allows the tester complete freedom to explore the entire functionality in question without pre scripted test cases.
From a scientific perspective the charter ensures that your left hemisphere of the brain, which is concerned with logic, is taken care which frees up the right brain to think creatively and come up with new ways to test.
Based on what we have done with some clients we have found issues for some stories/functionality which we would not have in the absence of predetermined test scripts which is usually based only on the acceptance criteria.
This also ensures bugs are found earlier in the life cycle. The key is exploratory testing makes it possible to find defects that the normal automation and manual testing activities miss.
Once defects are documented, it becomes easier to identify gaps in our existing test repository, which when updated with the new information leads to greater coverage.
- Establish Heuristics
We can also use heuristics to test some examples. A heuristic is a rule of thumb or educated guess. Some of the ones mentioned in the book “Lessons Learned from Software Testing” are mentioned below with my take on each.
- Test important tests first: Prioritize the most risky items first. You can’t test everything but you can always test as many as possible. By focusing on most important areas of concern we can get more value from our sessions.
- Avoid over scripting: This is the key to session based testing as well. We want the testers to think creatively and out of the box. You can’t be confined totally to just the script and you need to allow for testers to think out of the box and come with creative solutions.
- Implied requirements: Testers usually test against explicit requirements and acceptance criteria. The key is to go beyond that and test implied requirements as well. We need to keep the end user in perspective in our view to complete this activity.
- Collaborate with dev teams: Quality is the responsibility of the whole team. By involving different teams in the Software development life cycle we get full value for our testing effort.
- Rapid feedback: There should be constant communication between the dev team and QA team. Testing should be designed to provide immediate feedback to the dev team.
- Review documentation: There should be peer and external reviews to ensure quality of the documentation.
- Collaboration between teams
My recommendation is that any session based testing should be done in teams of 2 or more. This allows greater collaboration and encourages divergent viewpoints. Further, its will help identify more scenarios which can result in increased quality.
- Increasing test coverage
There is always a question if a QA team has done adequate testing to cover all scenarios. As we have mentioned throughout this article, it is not possible for regular scripting based on acceptance criteria to cover everything. There will always be gaps and these gaps can be addressed with the help of exploratory testing. At the end of a session the observations are documented and the resulting defects are entered. Once that is done an analysis should be done to see why these test cases were not scripted and those should be addressed with new test cases. This practice will ensure high test coverage.
Testing as a function has undergone a paradigm shift and it is one of the disciplines that have a bright future as a career going forward. Given discussions about robots taking over specific jobs, I strongly believe that exploratory testing, where we combine business domain expertise with testing skills and creative thinking to come up with better solutions, undoubtedly would lead to higher quality and improve customer satisfaction in a way that automation will not be able to get to, at least in the near future!