One of the things I have noticed on several (I’m almost tempted to say “all”) test automation projects an interesting side effect of test automation. Often, a tester will find more defects during the process of automating a test than by actually running the test.
What tends to happen is that a high degree of attention is required to automate a particular test. You need to make sure that everything is in place before a test becomes locked down and repeatable. Typically, a test automator, in building the test, will run the test ten to twenty times to ensure it all works perfectly – and to ensure that it will always work perfectly, especially for subsequent versions. This attention to detail elevates testers attention and causes them to notice issues or defects they might have otherwise not noticed.
While this is obviously not the primary benefit of automated testing, it is a very real and often recurring pattern. It’s significant enough that it should be considered when calculating test automation ROI (return on investment).
Please comment if you’ve had similar (or divergent) experiences!