Posted by: Brian | August 14, 2008

The Test Automation Side Effect

One of the things I have noticed on several (I’m almost tempted to say “all”) test automation projects an interesting side effect of test automation.  Often, a tester will find more defects during the process of automating a test than by actually running the test.

What tends to happen is that a high degree of attention is required to automate a particular test.  You need to make sure that everything is in place before a test becomes locked down and repeatable.   Typically, a test automator, in building the test, will run the test ten to twenty times to ensure it all works perfectly – and to ensure that it will always work perfectly, especially for subsequent versions.  This attention to detail elevates testers attention and causes them to notice issues or defects they might have otherwise not noticed.  

While this is obviously not the primary benefit of automated testing, it is a very real and often recurring pattern.  It’s significant enough that it should be considered when calculating test automation ROI (return on investment). 

Please comment if you’ve had similar (or divergent) experiences!


  1. Hi Brian

    I fully agree with this. In all my years of Test Automation (AND performance testing) I have seen the same effect. The main task for the automator is to take out the dull repeatable work for the testers…. ok and in some cases speed up execution…. but the fact they actually raise defects other testers might not easily find is always been understated!
    I am not sure this is significant enought to be included in the ROI, but it should certainly be another feather in the cap of the automator!

    Automation is difficult enough to sell, always make sure to stress any advantages even if they are difficult to quantify.


  2. I couldn’t agree with you more. Not only do you find bugs with the software but in when transforming a manual test case to an automated one, you find bugs in the document itself.
    Not that it matters that much since you will not be using the manual one, but another interesting byproduct nonetheless.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: