Read a good blog entry over at Techie Corner on Smoke Testing vs. Sanity Testing. The premise is simple enough. Many teams (including ours here at IBM Rational) perform smoke tests after every build to ensure fitness of the code. Perhaps this same concept could be applied to the fitness of the application, or more specifically the application’s ability to address its requirements.
The idea of a smoke test is to ensure the build is “good”. Definitions of “good” vary – but for our purposes, “good” means that it’s good enough to undergo further testing. If a build does not pass a smoke test, it’s not worth QA’s time to install and perform further validation as core functionaliy simply is not there. Without a smoke test, QA would worthlessly spend time installing the latest build, only to find that it doesn’t install, or is missing major functionality, which make further testing impossible.
A smoke test is really a basic sanity check on the fitness of the actual code, in terms of it’s ability to install and run. Why then not have a sanity test on the fitness of the application to meet core business objectives. Effectively, a sanity test is a subset of your regression suite to ensure that a given piece of functionality addresses its business requirement.
Sanity testing is a very RUP-ish concept – althought not explicitly called out in RUP (The Rational Unified Process). A basic tennet of RUP is to address high risk issues first. Sanity tests do exactly that, ensuring that key areas, or changed areas, meet a base minimum level of functionality. In effect, you are tiering your regression testing, in small groups (a parallel to RUP iterations), as opposed to a more typical, waterfall like approach of building a giant regression suite.
Nice stuff Techie Corner.