Friday, August 18, 2006

Specifications and Manual Tests

On Tyner Blain, Scott writes about the importance of manual testing. One passage from Scott's entry stood out to me:
As an engineer, I know that we can specify tolerances, inspect components, and test assemblies to make sure that products are “within specification.” And all of this testing can be automated - just like automated software testing. But passing these tests doesn’t make a product good, it merely indicates that the product is “within specification.” Did the Japanese manufacturers have tighter tolerances? Yes. But did they have better designs? Yes. And those better designs were about more than miles-per-gallon and horsepower and torque.
This passage draws attention to the distinction between a "specification" and a "requirement". A requirement is a specification that captures what really matters to users and stakeholders. Other kinds of specifications exist (e.g. feature and design specifications), but in the final analysis they are irrelevant unless they solve users' problems (and avoid causing new ones). You can deliver a product "within specfication" but that fails to solve and avoid users' problems.

One of the reasons that some product managers have such a difficult time documenting true requirements - instead delving into design - is that they are too focused on the practical aspects of testability. Hopefully, your product manager writes requirements that are testable in principle. But if your product manager focuses on the expediencies of testing, such as whether it would be easy to automate testing of a requirement, then you will likely end up with a product that's "within spec" but doesn't address stakeholder needs.

No comments :