Result Variables

Over lunch with a few colleagues a few weeks ago, someone mentioned that he recently interviewed a QA candidate that gave the best answers to his questions out of everyone he’d ever interviewed before.  Why?  Because, he said, she was “results-driven” in her method of testing.  It turns out that what this colleague was describing about this candidate, is a particularly useful, fundamental, yet woefully underutilized approach to domain testing. (If you don’t remember what domain testing is, see my previous post)

 

Simply put, the idea is to test for the result variable as the primary objective of the test, in addition to only the input variables.  Say you’re testing a program that adds two numbers.  Typically, you would want to know something about these numbers to input, right?  What are their valid ranges?  What should happen if you entered max+1 or min-1 ?  Is zero OK?  What about negative numbers? Or decimals?  And if decimals are allowed, then to what precision?

 

These are all legitimate questions assuming that your primary objective is to test the inputting of the variables, but it doesn’t reveal much about the result of the calculation.  Even though checking for a buffer overflow on an input filter may be important, it may be more interesting to construct your input data to force values of interest on the output variable.  In the Domain Testing Workbook, Cem Kaner calls this testing for the consequence. “If a program does something or cannot do something as a result of data that you entered, that’s a consequence of the entry of the data.”

 

Consider the same set of questions above, except instead, focus on the result variable.  If you’re concerned about boundary testing, then if the maximum allowable value for the result variable is 100, for example, you have many interesting way to try to cross that boundary. (For example, enter 100 + 1; 0 + 101; -1 + 102; 99.99 + 0.02)  Testing with the primary focus of the result variable can generate a much richer set of tests to work with, and may lead you to discover more interesting information about your program than focusing on input boundaries alone.  This subtle distinction in the way you think about your tests could reap large benefits in the long run.