Domain Testing

[Originally posted February 2014]

 

Earlier this year I took a trip down to Melbourne.  No, not all the way to Australia – I went to Melbourne, Florida; a small city on the eastern Floridian coastline, not too far from more famous Orlando, and certainly warmer than a New York January.

 

Melbourne is also the location of the Florida Institute of Technology, or as the locals call it, “Florida Tech.”  Cem Kaner, famous Testing author and speaker, originator of the Black Box Software Testing (BBST) education series, and my longtime professional friend is a professor of computer sciences there, and runs the Florida Tech’s Center for Software Testing, Education, and Research.  Cem, along with co-authors  Sowmya Padmanabhan and Doug Hoffman, have recently published a new 460-page softcover book, The Domain Testing Workbook, a companion to the latest online course in the BBST series.  What I have attended at Florida Tech was the first-ever live pilot of the Domain Testing class, the 4^th in the BBST series.

 

You may think, as I once did, that Domain Testing is about testing software from a business or domain knowledge point of view.  This, actually, is not what the course was about at all.  Instead, Domain Testing is all about using variables and values of variables as the building blocks for your tests.  Think about testing as the domain of possible inputs and outputs to a system in a classic black box testing context.  The class was very intensive, and went far beyond and far deeper than simple equivalence class partitioning and boundary analysis.  There is surprisingly a LOT to know about testing variables.

 

This is not the place for an in-depth treatment of Domain Testing, but I want to relay a few key highlights that resonated with me:

 

I liked that the material was structured around a Domain Testing Schema.  The schema is a high level checklist that serves as an outline on how to do Domain Analysis, which is considered a key skill for software testers.  Starting with a variable tour, students were instructed to identify and classify the variables of a particular area of a sample program, using a mind-mapping tool of our choice.  Then, for a few of the variables, we were coached how to dig deep into them:  What are the variables’ primary dimensions and data types?  Could the values be ordered, and if so, on what scale?  Is it an input variable or a result variable?  Or maybe a temporary storage-related variable?  What are the consequences of the variable?  For example, could it be a variable that holds intermediate values of calculations, whose values constrain the set of valid values for the input variable that you’re testing?  And what are some of its secondary dimensions?

 

This variable domain analysis is a critical driver for another key skill for testers – technical analysis of potential software risk.  Depending on what kind of variable you are working with, different classes of risk are at play.  For example, bugs that would occur with an input string variable would not necessarily happen on an internal floating-point variable or an enumerated-list variable, and so different classes of tests need to be designed to trigger the different potential failure types.  One of our assignments was to come up with five different potential risks associated with an integer, floating-point, and string type variable.  For each of these 15 failure modes (5 risks for 3 variables), we needed to design at least one test that was specifically designed as a best-case representative to test for that risk, and explain what makes that test more powerful than other valid tests for that data-type-risk.  Like I said, the class was very intensive.

 

Which brings me to my concluding thought:  There was homework.  Homework!  I couldn’t believe it. But in retrospect, the homework is what made all the difference in the learning experience.  As much as I understood what was happening in class, when I sat down to work on the examples on my own, it was a whole different story.  The very action of working out the variable risk equivalence tables, making mistakes, and getting and applying feedback was the real learning experience, and was much more valuable than following the lecture in class.  And that, the cornerstone of the BBST courses, was yet another takeaway from my trip to Melbourne  - you can’t really learn how to do things unless you practice them.