Unit Test vs Code Coverage vs Use Case


I’m going philosophical on this one.

One of the myriad duties I’ve performed in my positions as a consultant is to implement unit tests on some critical part of the code. Realistically all of the code is critical, otherwise why would someone pay me to write it? Unfortunately there will come a time for every software consultant where they will come across some code, that doesn’t have a purpose (hopefully it is in the anymore sense, but that is not always the case). This is why I choose to talk about the “Use Case”, the measure against which every line of code should be gauged.

The Use Case

The Use Case is the reason that someone would use your code, what they intend to ‘use’ your code to do… Let’s take for example the classic game Pac-Man; some potential use cases may be: A player plays a game. However this is not granular enough to actually tell a developer what it is that the developer needs to do. Were I given this as my design doc and told to ‘make it happen’ I could come back with Mario Bros. !! However if the use case is detailed enough, say: The user see’s a yellow pie shape that eats dots which give you points (10 per. dot) this is now something to work from. this is further described by going into detail about ‘power up dots’ ghosts, lives, dying, going from one edge to the other , you can’t go thru walls etc.. You start to get a great feel for what the game is going to actually be. So make your use cases descriptive enough that you shouldn’t need to give additional  Q&A with the developers. From a well defined set of Use Cases the developer can construct code and Unit tests to meet those requirements.

The Unit Test

This is something that when I was first exposed to I was apprehensive about, thought it was a waste of time (and the list goes on..). I have come to appreciate the elegance and security of properly unit tested code. Now while working with others, my clients or even my coworkers I find myself discussing with or even convincing them of the merits of unit testing. I firmly believe that every line of code should be properly documented and tested by at least one unit test. One of the hardest ideas to properly grasp or appreciate is that if all of the small pieces are working the way they should, and if their integration is tested (Integration testing of the Units) then you can rest assured that your code will be bug free.. almost. To make sure that your code is solid you will need to get good code coverage.

Code Coverage

Code coverage is the amount of code that the unit test actually covers. Here however is in my opinion the lynch pin of the whole triumvirate. The test is only as smart or dumb as the person writing it! Lets look at a sample lets say we have a function that lets us know if a float value is within epsilon of another value:

namespace LibraryTests
{
	public class FloatHelpers
	{
		public bool WithinRange(float a, float b, float epsilon) 
		{
			return a - b > epsilon ? b - a > epsilon : false; 
		}
	}
}

There are a number of closures on this code, o.k. on the surface and as far as the code coverage analyzer is concerned 6 blocks:

Visual Studio is satisfied by two simple tests:

		[TestMethod()]
		public void WithinRangeTestReturnFalseCorrectly()
		{
			FloatHelpers target = new FloatHelpers();  
			float a = 0F;  
			float b = 0.9999F;  
			float epsilon = 0.1f; 
			bool expected = falsebool actual;
			actual = target.WithinRange(a, b, epsilon);
			Assert.AreEqual(expected, actual);
		}
		[TestMethod()]
		public void WithinRangeTestReturnsTrueCorrectly()
		{
			FloatHelpers target = new FloatHelpers();
			float a = 0F;
			float b = 0.00999F;
			float epsilon = 0.1f;
			bool expected = true;
			bool actual;
			actual = target.WithinRange(a, b, epsilon);
			Assert.AreEqual(expected, actual);
		}

However we aren’t properly encapsulating the use case, imagine that we are doing work on very small numbers approaching epsilon where it might make more sense to actually be using a double:

 
		[TestMethod()]
		public void WithinRangeTestFailsIncertainRegions()
		{
			FloatHelpers target = new FloatHelpers();
			float a = 0F;
			float b = float.Epsilon - float.Epsilon / 100 ;
			float epsilon = float.Epsilon;
			bool expected = true;
			bool actual;
			actual = target.WithinRange(a, b, epsilon);
			Assert.AreEqual(expected, actual);
		}

We are clearly within the range, in fact we should be exactly float.Epsilon/100 inside it, however due to the nature of floats we are in a region of unexpected behavior, which will almost always (I say almost because we are approaching epsilon and there is a chance that it will work) yield:

So it is clear that although the code coverage is good the unit tests or the code are not built properly. Unfortunately Visual Studio can never be made smart enough (o.k. maybe some day but it isn’t today) to figure out what the user is intending to accomplish. which as it would happen brings us full circle back to the Use Case and what the requirement for this block of code really is. If we are attempting to get a close enough feel for these numbers (like in an AI, where sloppy math can be advantageous)  then this code is perfect and meets the Use case, however if we are calculating the distance to earth from our spaceship, and we want to be precise so that we don’t die in a fiery ball, then we may want to use a more robust approach. Of course a math example is easy to say one way or the other, however there are plenty of other situations that you will encounter that will test your ability to understand what your code is doing and why.

Recommendations

I recommend using an IOC container (in general really) and something like Spackle.Net, to generate random data, which can be very handy in finding periodic errors that you didn’t account for.

Advertisements

About Larry Louisiana

I'm a Microsoft Partner Consultant.
This entry was posted in Game Development, Math, Programming and tagged , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s