«« Asynchronous View Update with setInput() Works Well Does Java Need a Higher Level Pattern Validation? »»
blog header image
Seeing the Forest for the Tests

"How long would it take you to write method X if you didn't unit test it?"

It's a fair question, often asked by managers looking to cut corners. Not that I don't have confidence in my coding skills but I wouldn't consider writing code without unit tests any more. Here's why ...

First of all test-driven development (TDD), when done properly, gives you the the minimum amount of code needed to satisfy the requirements of the method. This minimizes cruft gathering in the code. For those that are unfamiliar with test-driven development, here's how I interpret and use TDD.

Say you have a function you know will have 3 parameters and return one value. Start with the first parameter and ask yourself a few simple questions: what are the ranges of values this Object (or primitive value like int) can represent? Can I pass in null as a legal value, as if so what is the output? Is this parameter used in combination with any of the other parameters to produce the return value or is it used in the programming logic?

I usually write the null tests first, because they are the most basic requirements. When you pass an Object to a method chances are you want to use one of its methods. If the parameter is null when you try to call the method the VM is just going to throw NullPointerException anyway.

So write one test at a time. Make sure it fails and then change the method code to make the test pass. Having the test initially fail is crucial because you could have a bug in your code if you expect a test to fail and it unexpectedly passes. As you add tests, the tests you have already written act as regression sensors. So if while figuring out how to make test 9 pass you break tests 4 and 5 you'll know right away. Regression tests are one of the simplest ways to make strong code.

Consider the alternative: writing code from your brain to the editor, figuring out all of the possible paths in your head and keeping track of possible regressions as you modify and add to the method. Figuring out obscure bugs by using the debugger and printf()s. It's just too much to worry about. No wonder some coders can't think long term: they are too worried about breaking something in the method they are working on in the short term. If you have really great unit regression tests, you worry less about that and can free your mind up for the big picture.

Using TDD I find that I use the debugger and printf()s very rarely. Unit tests usually isolate a problem with code well enough that you don't need to. When you're starting out with TDD you might get a bit ahead of yourself and write a lot of code sometimes and personally I find this is when the program gets buggy again. Back up and take it one test at a time. It's not glamourous but it gets the job done right.

When you come back to the code in six months if you have unit tests you can read them to figure out the expected behaviour of the method. Unit tests also give a good idea how how to properly use the method in production. Without unit tests you have to grok the code, and comments if there are any and maybe read some design documents. Unit tests are a great way of documenting code without writing artifacts. It's laziness but it's constructive laziness.

So when a manager asks me "How long would it take you to write method X if you didn't unit test it?" I'll probably reply "about the same amount of time". However, I'll add that while I'll guarantee the tested version will run as expected because I'm writing a unit testing contract at the same time I'm writing the code, there's no quick way to guarantee the behaviour of the untested version even if it is actually 100% correct. You'd have to take the code and audit it line by line, using the design documents (assuming you made them) as reference. A manager that studies software development will probably let you take the time to write the tests and invest in the long term future of the project.

Posted at February 19, 2004 at 10:18 PM EST
Last updated February 19, 2004 at 10:18 PM EST
Comments

And that's how you do it. Ryan, good job explaining all the non-test-writers why someone should write any tests. I suggest you don't do it any other way because it is going to take you more time and more headaches. It takes you a couple of minutes anyway, saves you a lot of trouble and in the end huge amount of time on testing.
Once your unit tests are done and you start gluing objects together, start writing integration tests in similar manner.
One thing only: I have seen people write tests for getters and setters, and I don't think that is necessary, because there is no functionality in these methods, so why do it?

» Posted by: Aleks at February 20, 2004 08:56 AM

True, but it doesn't take a lot of time to test setters and getters so I do it anyway. You usually take for granted that these methods work properly, so if you ever had a bug in a getting (ie. returning the wrong private variable my mistake), it's probably the last place you'd look.

Testing them well ahead of time means you don't have to worry about that code in the future. I usually only have one test for both, so the work is minimal: set a value and then make sure you can get it back.

» Posted by: Ryan at February 20, 2004 09:17 AM

To add to the getter / setter point. Yes, test drive.... for two reasons...

1) It is possible to break - so test it. The reason for having getting and setters are to avoid exposing the variable directly is to hide the fact that you are infact using a variable... perhaps these methods become more complex internally - your unit tests should hold.

2) You will think twice about whether or not you need the get / set methods in the first place. As opposed to just 'hey, I have a variable - lets wrap a get/set around it', you think 'no one needs to get this variable directly, so i'm not going to expose it'. And then if / when someone else will need access to the variable, you can think 'well, can they set it null, or negative, etc' and you can build your tests around that.

» Posted by: aforward at February 20, 2004 08:23 PM

This post post actually stemmed from a conversation I shared with Ryan when he came to visit deloitte. I was actually asked the question, what is the overhead.

Philosphy (sp) aside, the pure mechanism of *getting* something done, I pegged it at 25% more work to test-drive your code. The question itself is about as useful as 'What is the overhead of following a receipe to bake cookies?'...

» Posted by: aforward at February 20, 2004 08:27 PM

That's a good analogy actually. If you baked cookies without a recipe it would probably take less or more time depending on the quality you personally strive for and how good you are at guessing (remembering the recipe from memory is still using the recipe).

A recipe gives you great cookies every time. Would you eat cookies that were made without a recipe? They'd probably taste like crap. Same goes for code without tests -- ultimately the person using the code isn't satisfied (no tests to verify the API contract OR that the code actually works).

TDD and unit testing philosophy aside, the truth is that it will take me longer to write a method without tests because I will continually have no confidence in it. Nevermind the time it will take the maintain or change it later! That's why I'm always emphasising long-term thinking.

I'm getting to the point now where it is actually FASTER to write a method with tests than without them, even though I'm producing more code in total. I believe that experienced TDD people can produce working methods at a much faster rate than non-testers if they practise enough even though they have to write method code as well as testing code.

Lines of code is a crap metric, even to extrapolate to time. There are too many other factors. Stop thinking in lines of code and think how long did it take to create a "method I trust"? Without tests the goal is elusive.

» Posted by: Ryan at February 20, 2004 09:22 PM

Right on!

» Posted by: aforward at February 21, 2004 10:39 AM

Writing test is a really important part of writing software. However, I think it's a lot harder to write unit tests in some cases. For many methods, it makes a lot of sense, and will probably save you a lot of work.

However, when doing our fourth year project, we found that writing unit tests was very hard. When running a method that would retrieve data from the database, and put in an object to make it easier to use, it is very difficult to do.

Getting a single record, and testing that the returned object, contained the information it was supposed to contain, required making an object, with the information, and then checking that the retrieved object, had the same info as what was expected.

This gets increasingly difficult as you have to deal with returning collections of objects, ensure that all the correct data is returned.

Of course, storing data in the database creates the same problems, you can store something, the database updates, but you may forget to set one column, so, your test would have to read the data back out of the database, and make sure everything was properly written to the database.

Is there a better way to go about testing this sort of thing?

» Posted by: Kibbee at February 21, 2004 10:46 AM

Testing databases is a common theme on Andrew's blog. Personally I don't have much experience with it.

AudioMan's "repository" is database-like but it is made with Java collections so it's a lot easier to load with test data. Unfortunately it doesn't understand SQL. :)

» Posted by: Ryan at February 21, 2004 11:08 AM

Wouldn't a slightly naive way to test a database be to create objects (with boundary values), write them to the database, read them back, and then do a .equals on them?

Yes, this is relying on your equals being correct, but that's an assumption at that point you should be able to make.

When ever you find that you are having a hard time testing your code, that usually means that it can do with a refactor. There are MANY ways to do anything in code, and some are a lot easier to test than others.

One of the things in my eXtream cookbook is how to test databases. One of the tricks that they do is to have a method that returns a db connection object. Have that apart from any code you have. That way you are able to use mock objects when you override the getConnection method (returning a mock db connection). Also, if you mock out some of your calls, your tests will run faster and you won't have to actually be connected to a database to run them or incur the overhead of setting up the db in a known state.

They show code like this:
public void getAccount(Account acc) {
  // get db connection
  // use db connection
  return account;
}

and they change it to:
public void getAccount(Account acc) {
  return getAccount(getDBConnection(), acc);
}

private Account getAccount(DBConnection conn, Account acc) {
  // use db connection
  return account;
}

» Posted by: Jim at February 21, 2004 12:44 PM
Google
 
Search scope: Web ryanlowe.ca