«« Non-Fattening Syntactic Sugar: Ruby's unless U2 and The Arcade Fire at the Corel Centre in Ottawa »»
blog header image
The Agile White Box Tester

Jim just wrote an interesting post about black box testers. In it he says: "Ultimately it's a developer's responsibility to ensure that things are working correctly..."

I like this line of thinking but unfortunately in my experience (with both large and small software companies) it's quite new and uncommon. The best thing we can do while we're extolling the benefits of that approach is remember that most teams don't believe or understand it.

Why? Many managers think that having developers test is a waste of development resources which will slow down the project. Many developers think testing is boring. Jim and I may see those justifications as short-sighted -- not in the overall long-term best interests of the project. Until managers and developers change their minds, the old way of thinking -- "let QA do it" -- will reign supreme.

Just like there's no reason why developers can't test, there's no reason why testers can't have the source code and get the white box perspective. I read an excellent book called Testing Extreme Programming that had a major theme: some people are really good at testing, and agile processes like XP do have a place for them. Not everyone on an agile team has to be a developer that tests.

Testers in an agile process can ensure that developers are testing their code properly (broken tests won't help find broken code) and help developers make tests faster and with better quality. Testers can really teach developers a thing or two about testing (testing mentor) for the benefit of the entire project. Testers can also ensure that the tests are complete: that no functional or unit tests are missing -- because missing tests will not find missing or broken behaviour in the software.

Developers may be tempted to take shortcuts when testing in the name of short-term efficiency and the testing team can ensure that developers are being honest. It's a little bit of policing, sure ... but everyone on the team needs to understand it's for the overall health of the project and to help people learn to code and test better, not to embarass people. A good team attitude about this type of feedback can really help.

If developers have a defensive attitude about the quality of their code -- as they sometimes do on teams with seperate development and QA camps -- the project will have a detrimental internal battle going on, and no one wins.

It's the responsibility of the project manager(s) to set the tone of the project so these teams are learning from each other instead of at each other's throats as adversaries, trying to make each other look stupid.

Posted at November 24, 2005 at 11:55 AM EST
Last updated November 24, 2005 at 11:55 AM EST
Comments

If you are working with people who are not writing tests for their code, the structure / architecture of the application they will pick might not allow you to easily test it. Then you have a chunk of code that it is really difficult / slow to test.

I think that code people write will be different when they are constantly asking themselves "how will I test this?".

If you have to change the app. to make it easier to test (separate logic from persistence, different concerns, etc...) then you will be fighting arguments that start with "why did you change the code that I worked on?". That and the fact that if you start refactoring *before* there are unit tests than you run the risks of breaking something without knowing.

I still feel that "test first" is the most efficient way to go.

» Posted by: Jim at November 24, 2005 01:33 PM

Exactly why everyone needs to be on the same page. Again, that's the project manager's responsibility.

» Posted by: Ryan at November 24, 2005 01:36 PM

but I don't get how everyone being on the same page will teach the developers to structure the application in a way that facilitates testing?

(yes, I know I've been using big words lately)

» Posted by: Jim at November 24, 2005 05:45 PM

If everyone is on the same page it makes everything easier, not just testing.

If developers are not on the same page with respect to testing, some may make tests while others don't care if they break them or not. The tests may still be good on their own but the suite becomes useless as a whole because it's always failing somewhere and it's never fixed.

Sure, developers need to take personal responsibility for the code they write. But the TEAM has to take responsibility for delivering a quality product, which takes integration of their parts. If you keep breaking a teammate's tests, you're not integrating your code well and you're not making it easy to spot problems in the future. The suite will always be broken because of a few bad apples that don't care about the tests.

Broken Window Theory: *indrectly* it says that it only takes one person being sloppy to bring the whole project down because the team's quality standards are as low as the lowest person. If everyone has high standards and the test suite always passes it's much easier to see when errors happen because it's an anomoly. Usually the suite is green.

We got to this because you said in your first comment that it was a situation where you're "working with people who are not writing tests for their code". It's much more of an uphill battle than getting everyone on the same page at the start.

So: "how will that teach developers to structure the application in a way that facilitates testing"? That comes with experience through hours and hours of testing and with good test mentoring, research and reading about what other people do. There's no silver bullet, people just need to learn the most effective ways to test and that takes time. I learn about new testing approaches every day, and I'm sure you do too.

How do you get people to test? I like to show people that tests can be used to verify little things, get them hooked on it and work up from there. It worked on you, Jim. ;)

Once they understand how unit testing can help, show them TDD and really impress them.

» Posted by: Ryan at November 24, 2005 06:01 PM

This all works well and good in theory, and I think that testing is a good thing. However, if you are working on an existing project, which is many years old, and proper testing methods haven't been employed in the past, then sometimes it takes a lot of work, maybe too much, to changes things so that they are easily testable. If you know you have to write tests for something, then you will code it in a way that facilitates that. But, if you don't have to write tests, you sometimes write code that's almost impossible to write tests for in its current state. There's a lot that has changed with software development in the last 5 years, and when you're working with code that has been around for 10 years, it's not always easy to employ methods that we've finally figured out are good ideas. Also, having everyone on the same page is the most important thing in all of this. You can want to write your own tests all you want, but if none of the other developers are writing tests,
and the project manager doesn't really care if you write tests or not, and would rather just have you spend your time writing code, then sometimes its just better to go with the flow, then to fight against current.

» Posted by: Kibbee at November 24, 2005 09:54 PM

I agree with Kibbee to a point: sometimes it's worth fighting the current and sometimes it's not. For example, Andrew has made good headway. He probably has a lot more patience than most of us though.

Personally I'd rather not be on a team that doesn't unit test and build their project daily at the very least. You better believe I ask those kinds of questions at interviews.

» Posted by: Ryan at November 24, 2005 10:00 PM
Google
 
Search scope: Web ryanlowe.ca