«« Week 05 Status Report Eclipse: How to Run All Tests in Multiple Projects? »»
blog header image
Software Engineering and Feedback

Managing the AudioMan software project has reinforced an opinion I've had of software engineering for a few years: software project management is all about control. I'm not talking about control in the sense of social power, I'm talking about project control with feedback loops.

Feedback is what the other engineering professions use to control processes, and so should software engineering. I had that realisation while I was still taking electrical engineering classes and studying signal feedback loops.

There are two aspects to feedback: measuring and responding. You can't respond well if you don't measure well. It's actually kind of silly that we learned how to respond to software management problems better than we learned how to measure. Why? Because measuring a software project is still fairly new, and responding without measuring -- even incorrectly -- is not that new.

Software project management has a close relationship to business project management. Business projects have balance sheets -- a business project consumes and generates cash. The overall success or health of a business project could be determined largely by looking at cashflow over a period of time.

There's a business project wrapped around software projects to sell the software and pay the developers. Software projects consume resources (person hours) and generate code. The health of a software project is more directly related to the health of the code. The business project around it could very well succeed with an unhealthy software project inside of it but it is more likely to fail. It all depends on how each are managed and coupled together -- but let's just talk about software project measuring.

Unit tests are a good way to ensure that a project stays healthy. If you break an expected behaviour a unit test fails and you know about it. How quickly you know about the breakage depends on how often you build and test the project. See continuous integration and JUnit.

Code coverage analysis tools let you see weakly tested areas of code in the software project. Weakly tested areas won't catch breakages in the code as well as strongly tested areas. To make sure you get all of the benefits of unit testing throughout your project, you can monitor code coverage numbers. See tools like EMMA and Clover.

Sometimes it's good to know when people check-in code to the repository and what that code was. Maybe someone modified an area that can't be unit tested (ie. user interface code). A tester could identify that area as recently changed and optimize their overall testing effort by concentrating on this changed area of code for a while. Code that has recently changed is sometimes more likely to have unfound defects. See tools like CVSspam.

Monitoring tools have an interesting affect on developers: when developers know they are being watched, they are more careful. They don't want to embarrass themselves by breaking the build or holding the rest of the team up. This is a variation of the broken window theory. Tools make developers accountable for their actions and that's a good thing. Another benefit of close monitoring is that developers are more likely to learn from their mistakes if they get quick feedback instead of a bug report months later ... or never.

Monitoring tools enable flexibility, refactoring and ultimately development freedom. Tools like unit testing allow the development team to take bigger risks that are not practical without unit tests from a code stability standpoint.

When the project is monitored closely developers might get the sense that they have less freedom. This is the kind of thing that has to be disproved over time -- the tools proving themselves to be freedom-enhancers. It doesn't seem to be enough to give the benefits of project monitoring anecdotally because it is a different development paradigm.

Another reason? Personally accountability scares some people. How can that be worked out?

Posted at March 30, 2005 at 08:56 AM EST
Last updated March 30, 2005 at 08:56 AM EST
Comments

This may sound harsh, but if someone is scared to be personally accountable for their work, then I'ld fire them. You can't have people working with you that don't care about their own work. It's like having a teammate on your sports team that doesn't believe that his mistakes should be looked at by himself.

I'm sure you'll get production from them, just not the best work. I try and disassociate myself from "babies" who need their hands held. This isn't so much a software engineering issue as much as a psychological issue that is seen in many fields. But definitely an important one.

As well, you're absolutely write about the measurement. In other fields, you have tools that constantly tell you how your machine/product is doing.

» Posted by: roy at March 30, 2005 12:45 PM

I think you just have to make it a way of working that you hold people personally responsible. If you broke the build, then fix it. This is a great tradition in software. In a book I read, it used to be at Microsoft that if you broke the build, _you_ became the one who was in charge of the builds, day after day. Until the build broke. Then you passed the baton to the new build breaker.

Also, you can flip it around and offer rewards. For example, you could have a prize for the person who gets the most code coverage or most unit tests per week. We did something like this on my team, with the prize being a Blizzard at the local Dairy Queen. :)

» Posted by: Luis de la Rosa at March 30, 2005 10:02 PM

I agree completely Luis, especially with putting an emphasis on rewarding instead of punishing.

It's something a project manager has to take initiative and do, to make people accountable. Otherwise people will just do whatever they want.

» Posted by: Ryan at March 31, 2005 04:50 AM


Two nit picky remarks.... just like little girls and math - it's not the math that's difficult, it's their perception that it's difficult that makes it difficult.

My point, User Interface Testing IS NOT HARD, you just think it is.

Another small point, I measure a pyramid of metrics.... coverage, testing and efficiency. Coverage is how well do my tests cover the code. Testing is what I test (how many). Efficiency is two fold.... how much test code was required (relative to production code) to produce the measured coverage and testing - and second how much time did it take (i.e. the duration of the tests).

» Posted by: andrew at March 31, 2005 10:57 AM

Good points Andrew.

User interface testing isn't hard -- it's often just time consuming. It depends on the user interface. Some are easier to test than others. Some are easier to test with unit tests than others.

» Posted by: Ryan at March 31, 2005 11:00 AM
Google
 
Search scope: Web ryanlowe.ca