So you've read my previous post on AudioMan's process problems and now I'll go into how I'm going to try to fix them with an established process.

Metrics

One thing that AudioMan does relatively well is collect quality data. Presently two tools are used to do this: JUnit and jcoverage, unit testing tool and code coverage tools respectively. These tools make it easy to see when the code is starting to get nasty and raise alarm bells to fix it before it gets too hairy.

To improve the use of these tools, I will start doing nightly builds of the project. Combined with small check-ins, the risk of going down a bad road quality-wise is reduced, which in turn reduces the risk of wasted time. Another important aspect is that the nightly builds can be monitored by the project manager with no help from developers. He doesn't have to know how to build the project, he just goes to the project web site to check the results of the nightly builds.

Functional Specifications

Open source projects are notoriously weak in this area, and I think it's important to understand why. A functional spec is a detailed account of how the software performs from the user's perspective. A new functional spec is made for each new official release of the product, building on the old versions of the spec.

This spec becomes the official word on how the software is to perform. Without this specification, you have to go through email conversations and newsgroup postings to find the results of decisions about features. This document organizes these results and could point to the discussions about why the choices were made. So it helps reduce questions about how the software is meant to perform and why. The spec also helps to quickly determine whether a defect is actually a defect or an issue that hasn't been addressed by the spec yet and requires elaboration.

Open source projects don't write functional specs, instead they rely on increased communication through defect-tracking tools like Bugzilla or email, newsgroups, IRC, etc. They also have people who can keep track of subprojects and make decisions unilaterally. These people have the time to answer questions about what's in and what's out. Writing a functional spec isn't a very sexy job for these people, they would rather communicate directly. Again, motivation becomes a factor for open source developers but the communication overhead increases and it becomes more difficult to start contributing.

On large closed source projects the functional spec is used by:

* developers to implement features
* quality assurance people to verify and find defects in the product
* documenters to write help files, release notes and the web site
* managers to plan product releases and schedules

and probably many other things I've forgotten. Can you still be agile and have a functional specification? Yes I believe so. The person writing the functional spec could be watching the developers implement and change features as a third person and document this and coordinate this transition.

In fact, before an iteration takes place a functional specification can be drafted outlining how the new software will perform with the new features for that iteration. The behaviour will be written by the developers themselves when they estimate how long a story will take, and compiled into the functional spec document by the project manager where it becomes the Word of God in the event of a disagreement.

During the iteration the functional spec may be changed slightly to reflect the difference between estimation and reality once the feature is implemented. Developers usually learn along the way, so the process has to account for that. Modifying the functional spec becomes a task throughout the iteration then, and the spec should be in a place where people can see the most recent version whenever they need it. It should also be updated in a timely mannor!

Thoughts on functional specs were inspired by Joel's four part thoughts on the subject.

Update 8:36AM: Here's a first draft of the functional specification for AudioMan 0.4. It makes AudioMan look pretty small. :)

Functional Testing

Functional testing is another neglected area of open source projects. They seem to like to follow the Build-It-Fix-It method for finding bugs rather than proactively searching for them. They rely on enough interested eyeballs to pound defects out of the product. Of course, on closed source projects this isn't an option, and teams of quality personelle are required to do it.

To solve this problem, AudioMan will have a functional testing effort connect the functional spec to the tests that verify it is implemented properly. The goal is to automate as many of these tests as possible so that they can be executed nightly. However, the GUI may have to be tested manually so a GUI Testing Spreadsheet will comprehensively outline all of the tests that have to be verified for each release.

Without the connection between the functional spec and the functional testing effort, corner cases can be missed which lets defects go unnoticed until much later when it is more painful to fix them.