Here's an idea for an acceptance testing tool I'm looking for. I say "looking for" instead of "want to build" because I'm sure 100 other people have already thought of it.

The AudioMan project uses a spreadsheet to keep track of customer acceptance tests. These tests are derived (and grouped into) use cases given to us by the customer as feature requests. Each test consists of a number of steps and each step has an expected result: a behaviour by the application. If a step fails we log a bug and note it on the spreadsheet next to the failure.

This sounds simple enough, but when you are adding features all of the time the spreadsheet is continually changing. Stuff like adding tests, removing tests for features that were taken out, changing steps or expected behaviours because the user interface has changed.

So you end up with a new version of the spreadsheet every week. If you test the latest build with the latest spreadsheet all of the time it's hard to track your progress from just metrics because it might seem like the "completed %" never changes.

It would be nice to have this all in a web app, that users can log into and put results into. They don't even have to do a whole test run, just part of one. Then you combine results from all of the users to find what you're really looking for: unexpected regressions. You already know what hasn't been implemented, that isn't interesting. The whole reason why you want these results is to make sure the stuff that worked last week still works. And if you had a lot of users putting testing results into the system (hey we can dream, right?) that information would be pretty handy ... so making it easy to use is important.

But the real kicker is being able to use the testing results from a run done with "spreadsheet" version 40 and compare them to the results done from "spreadsheet" 30. Of course 30 will have less tests, probably have a few that we retired and aren't in 40 and will definitely have a few tests with changed steps/behaviours. This web app has to be able to know this and sort through it. All I want to be able to see is progress and regressions.

Find this web app for me, would ya guys? Thanks. This would have been a good fourth year software engineering project for somebody too. A little PHP and mySQL.