«« Managing Yesterday's Test Methods Why Pyra for Google? »»
blog header image
Administering A and B

I realised that test suites are the best way to administer A and B tests. You keep the tests where they always were but move them between Test Suite A and Test Suite B. No complicated software necessary -- and Test Suite A always stays green.

Posted at February 22, 2003 at 11:24 AM EST
Last updated February 22, 2003 at 11:24 AM EST
Comments

Ah, that is a nice simple solution. I like that.

Do you think that this would be a good plan for us? Having two suites?

» Posted by: Jimbo Jones at February 22, 2003 01:40 PM

I have a question. When you find a new bug, and you make a test case for it, does it go into the "A" suite or "B" suite?

You are not checking in code that breaks the build, you are just adding a test that you had not thought of before (or had just incountered in user / customer testing).

If you put it into "A", it will no longer be green, but it doesn't mean that the code there works all the time anyways... you are sort of lying to yourself thinking that the code passes all tests.

So, do you lie to yourself, or break the "always green" rule?

» Posted by: Jimbo Jones at February 23, 2003 01:58 PM

Nah, you don't lie to yourself. New bug test methods go into the B suite because they fail. You are only lying to yourself if you think your code has 0 bugs. :)

» Posted by: Ryan at February 23, 2003 11:37 PM
Google
 
Search scope: Web ryanlowe.ca