« August 2004 October 2004 »
blog header image
# A Better View on Testing

Writing good tests is still hard these days and it's a lot harder than it needs to be. I see a lot of places where things can be better organized. The main problem is that most people look at tests in source code files.

Writing tests is pretty easy but tedious. The hard part is keeping track of what you've tested and what you have left to test. Then you have to keep track of this as your code is refactored over months and years. Code coverage analysis tools help a little bit here but they are far from perfect. Even if you test first you can still miss things.

I described the testing coverage problem a year ago from an IDE angle. In that post you can see how Eclipse's Outline view makes it easier to see all of the method names in the source code file (test case for a class, which usually contains several methods). If you name your methods descriptively you can get a decent idea of how much your testing covers. So that's a basic example of a view that's not source code.

It would be much better if I could get a condensed table like the one in that post. Then I could look at it and the code and check the code paths off. This kind of table would be handy in an Eclipse view when I'm editing a test case. It would even be handy when doing test-first.

Generating a table like this from regular unit tests would take some parsing, data mining and guessing. Even if you went into the test and looked at the methods it used, how would you known which one was the "main testing target"? Maybe there isn't one for that method, it's just a sequence of calls that must work. People don't put enough data in their unit tests to describe their intentions, they just make test methods of code sequences they think the API will have to do and bang on methods to make sure they don't break. Organization is only as good as the team's discipline.

That's where the tests-generated-from-XML approach I mentioned yesterday differs a bit. Since it's generated from specific data in the first place, you can get your testers to enter this data or at least be able to tell when it's missing. The generated tests might not use half this data but you can use it in other places and do much more powerful data mining and reporting on your testing effort.

When I say "get your testers to do X" I mean that they will see the benefits of entering it because it's used in a report that helps them be more efficient day-to-day. The data they are entering doesn't get lost or never used like a Word file that gets archived on some file server. This stuff goes right into the CVS alongside the code. It's a first class citizen.

An alternative to this is to store all of this data in the source files themselves, like Javadoc does for code. That might clutter things up, but more importantly there's no standard for that and I don't feel like sitting around waiting for one. I need more test data now.

posted at September 28, 2004 at 07:28 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (0)

# A First Look at id3v2test.xml

Here's what id3v2test's text files look like so far. You can click on the permalink below to see it on a wider page.

<?xml version="1.0" encoding="UTF-8"?>
<ID3v2TestMethod author="Ryan Lowe" description="">
  <!-- test comment (ignored) -->
  <ValidateTag version="2.3" filename="ozzy.tag">
    <AssertTextInformation id="TYER">1983</AssertTextInformation>
    <AssertTextInformation id="TMED">DIG</AssertTextInformation>
    <AssertTextInformation id="TCON">Metal</AssertTextInformation>
    <AssertTextInformation id="TLEN">257358</AssertTextInformation>
    <AssertTextInformation id="TRCK">01</AssertTextInformation>
    <AssertTextInformation id="TALB">Bark At The Moon</AssertTextInformation>
    <AssertTextInformation id="TPE1">Ozzy Osbourne</AssertTextInformation>
    <AssertTextInformation id="TIT2">Bark At The Moon</AssertTextInformation>
    <AssertComments description="MusicMatch_Tempo">None</AssertComments>
    <AssertComments description="MusicMatch_Mood">None</AssertComments>
    <AssertComments description="MusicMatch_Situation">None</AssertComments>
    <AssertComments description="MusicMatch_Preference">None</AssertComments>
    <!-- more ignored comments...
      Still need to check for:
      Unsupported Frame WOAR of size 20
      Unsupported Frame APIC of size 2860

ValidateTag is the first and only activity so far. Other ones could modify the tag, create a new tag, etc.

The two Assert types generate assert checks in the unit testing code, verifying that certain conditions are true. Each frame type will have to be verified differently, so they'll all have their own Assert tags.

This file will generate a single acceptance test method. What's the advantage of doing this in XML?

  1. The big one: centralizing acceptance testing for a spec. Different languages can implement their own unit test generators based on this data and everyone can share the acceptance tests.
  2. The person making the test doesn't have to know how to program. Whether this format is more readable than code is debatable. :)
  3. The format of the XML can be validated. Lots can be done here: make sure tags aren't missing, attributes are the right types, etc. Could even make an Eclipse editor like Ant's.
  4. The test methods can be made by lots of different people without having to worry about sharing Java unit test case .java files. The methods can also be moved around easier within the test suite.


  1. You have to regenerate the unit tests when you change any of the suite's XML files.
  2. Not to mention you have to find or make a test suite generator for your language/test suite.
  3. The XML format is brittle: not as easily refactored as say Java in a good IDE like Eclipse. If the XML format changes or becomes more strict, you have to change all of the old acceptance test XML files that violate the new restriction. So think ahead there a bit and plan for less refactorability.

As always, comments are welcome.

posted at September 26, 2004 at 10:56 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (3)

# Enough Talk: id3v2test Begins

I am starting to implement my id3v2 testing idea. Here are a few of the early details:


Right now there are two projects:

id3v2test: all of the test files
id3v2test_java: a Java test generator

directory structure of id3v2test

id3v2test is laid out in a directory hierarchy that can have an arbitrary number of levels.

A test method is created by making a new directory with the prefix test. They can appear anywhere below the second level in the directory hierarchy (they must have at least one parent, the test case name). A test method directory must contain:

- the file to test, like test.mp3 or test.tag
- the test script/results, like test.xml

and optionally other files to support the test method.

The directory above a test method directory is the test case name. The directory structure above that is the package name for the test case.

In order to stay language-agnostic, this is all that id3v2test will contain.


The id3v2test_java project generates a test suite from the data contained in id3v2test. Yes, it actually generates Java code.

At generation time, the implementing library of the id3v2 spec is bound to the test suite and the entire directory structure needed to run the tests is created (mostly copied from id3v2test).

The id3v2test_java generator is tightly bound to Java and JUnit. The id3v2 project being tested will be parameterized, may be changed easily and a new test suite generated that targets the id3v2 project. Initially I am using jid3rL.

Other languages (or unit test frameworks) must implement their own test generators that use the data in id3v2test. I'd like to build at least three myself: Java, python and .NET (C#).

posted at September 26, 2004 at 06:36 AM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (0)

# Testing Multiple Spec Implementations: Fostering a Community

I like the idea of creating a project around spec validation and user acceptance. It seems like a reasonable thing to do for a third party, or even a second party implementing the spec. But the open source world has a few advantages that I can leverage, plus there are the inherent advantages of having an old and unchanging spec like id3v2 as well.

What I need to do then is figure out a way to get considerable volume behind this testing effort. Like code, tests need to be moderated in a way that creates a bit of a human bottleneck...

Let's say I started a web server that took mp3 files as input, read their tags and spit out the data in the id3v2 tag as a result. A user could look at this output and make a decision about its correctness. There's one level of validation already, and the user didn't have to download any code at all, just upload a file to my server.

If the user thinks the output is wrong, the user enters a comment saying what the correct tag should be. Or in later versions the web interface could support tag construction to pointpoint specific problem areas.

Then someone on the project has to go through these submissions and check them over. The ones that the users perceive are incorrect might be useful (true negatives) but every entry is a possible error, especially false positives that the users miss. This can start out as a manual validation process done by people (me), automating bottlenecks where possible/practical. A good server-side interface would also help.

The compiled tests also need to be categorized/sorted by people capable of moderating that.

The advantage with this approach is that the community is in a sense building the user acceptance test suite -- THEY are saying what they expect from a library that conforms to the test suite, not the writer of the library. It still has to follow the spec, but like most specs written by humans there's often room for interpretation.

With enough files you can get a really good idea of de facto standards that have popped up in the most popular tag editors and music players and be able to recognize them and handle them. The pool of sample files could come from all over the place -- in many different languages and variations even -- and that would stress test libraries.

A possible testing hierarchy could involve the following dimensions:

- id3v2 version (2.2, 2.3 or 2.4)
- frame type
- frame popularity
- frame complexity

The popularity and complexity dimensions are particularly interesting. It could allow libraries to conform to a "basic", "intermediate", "advantanced" or "complete" stages of implementation, for example.

People wouldn't have to implement the whole spec to see the benefit of the acceptance testing, they could roll their implementation out in layers and know exactly where they stand against the specs (versions 2.2, 2.3 and 2.4). They might stard with the most popular and basic tags first and then move on to the more complicated ones later.

There are a few things I could do to get more publicity for this project more test file submissions. It would all about creating a community around it, hellbent on exposing errors in libraries. Some people really enjoy breaking software *cough*Roy*cough*...

- post it on freshmeat
- make a sourceforge project for it
- test some of the existing popular id3v2 libraries myself, and suck their developers and users into the testing program to improve their quality
- test my own libraries

I still don't see much of a way to completely automate the process but giving users a server to test files with plus a bugzilla-like system attached to it could definitely get me a wider variety of tests, leaving me with the job of sorting them out. (I could actually use a custom front-end to bugzilla for it -- great idea)

The main point, I think, is that the community needs solid open source id3v2 libraries. It needs solid libraries period. The specs have been around for a long time and the libraries still aren't out there. Why not? If we knew where these libraries stood in terms of completion, there might be a better chance of maintaining them and getting them really solid.

Having a third party validate them could also increase user trust in the good libraries that they actually work ... and that would help eliminate the feeling of uncertainty that I get with open source libraries. Something like this could also be extended to other libraries as well. Wouldn't that be weird -- a SourceForge for spec/library validation through user-submitted acceptance tests? Sounds farfetched enough to be a good idea...

posted at September 24, 2004 at 08:35 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (0)

# Can the Rest of the World Influence the US Presidential Election?

I'm curious: given that Canada is decidedly left-wing compared to the United States, does the majority of the Canadian public plan to back Democrat John Kerry in the presidential election this fall?

I don't mean the federal government of Canada backing Kerry -- in the event of a Bush win that could be disastrous politics. But Canadian citizens could hold some sort of rally in support of Kerry to show the United States public who many of us agree with (or disagree with).

Not that the US public would care what their friendly neighbor to the North thinks ... but if many countries of influence have similar rallies, like Germany and France or even stronger allies like the UK and Australia, it could send a message to the people of the United States that who they vote for president doesn't just affect them, it affects politics world-wide. It affects everyone on the planet.

All over the world people demonstratred against the war in Iraq (watch the video at the bottom). Maybe they'll demonstrate against the incumbent United States president this fall as well.

See also:
Kerry vs Bush in Other Countries
GMI World Poll June 14 2004
GMI World Poll August 2004

posted at September 21, 2004 at 06:51 AM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (4)

# Bad Release Engineering Can Sink Your Project

Dave Winer indirectly underscores the vital importance of good release engineering for a software project when he tries to install Sims 2.

It doesn't matter how amazing your software is. If your user can't easily and properly install your product all he has is a few useless (and expensive) polycarbonate discs. You've wasted his time and money and ruined your reputation all in one clean shot.

This case could be particularly damaging to EA, as thousands of people will note that Dave had problems installing Sims 2. Does that make you want to rush out and buy the product or wait for them to patch the problem and stamp new DVDs? Does it make you want to buy it at all?

It doesn't really matter what Dave's issue was, or how minor the problem is. This is probably one of those 1 in 10,000 weird configuration bugs. The bottom line is: it just doesn't work out of the box, and that can lose you customers and essential word of mouth (buzz, viral marketing = more potential customers).

Good release engineering alone can't make a successful software company but bad release engineering sure can sink it in a hurry. Good release engineering is probably underestimated by most software developers, but even on internal projects it is essential. It will save you a lot of time down the road anyway, so why not just do it?

update As thousands of people watch, Dave has more problems. Is it just me or is this very bad publicity?

I should be more clear about EA though, because they probably have very good release engineering. It's one weird case like this exposed by a person in a very public position that can ruin a product release. People will still buy Sims2 but some of Dave's readers may think twice.

Just imagine if Dave was a reviewer for the New York Times and he ran into a weird problem like this. That could really sink your product. On the other hand checking *all hardware configurations*, just like checking for *all possible bugs* is a good way to never ship the product. Shipping is goal #1 people.

Bottom line: minimize risk in your software development. Most of the time it's not practical to completely eliminate even one project risk.

posted at September 21, 2004 at 05:35 AM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (2)

# Testing Implementations of the Same Spec in Different Languages

Now that I know a fair bit about a small domain like id3v2, it would be interesting to implement id3v2 in a few different languages and compare the implementations. The languages that come to mind are Java, C# and Python (and maybe Smalltalk and Ruby).

The comparisons would have to take into account that I learn more about the domain as I go and subsequent implementations might be easier. It would also have to take into account my familiarity with the languages, which might make implementations in new languages like Python harder.

It would be quite interesting from a testing point of view: how could I reduce testing duplication between the language implementations? It might be unreasonable to assume that you could use the exact same APIs, and thus the same unit tests just traslated into the different languages. Each language has its own specialties, shortcuts, longcuts, syntactic sugar, etc...

However, it might be more reasonable to assume that you could use the same mp3 test files on each one, which is at the level of acceptance testing. I could make a suite of mp3 files with id3v2 headers. Each file would have a corresponding text file (maybe XML?) that will describe the contents of the file, the nature of the test and the expected output (ie. the tag in XML format).

Then I'd write an acceptance test reader for each language I'm writing the library in, and run through all of these acceptance test files. With the acceptance test readers complete, it would be easy to see where each language stood in terms of implementation completeness relative to each other. Also, adding a test to the acceptance test suite means that it will be run against all implementations with no additional work.

Of course you would still have lower level unit testing for each library, but it's unlikely there would be much overlap unless the languages were very similar. Some languages like to use patterns that others don't, and some languages have constructs that I can take advantage of. The statically-typed languages will have more rigid APIs, where dynamically-typed language libraries like Python might be more loose. So their unit testing will likely be different, and it would be better if I didn't limit their unit testing or even their APIs based on a lowest-common denominator approach.

This cross-language acceptance testing idea for a spec is interesting though. I wonder if it's already been done for other specs? It would be in the spec author's best interest to define a specification and provide a cross-platform acceptance testing framework that third parties could use to confirm that the spec is actually implemented by that library. It could expose areas of ambiguity in the spec, and dictate the behaviour of the spec in a more interactive way.

Sun does this for Java, but it's a bit different since the APIs for all of the Java implementations by different vendors and for different platforms are the same for a given version of Java -- they have to be. It seems like it would be much easier to acceptance test an API like Java because it would just be unit testing. There are probably complicated corner cases though. :)

posted at September 21, 2004 at 05:09 AM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (0)

# An Early Firefox Marketing Push Improves 1.0 Stability

Firefox is an open source and free web browser made by the Mozilla project team. The Mozilla project started years ago when Netscape decided to make their browser open source, after losing considerable market share to Microsoft's Internet Explorer.

Firefox is now closing in on a very important 1.0 release, signalling it's maturity not only to regular users but also to corporations looking for a better and more secure web browser. To increase the adoption of Firefox, several marketing initiatives have been put into place by the Mozilla project to generate buzz.

One of them has been SpreadFireFox, which had a goal of one million downloads of Firefox in ten days. In just over four days, they have already reached that goal and are now aiming for two million.

What's interesting about this marketing push is that this release of Firefox is not 1.0, it's 1.0 Preview Release (some people prefer to call it the 0.10 Preview Release). Why would the Mozilla team encourage widespread adoption of software that wasn't "1.0 stable"? From a release engineering and project management perspective, this is a very good question.

Release engineering (sometimes called RELENG or just RE) is planning for software releases, making the releases (often as installers) that people download, creating and using tools to help them with release processes and monitoring the general health of the project. RE is the single release point of the project, and all of the bits of the software have to go through that team.

The project manager (PM) needs to work with the RE team to make a release schedule that the RE team uses to plan builds and releases culminating in a final high quality release. Often at the end of a development cycle, like Firefox reaching 1.0, several test releases are issued. Sometimes they are called release candidates (RC) or other names.

If you check out the Mozilla project roadmap page, you'll see the milestone schedule near the bottom. 1.0 PR is actually before the RCs in this schedule, meaning that RE has planned a lot more time for testing and code stabilization before the final 1.0 release. There's a big Firefox marketing push is going on right now, so what gives?

It's very possible that the Mozilla team is increasing adoption early to find more bugs before the 1.0 release. There's an automatic bug reporting tool within Firefox that collects stack traces when errors occur. With more people using the product, they'll find more issues with it and be able to fix these issues in time for 1.0.

They'll also be able to get Firefox on a lot of different configurations. This is really important, because they can verify that Firefox does not conflict with applications that users are already using.

The risk they are taking is that if people have big problems with Firefox they could be completely turned off of it. Why would they use a buggy browser when they think IE works fine? They probably already think there is no reason to switch. They don't need Firefox to give them a reason to switch back to IE. Firefox usually doesn't have these quality problems though -- updates are usually for security problems or new features, not major defects.

I think this has a lot to do with the fact that Firefox is open source and many people are interested in the project. In order to get the kind of quality that Firefox enjoys, you need to have a lot of hackers looking at code and a good solid review process like the Mozilla team. Closed-source projects do not have an army of code auditors so they need better formal testing, like unit testing and a quality assurance (QA) team, to compensate.

When you have a high quality product like Firefox it makes sense to increase the number of people using and testing it before a major release. If your product were lower quality and you needed much more testing, you'd be taking a big chance that people would never use your product again because they ran into problems. Even though you might need a lot of testing on a lower quality project, you shouldn't try to get more end users. You should try to improve quality yourself and/or recruit testers that know the product is buggy. It would be a bad idea to displease users with a buggy product.

Is the Mozilla team taking a big risk increasing adoption this early? I don't think so. The quality of Firefox has always been quite high so they shouldn't run into too many quality problems. They will gain much more by increasing the adoption of their product and tweaking it before 1.0.

It really pays to keep your project in a high quality state all of the time with testing (or in Firefox's case, a lot of peer review). You can give it to any one at any time and it will just work and that's a great feeling. Having a high quality project also encourages high quality development, like the broken window theory talks about.

Go Firefox go.

update: I forgot to mention that release engineering teams can use processes like continuous integration (CI) to spot broken windows in the project as early as possible.

The CI process needs the right tools to work well, not just a compiler. You need a good unit testing framework, like JUnit, and a good code coverage analysis tool like jcoverage will help you find areas that haven't been haven't been tested well and are more likely to have defects. Other testing tools can also be used.

This is part of monitoring the health of the project, and it's the responsibility of RE and the project manager to ultimately ensure that the project is healthy.

It might seem as though the developers are working in a highly controlled way, but as PM it's your job to convince them that it's for the good of the project that it's monitored like this. There's still a lot of freedom for developers to do what they need to do, so long as they don't damage the project doing it. In fact because they can see if they break something after they take a risk, they have more freedom to take those risks.

posted at September 18, 2004 at 11:48 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (0)

# Adventures in Disabling VgaSave

This story is long and probably amusing to people who aren't me. I'm posting it for your enjoyment. Chalk this one up in the "worst personal IT experience so far" column...

I was at a friend's place and her roommate's DVD player on her notebook computer wasn't working properly. It said something like "requires DDraw, cannot use the current display mode".

So I thought maybe she didn't have the latest version of DirectX. I went to the Windows Update web site, and before I could check to see if she had installed DirectX I had to download a bunch of security updates. Install updates and reboot three times only to discover that she doesn't need a DirectX update. D'oh.

So I check the video driver. Blank. Blank? Yep. I click on Adapter --> Properties and it says "VgaSave". I was like WTF? So I go to the laptop's manufacturer (Acer) to get the real video drivers, try to install them and it says that the current drivers are in use, so I have to disable them to install the new ones. OK, so I disable VgaSave.

I still can't install the new drivers, and I install a notebook status tool from Acer thinking that maybe it can fix the driver problem. After installation it wants a restart, so I restart.

On reboot the machine shows the product screen (Acer), then the Windows loading bar animating and then nothing. Blackness. OK, I had no idea what was wrong at this point so I try to boot into safe mode. Nope, doesn't work. Command prompt? Nope. I can't do anything. Everything else still works though. I can hear the sound, and the computer is thinking like it's booting up.

So I look up VgaSave online and find this page on Wikipedia. Crap! I just screwed the pooch completely.

Luckily, the computer's owner has a good sense of humour (and some blind faith)! I look up how to fix this problem. Apparently VgaSave is a fallback video service for NT operating systems. When your video card doesn't work, or you use Safe Mode, it uses VgaSave. If you disable this service you have no fallback! That's where I was.

So I needed to re-enable the service and I read that I can do this through the NT Recovery Console (RC), which requires the administrator password. I boot up the RC and try a blank Administrator password. No go. I ask the notebook's owner for some possible passwords. No dice. I try stupid common ones like password, god, etc. Nope.

The last Windows install on this machine was done by CompuSmart, a computer repair shop in Ottawa. They would have set the Administrator password on install, probably to blank. But blank didn't work! So I search the net more info...

Apparently because of security, XP changed the way it stores passwords internally. This changed sometime between the original release of XP and now, though I couldn't pinpoint exactly when. The RC I was using was from the XP CD (SP 1a) and I thought maybe it read the passwords the old way. So I downloaded the bootdisk maker from Microsoft's web site to get the new RC. It's SIX DISKS and it takes about 5 minutes to make 'em all.

I make the six disks on another machine and take them to the notebook. Using boot disks takes a long time, probably ten minutes. I get to disk 5 and it won't read. I redo the disks with the same disks and disk 5 is still toast. I redo them AGAIN with a new disk for disk 5 and I get an error that a file on the disk was corrupted: means bad boot disk maker!

At this point I was calm on the outside, snapping on the inside and thanking God I'm not an IT administrator and actually do this for a living. At that point it was late so I gave up for then, and took the machine with me.

So today I'm still fooling around with it some more. The blank Adminstrator password doesn't work with XP Home boot disks. It doesn't work with XP Home SP1, XP Pro SP1, XP Home SP2 boot disks. I actually made the boot disks for them all. No dice.

I call up CompuSmart and ask them about the Administrator passwords for new systems. The dude said "we leave them blank". Ok. The blank passwords didn't work on any version of RC! I was stuck.

At this point, I was at my last resort: use a 3rd party utility to change the Adminstrator password. Microsoft obviously does not recommend doing this -- the only ways Microsoft recommends changing the Adminstrator password is through the GUI in XP or a password reset disk. Obviously I couldn't get in the GUI -- and who actually makes recovery disks? Not students. I was out of official options.

So I downloaded a neat little tool called the Offline NT Password and Registry Editor (ONTPRE). It actually boots into Linux(!), and edits the Windows registry, including the SAM file that contains all of the users and passwords.

So I get into ONTPRE and check out the users. Administrator is disabled and locked. Good lord, Compusmart lied to me. My urge to tear my hair out after this ordeal is wrestled to the ground by a feeling of relief. I didn't have to reformat. That is good news. Thank you, ONTPRE!

So I boot up RC (with the six boot disks, natch), use the blank Administrator password and voila, I'm in the RC. To enable VgaSave again, I just typed.


at the RC command line.

Rebooted the machine and everything is back to normal. Will I try to screw with this machine some more to actually get the drivers working properly? Hell yes, because now I know how to fix it if I screw up again. Ha. :)

update: after yet ANOTHER hiccup, this time installing XP service pack 2, I'm all done. I got an error midway through the installer and had an error dialog that wouldn't go away, so I had to force quit the installer. Then Windows Update thought I had the full SP2, so I had to uninstall the partial and redo the install again.

You should have seen the spyware that Ad Aware found on this machine. I've never seen that much on one computer before. Now IE is hiding and Firefox is on the desktop. If you don't want spyware or random popup windows, you should use Firefox too.

posted at September 17, 2004 at 01:42 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (31)

# Be Mindful of Code Entropy

Jim's been drinking the unit testing kool-aid. Awesome. I have a little more for him. :)

Seriously though, strictly speaking you shouldn't refactor just because the code *looks* icky. Here's the thing about unit tests: they can be just as buggy as the code they are testing. They can do a good job of pointing out regressions, but they will probably not get them all. It depends on the quality of the tests, which is hard to determine. (You're not going to test the tests, are you?)

In fact some teams, for the sake of agility and speed, only code unit tests for common or complicated situations/breakages and do not attempt to cover all possible regressions. If you have really great programmers, your team can get away with shortcuts like this.

You can easily refactor an "ugly" piece of working code into a beautiful piece of buggy code and be none the wiser. Even WORSE, is thinking that because you have unit tests you have NO BUGS in your newly refactored code. Ack! Unit testing should give you more confidence while you refactor but not that much confidence. :)

This is why extreme programming promotes refactoring only at the last possible moment. If you have a bunch of ugly but WORKING code, leave it until you absolutely need to refactor it to add a new feature or to make the code simpler to solve another bug. Then you won't risk adding bugs just making your code beautiful or fit some established pattern.

As a related aside, XP also promotes refactoring early in an iteration rather than later. This is why in XP you implement the "riskiest" bugs first, where part of the risk measurement is how much of my application's code will I have to refactor to add this feature?

The reason you don't want to refactor established code: this ugly working code has stood the test of time. It's been beaten down by your users, it's been tested to hell in the UI by manual testers and QA people determined to break it. A lot of work has gone into stabilizing that mess of code you hate.

As much as it might pain you to admit, if you refactor it you'll throw away all of the manual testing effort you put into it except the unit tests. The unit tests, as I said earlier, can be buggy and aren't nearly as comprehensive as dozens/hundreds/thousands of real man hours bashing against the application.

Developers especially have to resist the temptation to refactor code to "beautify" it at the end of an iteration or development cycle. They'll really be tempted to, but swat their hands. Playing with the code can just introduce new defects.

Bottom line: code entropy is your enemy and you should only refactor when absolutely necessary. The more code you can keep still and not stir up, the bettter. Don't let this scare you off from adding new features and refactoring, just keep in mind how much entropy you are introducing into your project.

Before you refactor, take a minute to think about how much testing effort it will take to get to the same quality level you're at right now despite your messy code. Is the refactor worth it?

posted at September 13, 2004 at 05:07 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (2)

# Return of the salty fruit

After much lamentation on my part about the removal of green olives from Subway's vegetable selection, I have good news: they're back. Atkins Schmatkins.

posted at September 13, 2004 at 03:01 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (2)

# My ears are ringing with the sound of muuuuusic

So many bands, so little money...

Sep 27: The Tea Party at CMH
Sep 30: The Stills at CMH
Oct 3: Billy Talent at CMH
Oct 4: Teitur at Barrymore's, Ottawa
Oct 9: The Killers at CMH
Oct 12: Interpol/The Secret Machines at Le Metropolis, Montreal
Nov 2: Muse at CMH

CMH = Capital Music Hall, Ottawa

posted at September 13, 2004 at 02:56 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (1)

# I'll take Python for 100, Alex

I'm starting to get more serious about learning a dynamically typed language. The main reason is perspective: if I am not personally using a language in this group, how can I expect to have an intelligent conversation about them, much less compare them to languages/platforms I am familiar with, like Java or C#/.NET.

There is also a geekness factor to using less-than-mainstream languages, as well as a curiousity about how the languages are apparently better suited to agile development methodologies. As a software engineer curious about process, I'd like to see how the structure of the language itself can affect project process, agility, quality and other aspects.

While I originally considered Smalltalk, Python seems like a better transition from what I'm already familiar with, mainly Java.

I still want to learn Smalltalk in the future though. Python has a niche that Smalltalk does not, for some reason. People are obviously choosing Python over Smalltalk or Ruby and I'd like to find how why. I can see the niche exists but I'm not able to describe it in any sort of useful way yet. As I attempt to understand the Python culture and its differences with the Smalltalk culture, I may write about it here.

Another point of consideration was Python's implementation in Java: Jython. Since it lets me use Java's extensive libraries while programming in Python, I can leverage all of the Java library knowledge I have and speed up the learning process.

The thing that concerns me about Python as a software engineer is the lack of well-established tools, as in the Java camp. This could very well be by design, or a perceived lack of need by Python programmers and the culture. It could also just be a reflection of Python's relatively young age and maturity.

I could discover that Python is very much a hacker's language like Perl, and it was never meant to be used on very large and highly organized projects. I could discover that this is just FUD, spread by the statically-typed language zealots. There are already large server-side projects like Zope written in Python.

I'd like to find the right tools, but I'll probably come across more as I'm working on Python. Here are the things I have in Java, and I'm looking for good Python counterparts. Listed under them a examples I have found, and I'll add to this list.

1. Integrated development environment (IDE)

Red Robin - Eclipse plugins

2. Unit testing framework

unittest AKA PyUnit

3. Project build tool

various ports of Ant

4. Code coverage analysis

code coverage is built into unittest (AKA PyUnit)

A sample project would also be useful in learning Python, just as AudioMan has been useful in learning Java/Eclipse/SWT/JFace. After going through some tutorials, especially Mark Pilgrim's Dive into Python, I will likely attempt to port jid3rL to Python.

posted at September 11, 2004 at 11:19 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (5)

# Why not inherently facilitate feed deltas?

Bob Wyman has a good summary about the RSS bandwidth situation and polling. It's interesting that he mentioned deltas and how this could save bandwidth.

What I'm really curious about is why feeds aren't organized to make sending deltas easier, rather than creating a delta system around existing feeds.

For example, a feed could be broken into a main index and feed text pages. The feed pages would contain the text of the blog posts in separate XML files. The main index would be a stripped down version of what an RSS feed looks like today: just post ids, titles, dates and pointers to the corresponding feed text pages.

A feed reader would use the main index to figure out which posts have been modified since the last time the feed was read. The reader could even be configured to not download the feed pages at all, like newsgroups, and just get them on demand when the user want to read the text. Obviously, that doesn't work well offline and doesn't even suit everyone's preferences. It's just an option.

No delta magic would have to be done to the RSS/Atom feed file on the server. If the reader already has a page in its cache, it would not need to read it again. You wouldn't even need to do HTTP conditional-get on the feed text, only the main index which is much much smaller. The main index would tell you if any of the feed pages have been updated, and the feed reader could fetch them.

I have a feeling though, that someone out there has a good reason why an RSS/Atom feed cannot be broken up like this. My burning question is: what is it? :)

posted at September 11, 2004 at 07:34 PM EST
last updated December 5-, 2005 at 05: 1 PM EST

»» permalink | comments (3)

# Time to Shift Gears Once Again

It's interesting -- with talk of Longhorn in the air, I thought AudioMan was doomed. The database that AudioMan maintains is essentially a layer of metadata above the file system accessible to one application: AudioMan.

WinFS, the proposed database-based file system for Longhorn has actual metadata built into it. Sure this may not be at the file system layer we are familiar with today, but it will be the file system abstraction that a Longhorn developer is presented with. This metadata is accessible by any program in the system, not just one. D'oh!

But then Longhorn got delayed, and WinFS got pushed out and initially I breathed a sigh of relief. AudioMan was safe. But just recently I've been thinking: what exactly is AudioMan supposed to do?

Players fill a defined niche, and collection browsers have become part of these players. It doesn't make sense to make another collection browser because they are already well done in players like iTunes and WinAmp. People aren't going to move away from the established players, and once they are using a player they don't want to have to exit that interface.

The angles of AudioMan that I liked were:

1. Complete collection management

Where are all of my files, not just the ones that play in iTunes because of DRM or Windows Media Player because of DRM or Real Player because of .... yep, DRM. I want a list of them ALL. There isn't a single player that displays all of your files because there isn't a single player that will play all of your files. Because of DRM, I doubt there will be in the near future. Competition will not allow it.

Collection management also includes backup management, a severely needed feature in not only music players (that pass as collection managers these days) but for the entire operating system! Sure hard drives are getting bigger, but so are backup media like writable DVDs. People want to know what's on ALL of their backup disks without having to load each one individually. Future operating systems will keep track of our backups for us, mostly so we can search all of them while they aren't in the disc drive and find our stuff!

2. Metadata

Complete metadata is the holy grail of data mining. In order to search for and find songs they have to be properly labelled. The bad thing is that labelling is a time-consuming and sometimes manual process. The situation is getting better because of services like CDDB, freedb and Musicbrainz, but it's still far from perfect.

Bottom line here: automate or the user won't take the time to do it. In other words it better be free, as in no effort. Without good metadata, searching is spotty and collection management becomes seriously hard.

I see two main issues in this space:

a) Identifying the actual audio in the recording and getting all of the information you can about it automatically.

b) Labelling the metadata and file name the way the user wants, so that the whole collection is consistent. This may also include moving files around into an established hierarchy, though with better searching technology well structured file system hierarchies are becoming less important.

3. File and tagging types

A tool that supports multiple file types has to support all of the file types' variations on storing metadata. It has to be able to read and write this metadata. Adding new file type support should be relatively straightforward (think of an Eclipse plugin-type model) but writing the support WELL will be the hard part. The emphasis should be on data integrity, with effective testing being paramount to just about everything else.

4. GUI

AudioMan's current user interface is better suited to browsing than organizing a collection. This is obviously a problem, because it puts users in a certain frame of mind. Users have associations with the iTunes interface that are hard to drive out.

It's like tools that copy the Windows Explorer interface, modify it slightly and wonder why people have problems managing the variations it introduces. It's seems to be more of a disadvantage than an advantage in being familiar, as it can severely limit development. You find yourself not wanting to stray from the legacy GUI when you add a few feature -- just for the sake of looking and feeling familiar -- and this is bad.


No regrets with AudioMan though, I think it's a pretty good little app. It's definitely not targetted for an audience, which is why it's not being used. Doesn't take a rocket surgeon to see that. :)

I will still use AudioMan as a platform for improving jid3rL and also to manage my own collection. Other than that I wouldn't expect this version of AudioMan to get any new features, it's a bit of a dead end. It's not as unfortunate as it sounds, really (if at all).

This project tought me a heck of a lot about project management, release management, quality, multi-threaded GUI applications, test driven development, the Java Collections framework, Java itself, Eclipse, SWT, code coverage and the list just goes on and on.

My main goal with AudioMan was to have a platform for discussion about software engineering. In that capacity it excelled. As a project however, it failed: there are only two people using it today. Software that's THAT custom isn't worth the time it took to develop it, but AudioMan was worth it because I learned so much from it.

Thanks to everyone that supported AudioMan over the past 2 years or so, especially the group that helped to start it all with me: Peter, Karen, Jim and Trevor. Here's to more learning and discussion in the future. Prost!

posted at September 11, 2004 at 01:13 AM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (2)

# RSS is doesn't scale? Numbers please.

There's bit a bit of blogger hubbub about MSDN not publishing full RSS feeds any more because of bandwidth issues. Looks like it only effects the "combined blogs" feed and all other individual blog feeds still have full text.

After some discussion on how others deal with bandwidth issues, James Robertson openly questioned if MSDN used conditional-get or compression features of HTTP to save bandwidth. In his comments Dare Obasanjo confirmed they indeed do use them.

But James was skeptical and cooked up some Smalltalk code to see for himself. Turns out MSDN doesn't use conditional-get or compression after all. No one from Microsoft has responded to this evidence yet.

Personally, I'm surprised no one has come up with some hard numbers.

How much bandwidth would MSDN save with compression and conditional-get? From what I understood, conditional-get doesn't save them anything with their combined feed (an RSS feed of all blogs put together), since it changes way too often. You have to pull down a new version every time.

For individual blogs, conditional-get would work well. To get some rough numbers, figure out the number of average posts per day per blog and the figure out how many requests per day on a blog are. Use this to find a rough percentage of how many could be returned as "not changed" (HTTP 304) by conditional-get and calculate bandwidth savings based on that. Heck, the server could keep track of not-changed responses (HTTP 304) vs. full RSS feed response and you could make exact calculations.

Compression on text like XML can be very good -- like 50%+ (try zipping up a few xml/html documents in WinZip and you'll see what I mean) and the longer a document is, the better it will compress because the dictionary is used more often. The difference between sending a terabyte and a half terabyte is significant.

Does this change any of the "scaling" issues of RSS? Documents will compress at the same % no matter how many people read them, so it scales at the same rate as uncompressed: linearly, with less overall volume.

Conditional-get also scales linearly as traffic increases, assuming all other things are equal. Often not all things are equal, and it does not scale linearly as users increase because of:

1. how often a person posts to their blog and changes the RSS feed (feed entropy)
2. how often each user's feed reader is pulling down the feed and how many times they get the full feed vs. a conditional-get "not changed" 304. If the server doesn't support conditional-get, the feed is redownloaded every time, increasing bandwidth costs.

Number 2 can be controlled like Slashdot. If a user reads Slashdot's RSS feed too often, the user is blocked temporarily. I like this solution because it puts the onus on the user to correct the behaviour of their feed reader. If they want to read the feed badly enough, they will fix their reader settings. This is good for slashdot because if you read the feed every 30 minutes or whatever the minimum is, 99.9% of the time it has changed. Conditional-get would buy slashdot almost nothing.

Unfortunately, that solution is not completely user-friendly. It would be nice if the server would tell the feed reader "hey, you can only read me once every 60 minutes!" and the feed reader corrected itself automatically for each feed. Maybe this already happens, I don't know.

Number 1 is controlled by the RSS feed author, what I like to call feed entropy. Conditional-get savings could be improved if a blogger made 5 posts a day all at the same time instead of spread out over the course of the day. Then feed readers would only have to get one version of the feed (the rest of the requests would return 304) instead of 5 new full feed downloads that day for that user. That's not a linear savings either, it's big.

Is it realistic to expect people to write blog posts like this? No. Could commercial sites like Salon and Wired take advantage of this? Sure, I bet they already do.

I'd really like it if someone in-the-know gave me some numbers and math so I could make an educated and informed decision about RSS/HTTP deadness and scalability. Until I see real numbers based on real traffic, I don't think I'll be completely convinced it's dead in the water.

posted at September 10, 2004 at 12:36 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (0)

# <junit> Ant task doesn't work in Eclipse

Here's another solution to a problem I'm hoping to get Googled: If you're running an Ant build.xml file in Eclipse, the <junit> task will not work by default. If you attempt to use the <junit> task with a fresh install of Eclipse, you will receive the following error message:

BUILD FAILED: [ECLIPSE_DIR]\workspace\[PROJECT_DIR]\build.xml:208: Could not create task or type of type: junit.

Ant could not find the task or a class this task relies upon.

This is common and has a number of causes; the usual
solutions are to read the manual pages then download and
install needed JAR files, or fix the build file:
- You have misspelt 'junit'.
Fix: check your spelling.
- The task needs an external JAR file to execute
and this is not found at the right place in the classpath.
Fix: check the documentation for dependencies.
Fix: declare the task.
- The task is an Ant optional task and optional.jar is absent
Fix: look for optional.jar in ANT_HOME/lib, download if needed
- The task was not built into optional.jar as dependent
libraries were not found at build time.
Fix: look in the JAR to verify, then rebuild with the needed
libraries, or download a release version from apache.org
- The build file was written for a later version of Ant
Fix: upgrade to at least the latest release version of Ant
- The task is not an Ant core or optional task
and needs to be declared using <taskdef>.

Remember that for JAR files to be visible to Ant tasks implemented
in ANT_HOME/lib, the files must be in the same directory or on the

Please neither file bug reports on this problem, nor email the
Ant mailing lists, until all of these causes have been explored,
as this is not an Ant bug.

The <junit> Ant task is an optional task and this is the standard Ant error message when Ant cannot find the supporting code for an optional task. Ant includes support for the tasks themselves, but not the internal code that actually does the work.

So you need to find a copy of junit.jar, the code that does JUnit work, and tell Ant where it is. The easiest way to do this is to go to Window --> Preferences, Ant --> Runtime. In the Classpath tab click on Global Entries and then Add External JARs....

Eclipse already has a copy of JUnit, so in the dialog find your Eclipse plugins directory ([ECLIPSE_DIR]/plugins/) and go to the JUnit plugin (presently org.junit_3.8.1) and select junit.jar. Now all of your Ant scripts will know how to do the <junit> Ant task.

posted at September 08, 2004 at 08:27 AM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (38)

# Blogging Might Get You Fired

It's dead easy to start a blog these days, and so we see more people doing it. Once they have a blog, people will talk about anything online -- it's almost as if they don't realise that anyone in the world with an Internet connection could read what they're writing.

Some bloggers might think that most people won't care about their insignificant little blog. This may be true until someone searches for a person, place or thing in a search engine and comes across your remarks on the subject. Google might put your blog front and center on a topic and there's not much you can do about other than deleting the blog post.

The global reach and the searchability of blogs is what gets people in trouble with their employers. Even though you might be blogging on your own time, what you say reflects on you as a person and ultimately reflects on your employer. If you talk about the details of your job expecting that no one will read them, that's taking a big risk. If you express an opinion that looks bad as an employee, you run the same risk.

It's not just blogging this applies to, it's just that everyday people aren't used to having a truly worldwide publishing medium like the Internet at their fingertips. They don't take the same care speaking publicly that, let's say, a Canadian Member of Parliament might use while speaking to the press each day. An MP is familiar with their publishing power, most bloggers don't seem to be aware of it.

If an employee wrote an editorial in a newspaper criticising their employer, they could very well be fired for it. Same for TV or radio. Why would blogs be treated any differently? It's not as simple as "blogging is something I do on my own time, so I can say whatever I want."

A recent example was a woman in Nunavut that was fired for candidly discussing the city on her blog. Normally this wouldn't raise eyebrows, right? But she was a tourism marketing officer for the city.

Rather than showing Nunavut as a squeeky clean tourist haven, she told it like it really was -- like posting pictures of a piles of rusted junk and criticising the bus service. The inside look at the city was pretty cool and she did this outside of her job, true ... but is this good marketing? Does it reflect well on the city?

A contract employee of Microsoft was fired for posting pictures of Macs sitting in a loading dock. An employee of Friendster was apparrently fired for blogging about their recent migration from Java to PHP server technologies.

The interesting thing about all three examples is that apparently none of them were warned -- they were just terminated. Surely there are bloggers that have been warned and removed offending posts, but the ones we hear about in the blogosphere are the ones with no warning at all.

The Internet is not a permanent medium necessarily; the posts can be removed and all of the attention in the blogging community avoided. There are things like the Wayback Machine that permanently record websites at a certain time, but it's possible a post can be removed before it does a lot of reputation damage.

Maybe this is a better road to take for employers, rather than seeming like bullies and drawing attention to the offending posts, which are often left up on the Internet after the firing anyway. Or maybe not. Maybe once the post is up the damage has been done. I'd be interested in a lawyer's opinion on that.

These firings aren't going to let up I don't think -- they'll be a regular occurrence. As the number of people with blogs increases exponentially, less bloggers will know about these firing precedents and the care they should take while blogging.

Some people don't know what's appropriate to blog about and what isn't. Maybe their managers should tell them, yeah, or maybe it should be explicitly in their contracts. Maybe employers should get employees to sign NDAs all of the time. That will make it nice and explicit and save employers a lot of trouble while saving employees from being fired. Unfortunately, all of that paper and legalese costs money.

But for employees of companies who don't make a blogging policy explicit, I suggest you tread lightly. Google, the Wayback Machine and others are watching you, even if your boss isn't.

posted at September 05, 2004 at 12:50 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (6)

# MT-Blacklist Kicks Ass

In just over a week, MT-Blacklist has already automatically blocked over 430 spam comments for this blog. I only had to moderate one by hand and I was notified by email.

MT-Blacklist, a Movable Type plugin, uses a list of keywords and URLs to block comments. This works because spammers are trying to improve Google rankings of certain web sites by flooding comments with links, so the target URLs are obvious. The keywords are shared by the MT-Blacklist plugin and distributed around to other MT-Blacklist installs so that comments containing the keywords are automatically blocked for other people. I suppose you could consider it a kind of distributed moderation -- and it's working really well!

It works so well that the author won a contest for best Movable Type plugin and got a brand new Apple G5, which he sold to pay off debts.

There are email spam blockers that also work on the same principle. The more information that is moderated and shared the smarter the filter gets, which leads to less false positives. It sure is better than an isolated spam filter: the whole network can respond almost at the same time.

posted at September 01, 2004 at 05:15 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (2)

# Java New I/O for Abstractions and ... Perf? Yes.

If you were used to dealing with Java I/O the old way, the New I/O APIs in Java 1.4 are really handy.

The first is the notion of channels. I use a FileChannel to copy the bytes of a source file to a destination file. The API is dead simple to use too:

RandomAccessFile s = new RandomAccessFile(sourceFile, "r");
RandomAccessFile d = new RandomAccessFile(destFile, "rw");

FileChannel source = s.getChannel();
FileChannel destination = d.getChannel();

source.transferTo(start, count, destination);

You don't have to read a chunk of bytes from the source file and write them to the destination file in a while loop -- it's all done for you. Roy has been testing this code for me and he says it's really fast too. Sweet.

Another nice class in nio is ByteBuffer, which I used in jid3rL. I also used his brother CharBuffer to read encoded buffers of chars[1]. A ByteBuffer is just a wrapper around a byte[], and a CharBuffer is a wrapper around a char[]. You can use the Charset class to decode a ByteBuffer to a CharBuffer or encode a CharBuffer to a ByteBuffer in a specific character encoding.

Buffers also keep track of where you are in the wrapped array. You can make your way through the buffer, pass it around to other methods and the current position and array length are passed as well. This makes ByteBuffer really handy for parsing bytes, and CharBuffer for parsing chars.

In particular I simplified my frame parsing code a lot by using a ByteBuffer and a method I made to pop the next null-terminated string off the front of it. It cleans up the frame parsing code nicely: I'm not looping through bytes looking for null in my frame parsing code any more, it's all abstracted. The great thing about this abstraction of course is that (encoded) String parsing is centralized now, leading to less errors in frame parsing code itself.

This is the intermediate step towards specifying frame fields with constants, like I mentioned before. These constants will specify for each frame field the type expected, it's maximum and minimum length, etc. Then you put it in a general frame field parser that understands how to parse all of the field types to objects from a byte[]. You get an array of parsed Objects back.

If all of the field values are in an Object array in the Frame subclasses, then the Frame superclass can handle general implementations of equals(), hashCode() and toString() and other methods. Not only will this save me from implementing all of these methods in the subclasses (there will be about forty), but it will also save me from testing them too.

Thinking about testing is often encouragement to get rid of code duplication, because if you have code duplication you'll end up with test duplication too. Who wants to write more tests then they have to?

[1] id3v2 supports Unicode UTF-8 and UTF-16.

posted at September 01, 2004 at 04:57 PM EST
last updated December 5-, 2005 at 02: 2 PM EST

»» permalink | comments (2)

Search scope: Web ryanlowe.ca