«« Enough Talk: id3v2test Begins A Better View on Testing »»
blog header image
A First Look at id3v2test.xml

Here's what id3v2test's text files look like so far. You can click on the permalink below to see it on a wider page.

<?xml version="1.0" encoding="UTF-8"?>
<ID3v2TestMethod author="Ryan Lowe" description="">
  <!-- test comment (ignored) -->
  <ValidateTag version="2.3" filename="ozzy.tag">
    <AssertTextInformation id="TYER">1983</AssertTextInformation>
    <AssertTextInformation id="TMED">DIG</AssertTextInformation>
    <AssertTextInformation id="TCON">Metal</AssertTextInformation>
    <AssertTextInformation id="TLEN">257358</AssertTextInformation>
    <AssertTextInformation id="TRCK">01</AssertTextInformation>
    <AssertTextInformation id="TALB">Bark At The Moon</AssertTextInformation>
    <AssertTextInformation id="TPE1">Ozzy Osbourne</AssertTextInformation>
    <AssertTextInformation id="TIT2">Bark At The Moon</AssertTextInformation>
    <AssertComments description="MusicMatch_Tempo">None</AssertComments>
    <AssertComments description="MusicMatch_Mood">None</AssertComments>
    <AssertComments description="MusicMatch_Situation">None</AssertComments>
    <AssertComments description="MusicMatch_Preference">None</AssertComments>
    <!-- more ignored comments...
      Still need to check for:
      Unsupported Frame WOAR of size 20
      Unsupported Frame APIC of size 2860
    -->
  </ValidateTag>
</ID3v2TestMethod>

ValidateTag is the first and only activity so far. Other ones could modify the tag, create a new tag, etc.

The two Assert types generate assert checks in the unit testing code, verifying that certain conditions are true. Each frame type will have to be verified differently, so they'll all have their own Assert tags.

This file will generate a single acceptance test method. What's the advantage of doing this in XML?

  1. The big one: centralizing acceptance testing for a spec. Different languages can implement their own unit test generators based on this data and everyone can share the acceptance tests.
  2. The person making the test doesn't have to know how to program. Whether this format is more readable than code is debatable. :)
  3. The format of the XML can be validated. Lots can be done here: make sure tags aren't missing, attributes are the right types, etc. Could even make an Eclipse editor like Ant's.
  4. The test methods can be made by lots of different people without having to worry about sharing Java unit test case .java files. The methods can also be moved around easier within the test suite.

Disadvantages...

  1. You have to regenerate the unit tests when you change any of the suite's XML files.
  2. Not to mention you have to find or make a test suite generator for your language/test suite.
  3. The XML format is brittle: not as easily refactored as say Java in a good IDE like Eclipse. If the XML format changes or becomes more strict, you have to change all of the old acceptance test XML files that violate the new restriction. So think ahead there a bit and plan for less refactorability.

As always, comments are welcome.

Posted at September 26, 2004 at 10:56 PM EST
Last updated September 26, 2004 at 10:56 PM EST
Comments


I have not followed everything - but you may want to look into something like fitnesse for acceptance testing... either for design ideas, or perhaps more tightly coupled integration.

» Posted by: andrew at September 28, 2004 01:32 PM

What do you mean by "tightly coupled integration"?

The problem with tying acceptance tests to a specific product like Fitnesse is that maybe I can't run them on platform X or with programming language Y.

» Posted by: Ryan at September 28, 2004 01:41 PM

Hence the tight coupling.

Examples would be...

1) parsing fitnesse files (either reading or writing)
2) using fitnesse business objects (importing the core framework for example)

As you said, doing so can limit the 'run everywhere' approach... we can talk more in person so I don't seem like I am mincing words (just don't feel like typing right now).

» Posted by: andrew at September 28, 2004 05:00 PM
Google
 
Search scope: Web ryanlowe.ca