«« Metadata in Read-Only Files Wanted: Graphic Designer for AudioMan »»
blog header image
Database Conversion on Software Updates

The database in AudioMan is a bit of a mixed blessing. It allows quick access and searching of metadata but it can also go out of sync with the file system. This is because AudioMan operates one level above the file system, not within it.

Another database stumbling block was introduced by the support for removable media: database conversion between software upgrades. When a user gets the new version of AudioMan, the database needs to be converted from the old version if the schema has changed. Now that all of the removable media data is stored only in the database (and not in files as well) the database contents are increasingly important and must be preserved between upgrades.

Database schemas morph as features are added, removed and tweaked so a strategy is necessary to manage this change. A flexible database schema helps, but it's not always enough to see changes into the future.

Something Roy and I were talking about using XML as a middle man. On a database conversion, the contents of the database could be exported to XML from the old schema and then imported from XML to the new schema. Does XML offer more flexibility than going from old database to new database? That's an interesting question because the XML format itself might have to change too!

Roy wants me to export the collection as XML anyway so he can do some neat transforms on the data, like making a nice HTML list. So I'll have this XML file handy anyway.

By the way, I've thought of a better phrase than removable media. How about backup disc? In the top menu, the menu item would be Add backup disc....

Posted at July 15, 2004 at 04:20 PM EST
Last updated July 15, 2004 at 04:20 PM EST
Comments

In case you don't want to go just the xml route, I would recommend that we you go from one db structure to another, it is done in many steps. i.e. don't assume that your users are jumping *only one* version. They might jump 5 levels.

I guess in this case, you should also have a table in the db that stores info about what schema version it is.

Just so that you can always apply each transition in order. Having tests that changed your data in the db from each starting db structure to the next to the next... to the newest would ensure that your "upgrades" always worked.

This sounds like a bit of work, but it'll probably pay off before you know it.

» Posted by: Jim at July 15, 2004 04:53 PM

Yep, that's exactly what I was thinking. But suppose the XML format was more flexible and it never changed when the database schema changed 5 times.

Then to go from any version to db version 5, you just convert to XML and import into db5. The other way if you wanted to go from version 1 you'd have to convert from db1 to db2 to db3 to db4 to db5. This way you could just go db1 to XML to db5.

Now that's all fine in theory, but ... is XML actually more flexible? It wouldn't have to have the same layout as the database, so technically it could have a more flexible/robust "schema".

These are the questions we'll have to answer! :)

» Posted by: Ryan at July 15, 2004 05:01 PM
Google
 
Search scope: Web ryanlowe.ca