«« Automatic Object Test Generation jid3rL Frame Storage Types »»
blog header image
id3v2 Frame Gigantism

Now that I'm done doing simple id3v2 tags with jid3v2, a small hurdle has come up. Images. Up until now, I've been storing the bytes of frames in byte arrays. When you get to images, which can be 100 kilobytes or more, then you start chewing up RAM with that kind of code.

Images aren't the only frames with this problem. Technically in id3v2 the frames have very high length limits, which are specified in the frame header and differ by id3 version. 2.2 uses three bytes, 2.3 uses four bytes and 2.4 uses four sync-safe bytes. An effective library should be able to handle these limits.

2.2: 3x8 = 24 bits, a range of 0 to 16,777,216
2.3: 4x8 = 32 bits, a range of 0 to 4,294,967,296**
2.4: 4x7 = 28 bits, a range of 0 to 268,435,456

**Note that the tag length is also a syncsafe 4 byte number (28 bits), which is the total length of all of the frames. So a frame with a length over 2^28 bytes isn't possible.

Those are some big maximum frame sizes. The way I see it, I have two options for these large frames. The first is to make a copy of these long frames and store as files somewhere; maybe somewhere in the user's temp directory, maybe a directory that you specify to the library that's in your application's directory. Then if you need to write that frame to a tag, you copy that frame's file over to the new tag. An obvious disadvantage of this approach is that it takes time to copy bytes like this.

The second option is to make a reference to the frame: the file, the starting position (offset) and the length. Then you don't have to copy the data in the frame to a temp directory in between. The disadvantage is that you can't move the source file, or the frame reference points to a missing file and the frame can't be written -- d'oh.

This might make you question the lifecycle of a Tag2 object produced from the jid3rL library. Will it be used right away, or will it have to stay around for a long time? In the case of AudioMan, the data is transferred to another object almost immediately, and the Tag2 is discarded. But as a library writer I don't think I can assume that will always be the case, which seems to make the temp file option a better choice. Thoughts?

Posted at August 18, 2004 at 08:48 PM EST
Last updated August 18, 2004 at 08:48 PM EST
Comments

This is a tough one. Gonna have to think about this.

» Posted by: roy at August 19, 2004 01:55 AM

If you get it working right, i'd love to start storing songs as metadata.

» Posted by: Kibbee at August 19, 2004 07:34 AM

what? Storing songs as metadata? Metadata for what? Do you want to describe a song with a song?

» Posted by: Jim at August 19, 2004 08:33 AM

Hehe, yes I'm also confused. :)

» Posted by: Ryan at August 19, 2004 08:56 AM

Either way, it looks like a cap on frame sizes will have to be made. Maybe I'll only support frame sizes of 1 Megabyte as the default, which could be changed by a parameter.

I'm hesitating about giving this library settings and "state", though. It just makes it that much more complicated.

» Posted by: Ryan at August 19, 2004 09:11 AM


I would do the *pointer* within the file - making sure that you have appropriate checks to let the library user know what has gone wrong (i.e. someone moves the file, someone changed the permissions on the file, etc.)

Now, if your own application were to move the files around - then it should clean up after itself and keep those references valid.

» Posted by: dru at August 25, 2004 11:32 PM

The frame could move by as little as a byte by the time you read it as a file pointer and then use that file pointer. It could also disappear completely, so I have to recheck that it still exists. If it does, it can move and everything is still OK.

The only problem with "cleaning up after myself" is that these files are in the wild -- not in a closed system. I can't control what other applications do with these tags, so I always have to assume the worst. I can't assume that a file and an offset will always be valid.

» Posted by: Ryan at August 25, 2004 11:37 PM
Google
 
Search scope: Web ryanlowe.ca