|« June 2004||August 2004 »|
I'm Ryan Lowe, a Software Engineering graduate living in Ottawa, Canada. I like agile software development and Ruby on Rails.
I write this blog in Canadian English and don't use a spell checker. Typos happen.
» Full-time Ruby on Rails freelancer
» Full-time with Rails since May 2005
» Former committer for RadRails (now Aptana)
» I also have a few Rails side-projects in development:
1. wheretogoinTO.com Toronto nightlife
2. Hey Heads Up! TODO list and sharing
3. Layered Genealogy family history research
4. foos for foosball scoring
5. fanconcert for music fans (on hold)
Hiring Rails developers? I can telecommute by the hour from Ottawa, Canada
»» Email: rails AT ryanlowe DOT ca
Now hosted on Hey! Heads Up -- check it out!
Derek Lowe's (Ryan's older brother) words at Ryan's funeral
email@example.com no more
Forging Email Headers: Good, Bad or Ugly?
Sarcastic Dictionary (Part 1 of Many)
Twisting Rails is Risky Business
Risky Business? My Take on Early Alphas
Whoa, it's August 2007
A Postscript to "Growth at the grassroots"
»» All Blog Posts
David Heinemeier Hansson
James Duncan Davidson
Signal vs. Noise
Amy Hoy: (24)slash7
Luis de la Rosa
# The Java Recycle Bin RFE I Sent to Sun
This is the submission text verbatim, except that the email addresses have been somewhat spam-protected, the long link at the bottom incorportated into the text and some formatting to clean it up a bit -- Ryan
Update 6:39 AM Friday: here's a link to this RFE, though the email said that it may take a few days to show up. You also need a login to access the bug database.
This review ID is NOT visible on the "Java Developer Connection" (JDC).
We greatly appreciate your interest in improving the quality
Please be aware that the large volume of reports we receive
We currently have a three week response time for responding to
If you just reported an issue that could have a major
A DESCRIPTION OF THE REQUEST :
PLEASE NOTE: this problem is not specific to Windows. This RFE is to add a method to the Java spec, so that a cross-platform solution is available to Java developers.
java.io.File lacks a method to send a file to the operating system's Recycle Bin or Trash Can. Since the "Recycle Bin" paradigm has been adopted by several major operating systems (Windows, Mac OS X and Linux) it would be great to have a method in the File class to do this.
This RFE does not suggest changing the delete() method in any way. In RFE bug 4641147 that type of enhancement was suggested and I agree that will "break crossplatform compatibility".
Instead this RFE suggests that a new method be made, named something like moveToRecycleBin() or moveToTrash() in java.io.File. This method would use OS-specific calls to send the file to the Recycle Bin so it can be restored/undeleted by the user.
Many modern operating systems support the "Recycle Bin" paradigm. In order for Java to succeed on the desktop, J2SE developers need to have desktop-specific methods available to us.
In desktop software, there is a need sometimes for a simulated "file browser" in a view where file operations take place like rename/move/open/delete. It's quite common to have this in a IDE. In these file browsers, user's expect the operations to act just like the native file browser available to them in the desktop environment. On Windows, for example, this is Windows Explorer.
So in these file browsers when a user deletes a file, they expect it to show up in the Recycle Bin. This is typical for Windows applications like Visual Studio .NET 2003.
EXPECTED VERSUS ACTUAL BEHAVIOR :
A call to moveToRecycleBin() or moveToTrash() would move the file from its current location to the operating system-specific Recycle Bin where it may be successfully recovered by the user.
Like delete(), the method could return boolean indicating success or failure.
On platforms where a move to Recycle Bin may not make sense (like on some Unix operating systems without a desktop environment or J2EE), then a simple call-through to delete() would suffice because that is the expected behaviour on a platform without a Recycle Bin.
Since there are operating system specific tasks that need to be done during the move to the Recycle Bin, this ability is currently not available to Java developers. It is not as simple as moving the file to the (hidden) Recycle Bin directory on Windows, for example.
On Windows, the Win32 API shell function SHFileOperation() does a move to Recycle Bin. This would have to be called via JNI from Java.
# Pat Myself on the Feedback?
I've been getting some comments in meatspace about my blog from friends and relatives. The first thing was that a lot of my friends are reading it, some even without my knowledge. This is pretty cool -- I like that people can stumble upon my blog and I after months or years of not seeing me and read about me. It's easy and commitment free in a here's what I'm doing read it if you want kind of way. It's ... decidedly self-centered in its satisfaction too, but that's what blogs are all about and real bloggers won't have a problem admitting it.
The other comment is that I blog too much about technical stuff and AudioMan. I know I do this though, because this is my professional blog. I blog about tech issues and my projects to join in geeky technical discussions with other geeks. I also have this blog's address on my resume so that maybe one day I'll be magically whisked away to a dream job by someone that notices this place and likes my bad blog post title puns. Yeah, probably not.
So maybe there's a gap there. Maybe I do need a personal blog to talk about what I'm doing/thinking outside of high tech. I don't know about you, but I'm thinking that blog would be pretty boring! ... and yes, even more boring than this one.
Why? Because this is pretty much all I do that's safe to talk about on the Internet. :P heh ... and mystery is fun. I get to keep a little for myself.
PS> the new picture is a blog custom I meant to do earlier. It's like a picture in a newspaper byline signifying an editorial -- a face to the name, there ya go.
# Java Delete to Recycle Bin
It's interesting that even a small shift in the desktop paradigm hasn't been picked up by Java: the Recycle Bin. All of the popular operating systems have a variation of it but the only thing you can do in Java is permanently delete a file.
This is pretty inconvenient for AudioMan, which is trying to mimic Windows Explorer in some ways. So when I delete a track I want it to show up in the Recycle Bin. It's not as simple as moving the file to the Recycle Bin folder, which is hidden. There are other things that have to be done -- on Windows it's accessible via a Win32 API call: SHFileOperation().
So far the only solution I've found is to use JNI but that's not appealing to me as a Java developer. Wouldn't it be nice if SWT did this for me on Windows, Mac OS and Linux like it does for opening files? Eventually the Java library should do it instead, but the SWT guys might be a bit faster implementing it.
Maybe I should just write a patch for SWT myself ... I'll have to dust off my copy of Visual C++ 6.0. Nevermind that, how do I implement it on Mac OS and Linux? Oh boy ...
Update Monday 17:36: I submitted a bug report for "Move to Recycle Bin" to SWT and they replied that it is out of their scope, forwarding it to the Eclipse platform resources component. That's pretty reasonable, and wasn't completely unexpected.
They also suggested I ask Sun about including it in the File class (it's not in the JavaDocs for Java 5). Yeah, wouldn't that be nice? But who do I talk to about that? Is there a bug database? Yep, but I need to register. Are there Java bloggers I can talk to directly to find out why this hasn't been done yet? Maybe the answer is really simple. Does a JSR need to be written for something this small? I'm in pretty foreign territory here. :) Maybe Tim Bray knows someone I should talk to ....
... more legwork necessary. I'll keep y'all updated.
Sun has a bug for this (login required) but hasn't moved on it. The person reporting the bug wanted to replace delete() and deleteOnExit() with methods that send to the recycle bin. I agree with the rest of the people in the discussion that this is a bad idea. The top of the bug description says "Will break crossplatform compatibility. Wont fix."
I agree, don't change delete(). The File class just needs one method moveToRecycleBin() or moveToTrash() that can be used instead of delete(). For deletes in the GUI where the user expects to see the files moved to the recycle bin, it works. For other deletes (ie. configuration files that the user isn't interested in) you need the permanent delete. They both have their place.
Another interesting idea is to write a new class that manages a recycle bin -- moving files there, listing them and then recovering them. Now that all of the major platforms have the recycle bin paradigm, Java should get on the bandwagon and give us cross platform access to it, don't you think?
Update Thursday July 29 10:04 PM: I have submitted an RFE bug to Sun for this. I'd link to it, but it won't be visible on the bug site until it is reviewed. For Sun people who might be reading this, the RFE has the internal review ID 290373.
Update Monday August 1 10:35 AM: Here's a link to the RFE I submitted to Sun about this. If you want to see it in Java, you should vote for it.
# Search Text Box
In hindsight, I never should have kept AudioMan's old collection-based repository/database that long, because it affected the agility of the project. Every new "query" I wanted was a big task. I had delayed replacing it until after the huge MVC rewrite so that I had a known stable repository to work with -- it was about the only thing that didn't change.
Only when the repository started to limit the addition of almost every new feature did I actually replace it with a real relational database: hsqldb. So the project foundered a bit until the database swap.
Now the database allows me to mine the data a *lot* better. To show this off, I've added search to AudioMan. You select a source and then type in a few words into the search text box. AudioMan finds all of the tracks that match the search as you type. This also works on tracks on backed up discs that you've scanned into AudioMan, making it really easy to find where you backed up your stuff.
Check out the search text box in the latest build of AudioMan (nightly or integration, check the build times).
# AudioMan in the Medium-Term
With the ability to add backup discs to your collection, AudioMan is beginning to differentiate itself from file browsers and music players. AudioMan is more about overall collection management and organization.
I want to know where everything is, I want to search it, I want to know if all of my music is backed up. As people start collecting more and more digital music, these are the problems they will run into.
Right now I'm stabilizing and improving code coverage on AudioMan for the next development release (0.5.1). However the nightly builds are pretty stable -- go try it out and tell me what you think!
For the future, here's a tentative medium-term plan for AudioMan, in order:
I want iTunes-style find-as-you-type searching. The search will take place on whatever sources are selected -- the Whole Collection, just the local files On the Computer, some playlists and/or some backup discs.
If you want to know what disc you backed up a certain album on, searching will be a lot easier than browsing -- especially for large collections.
2. id3v2 Support
This has been the thorn in AudioMan's side for quite a while. I don't think I have much interest in implementing my own id3v2 Java library; it's a lot of work in itself.
I'm going to start evaluating 3rd-party Java libraries to do this job very soon. I want to get this in!
3. Using TRM to Uniquely Identify Audio
TRM is going to make AudioMan a really great application. TRM produces an identification number, which seems to be like a hash code. The TRM number is based on the actual audio in the track, so even if the metadata or the file size changes, two files can be found to contain the same audio just by comparing their TRM numbers.
Once you get TRM numbers for every song in your collection, including backup discs, it's quite trivial to determine which tracks are backed up, and which ones aren't. Then the user can get a list of these unbacked-up tracks and burn them to a CD for safe keeping.
It will also help you see if you have duplicates of the same files on your hard drive, so you can free up disc space. As well, you'll be able to make sure that files you take off your hard drives are already backed up, or they'll be lost forever.
The sky is the limit here, people -- what would you like to see in AudioMan?
# Wanted: Graphic Designer for AudioMan
AudioMan needs the help of a graphic designer to make it look nice. A truth of software development is that people really care about what software looks like, and that includes pretty icons. The (sometimes implicit) thinking goes: if you don't care about how it looks, what else don't you care about? The same rule explains why having an attractive web site for your company is really really important.
So I need some original icons to start, from a capable graphic designer. Maybe even some help with layout and usability, though I don't think I'll find all three of those things in one person. I'm pretty picky about what I like and what I don't -- and since AudioMan is my show, I get the final say. Impress me! I'll listen to a reasoned argument for a cause.
What's in it for you? Well, if you're a graphic designer you'll have a canvas. A real, living, breathing piece of software that at least two people (Roy and I) will use. This software will never disappear because it is GPL -- you can keep a copy of the source code yourself if you like.
You can retain ownership of copyright of the graphics, so long as I am free to use them in AudioMan free of charge. I'll have to look into the copyright/ownership/licensing issues a bit more but everything should be cool. You'll also get credit in the About dialog for your hard work.
If you've never seen AudioMan, go ahead and download it. You'll want to take a look at the latest nightly or integration build since that's where all of the newest work is.
Right now I need icons for sources (Whole Collection, On this Computer, Playlist, Backup Disc), tracks (Read Only, Missing, Backup, Low Quality) and a search icon. All of them are 16x16 pixel GIF files with transparency.
Please leave a comment if you are interested. A link to an online portfolio of work would be great.
# Database Conversion on Software Updates
The database in AudioMan is a bit of a mixed blessing. It allows quick access and searching of metadata but it can also go out of sync with the file system. This is because AudioMan operates one level above the file system, not within it.
Another database stumbling block was introduced by the support for removable media: database conversion between software upgrades. When a user gets the new version of AudioMan, the database needs to be converted from the old version if the schema has changed. Now that all of the removable media data is stored only in the database (and not in files as well) the database contents are increasingly important and must be preserved between upgrades.
Database schemas morph as features are added, removed and tweaked so a strategy is necessary to manage this change. A flexible database schema helps, but it's not always enough to see changes into the future.
Something Roy and I were talking about using XML as a middle man. On a database conversion, the contents of the database could be exported to XML from the old schema and then imported from XML to the new schema. Does XML offer more flexibility than going from old database to new database? That's an interesting question because the XML format itself might have to change too!
Roy wants me to export the collection as XML anyway so he can do some neat transforms on the data, like making a nice HTML list. So I'll have this XML file handy anyway.
By the way, I've thought of a better phrase than removable media. How about backup disc? In the top menu, the menu item would be Add backup disc....
# Metadata in Read-Only Files
While I was thinking about removable media support in AudioMan an interesting problem occurred to me. Removable media could be read-only things like burned CD-R, CD-RW or DVDs. It could also be read-write media, like flash memory or an external hard drive. People also have the ability to add to a burned disc by leaving the "session" open.
The main thing though is that once a song is burned on a disc its metadata cannot change. If the metadata wasn't entered correctly before the disc was burned then you are up the creek -- that makes sorting/browsing/searching the files more difficult.
So it might be nice to be able to enter this information in AudioMan anyway, even though its not the same as the metadata in the file. That way you'll be able to browse and search your collection better, without having to worry about missing or incomplete metadata. The information will only be stored in AudioMan's database.
Would it be useful to display the actual read-only partial or missing metadata as well? I'm thinking it might be useless after you enter updated information.
# Removable Media Support
Besides some minor refactoring, I'm in a pretty good position with AudioMan to add the first distinguishing feature, except I don't have a very good name for it. So far I'm calling it "removable media support". Yeah, doesn't quite have a good ring to it.
Anyway, it will let you scan in your burned CDs of MP3s that you use for backups. Then you can find and browse songs in AudioMan that aren't on your hard drives. For serious audiophiles like myself, there can be 50 of these CDs or more. It's nice to know where your stuff is!
After this is added you can view the whole collection (everything) in AudioMan, just the local files on your hard drive, or browse each of the CDs you have. In the future I'd also like to be able to identify songs that haven't been burned yet, so you know what to back up next.
I'll be working on this for the next week or so.
BTW, the switch to hsqldb as a database went relatively well. A few minor hiccups, but it definitely makes you appreciate how nice a seamless interface is! Should have planned a bit better. You can download a nightly build with the new database or just wait until 0.5.1 is released in a week or two.
# June CD Run Review
Here's a review of the CDs I bought for June, and have subsequently dominated my iTunes listening.
Franz Ferdinand - [self titled]
Continuing my penchant for quirky sounding rock bands is Franz Ferdinand's self-titled effort. It's probably the drumming that makes me like this band so much -- like the tempo changes in Take Me Out. What's interesting for North American listeners, is that the band said they wanted to make "music girls could dance to", and it doesn't sound that dancable at first. Just shows how far apart we really are, eh? Fans of Montreal's The Stills and New York's Interpol will like this album too.
Favourite tracks: 40', Take Me Out, Auf Achse, Come On Home, The Dark of the Matinée.
Thornley - Come Again
Big Wreck fans will be right at home with this album, which I bought early so I could get famliar with it before the Thornley concert I went to. The album is very solid -- after at least 14 listens I'm still going strong. Chris expected a bit more from Thornley, but I was pacified with the mainstream hard rock sound. Much to Chris' dismay, Thornley thanks Chad Kroeger of Nickelback in his liner notes. I never liked Nickelback's sound enough to buy one of their albums (they are on the radio enough anyway) but Thornley's album turned out to be thankfully less/subtly Kroeger/Nickelback influenced and more Chris Cornell/Soundgarden influenced.
Favourite tracks: Bright Side, The Going Rate (My Fix), Keep a Good Man Down, Easy Comes, Come Again, Falling to Pieces, So Far So Good
Morrissey, you are the Quarry
I feel guilty for not listening to enough Morrissey this month, but I just wasn't in the mood I guess -- and I'm the kind of person that listens to music based on my mood. So I'll save this review for next month.
Muse - Absolution
I like surprises, and Muse came out of nowhere in the middle of June from my friend Chris, who liked the harder songs Hysteria and Stockholme Syndrome and recommended them. The mostly hard album is counterbalanced by some really nice ballads, even a piano solo -- territory that can only be pulled off successfully by a brave UK band. The lead singer's voice is definitely Thom Yorke-like, but the Radiohead comparisons should stop there -- the rest of the band is much different. Apparently the band is quite popular in the UK but just hasn't caught on in North America yet. Expect that to change very soon; Muse's hard sound will be enjoyed by fans of brit bands Radiohead and Coldplay alike. I look forward to hearing earlier (and apparently harder) Muse albums.
Favourite tracks: Falling Away with You, Interlude, Endlessly, Hysteria, Stockholm Syndrome, Butterflies and Hurricanes
# Mac OS X 10.4 Tiger Announced
As usual, Steve Jobs' keynote presentation at the WWDC in San Francisco did not disappoint. Most of the announcements focused on the next major release of the Mac OS X operating system, named Tiger (10.4). If you have an hour watch the demos -- they are best seen, not read about.
Out of all of the stuff announced for Tiger, the searching API named Spotlight will have the most impact on users. It is quite interesting that not long after (in software development terms) Microsoft announced its intentions for Longhorn that the advanced searching features have appeared in Mac OS X.
Can we fairly attribute this to Microsoft's openness about Longhorn as a competitive disadvantage? I think partially yes, but we could also argue that given the mainstream success of Internet search engines that better searching technology on the desktop would become the next logical step. Using this reasoning one could argue Microsoft just telegraphed an obvious direction and it won't hurt them.
Nevertheless, Apple is goading Microsoft at the WWDC, which Steve says is a little "friendly competition". Best not get too "friendly" with the developer of the most popular office suite, eh Steve? They may bite back. The tone of the posters is a little over-the-top I think, instead of Apple's usual and preferable passive aggressiveness.
Who knows, Apple could have already been heading in this direction without Longhorn's inspiration, but Steve sure feels good about beating Microsoft to the punch by at least a year.
So what is Spotlight then? It's an API for searching files based on content. Indexes, which are maintained at a level just above the file system presumably, are used to retrieve files on the system matching search terms. The demo found files pretty quickly in a system with a good number of files in it.
Indexes, which Google and other Internet search engines also use to match search words with web pages, are created by taking the words of a document and pointing them to that document. Do this for every document in the file system, combine them and now you have an easy way to find documents containing a given word.
This is pretty good for file containing words (documents) but binary files are a bit different. The indexer has to be able to understand the file format in order to extract meaningful information about the file from it. This information about the contents of a file is commonly called metadata.
The most frustrating part about metadata is that users are often too lazy to input all of it. Possibly with better searching tools like Spotlight users will be motivated to input good metadata so that files can be found easier. It's a chicken and egg problem: users right now don't see much of a use for it.
As Steve mentioned in the keynote, searching will reduce the need for defined directory hierarchies for organization of hard drives. Today people spend a lot of time sorting files into their correct directories so that they can be found later. But with Spotlight people will save time by not having to organize their hard drives and doing explicit searches for files.
In the future it's not unreasonable to envision a dynamic file system hierarchy -- one that is created on the fly with metadata instead of rigidly in a file system. The user is then free to browse this dynamic file system presented to them based on criteria they choose. Like "show me all of my pictures, sorted by date and location".
What does this mean for AudioMan? In the operating system of the future metadata will be more important than it is now. We'll need tools to make metadata "tagging" of files more efficient and easy, and AudioMan intends to do just that. Don't be surprised if iTunes starts taking metadata tagging efficiency more seriously after Tiger is released.