~ryanlowe/blog: Transfer Explosion

Transfer Explosion

I was surprised to learn that in the month of March 2005 my ryanlowe.ca website alone chewed through 2 gigabytes of bandwidth (bytes sent to clients). It only has my blog and my resume on it! Since I only pay for up to 1 gigabyte this comes right out of my own pocket. That kind of math motivates me to find ways to trim the fat, so to speak. After looking at the transfer logs it seems there wasn't one single problem but a few...

The biggest problem seems to be bots, not people. There are search engine bots, yes, but there are also blog comment spamming bots that search through blogs. These bots use up my transfer quota mindlessly downloading and inspecting every page on my site. I really really really don't want to have to disable comments and trackbacks, even on my really old posts. But I just might have to do that in order to get rid of these comment spammers.

Another problem is images, including the "headshot" image at the top right corner of the main page of this blog. I'm debating whether or not I should just get rid of it but the headshot comes from a longstanding journalism tradition indicating that this is my opinion, not necessarily fact, and I like that. Screenshots on the main page were another offender. Overall, I should keep the size of images in mind when I post them.

A third problem are the RSS feeds, which are downloaded often by RSS clients (ie. hourly even if nothing has changed). I put whole posts in my feed so naturally they can get quite long (the fact that I'm a windbag aside). I know many of my readers like full posts in the RSS feed so I will probably just reduce the number of posts in the feed instead.

I wonder if the web server I'm hosted on supports compression? I know that some browsers support it so they can receive compressed data if the server supports it. Plain text (like HTML, RSS feeds) compresses very well.

Any other suggestions for reducing my transfer numbers? I'd like to get back under 1 gigabyte again.

Posted at April 11, 2005 at 01:26 PM EST
Last updated April 11, 2005 at 01:26 PM EST

Comments

Sounds stingy, but, try removing all white space in the HTML page before adding. I used to do that for my music page listing. Helped a lot ... I was working with a 300MB bandwidth limitation.

Do a comparison of page size of before and after.

Also, you're doing something wrong with the images. Your small image (the thumbnail) is actually bigger in size than your big image!!!?!

ww.ryanlowe.ca/blog/images/week07/AudioMan-week07.png
16.39 KB (16786 bytes)

http://www.ryanlowe.ca/blog/images/week07/AudioMan-week07-small.png
20.82 KB (21320 bytes)

I would check if that's the case with other images as well.

» Posted by: roy at April 11, 2005 02:03 PM

In that case Roy it has to do with the way that PNG compresses images. The larger image has straight lines, no blurring and large areas of the same pixel colour. The smaller image is blurred and cannot use compression techniques as effectively, giving it a bigger file size.

» Posted by: Ryan at April 11, 2005 02:11 PM

Not so sure about that. I use Irfanview, and I was able to compress the big image to a thumbnail, and the end size was 14.31 KB (14,655 Bytes) (200px 142px) and not 20.82 KB. PNG has 10 levels of compression. Max it out and you'll get a good result. No thumbnail should ever be larger than the large image.

» Posted by: roy at April 11, 2005 03:39 PM

Ahh good one. I didn't see a compression option in Photoshop for PNG, but I'll check it out. There's no reason for a blurry thumbnail to have low compression -- the quality is already gone.

» Posted by: Ryan at April 11, 2005 03:42 PM

...or you can cheat and save it as an 8bit GIF ~10KB (PNG is 24bit by default), then resave it as PNG with Max compression and your thumbnail is now an 8bit PNG with a size of 8.00 KB (8,189 Bytes)!!!

» Posted by: roy at April 11, 2005 03:48 PM

Haha, oh man ... I should know better than to ask Roy a simple question... ;)

» Posted by: Ryan at April 11, 2005 03:51 PM

Images can cut quite a bit into your bandwidth. Using proper compession can ease the amount of traffic generated.

I find it interesting the amount of traffic you get from bots. My site doesn't get much traffic, but traffic from bots tottalled last month at 25 MB. Which is pretty high I think, but nothing I can't afford. Maybe there is a way to block bots which are related to blog comment spamming. I don't seem to get any of these.

Lastly, you may want to look into changing hosting providers. I get 80 GB per month of transfer for $US 10 a month. With 1 GB of storage. Way more than I could ever possibly use. The service isn't the best, so i've considered switching. There are a lot of hosting providers out there, finding a good one is a challenge.

» Posted by: Kibbee at April 13, 2005 08:46 AM

Check out Google's calculated ad for this post:
"Homocon - Gay Republican; Not all Gay Men are Liberal; Check out this new Blog"

You have to wonder what triggered that.

» Posted by: Ben at April 13, 2005 05:55 PM

Ben: the word "Blog" probably.

» Posted by: Ryan at April 13, 2005 06:05 PM

If compressing the images doesn't save you enough bandwidth, try coralizing (http://www.coralcdn.org/) them.

Just change the link to: http://www.ryanlowe.ca.nyud.net:8090/blog/archives/images/headshot.jpg

» Posted by: Tim at April 16, 2005 06:57 AM

Whoa that's cool Tim, Thanks!

» Posted by: Ryan at April 16, 2005 07:01 AM