There's no place like home

I managed to finish up the new round of testing this evening and have posted the final results!

They are a little different than I expected them to be, but at least I have good data to support the configuration of my new filer.

Go check the results out!

Well, last night I discovered that my testing methodology was flawed and the results are all over the board.

Plus I changed the test configuration in the middle of the testing.

So, I'm going to backup and take another run at it.

I had run just one test per drive configuration and the results are weird. Then last night I ran multiple tests on one configuration and those results didn't match the first test for that configuration.

And I upgraded the firmware on the IDE controller cards in the middle of the testing. D'oh.

So my new plan is this:

• Upgrade the BIOS on the motherboard (if it needs it.)
  
• Reset the PnP BIOS configuration after each controller card swap - I think this might have caused issues.
  
• Run the test 10 times for each configuration and average the results. This should even out some of the whacky results I saw.
  
• Not change the test configuration in the middle of the tests!
  

So it will take a bit to do all this. Maybe I can get it done this weekend. We'll see.

The motherboard is an X5DPE-G2, so it has four PCI busses on it.

Nate brought me a Promise Ultra100 TX2 dual channel IDE controller (thanks Nate!) today, and I think I'll try and get three to put in the box, so I'm bidding on two more on eBay right now.

Then I'll have eight IDE channels in the box and I should get some pretty good performance out if it. I'll have to think about where to put which drives for maximum performance, but I think it will be nice and fast.

I'm thinking about putting the five RAID 5 drives on the Promise cards - one drive per channel, and the mirrored pair for the OS on the on-board controllers. I can hang the CD-ROM drive off one of the on-boards as I don't need to worry too much about that drive's performance.

Or does it make more sense to have the five RAID 5 drives on the three Promise cards and the two on-boards? Then they are on three different PCI busses and the on-board controllers. Would that be faster?

And where do I hang the OS drives? What can I boot from?

I guess I can build it and benchmark it both ways. No reason not to mess about and try different configurations.

I can't wait for all the parts to get here! This will be fun!

Finally!

I have five 250GB IDE hard drives that I bought from Nate (thanks Nate!) a while ago. I also have some old gear that I picked up with a Xeon motherboard and CPU.

But the motherboard is in a 1RU case. It's hard to shoe horn five hard drives into a 1RU case. (Yes, with the right case you can do it, but with an EATX motherboard that is 12" x 13", it's not likely.)

I finally got enough spare cash together and bought a new 4RU case. It looks like a nice case, it's got six 5-1/4" bays and two 3-1/2" bays. So I can get my five 250GB drives in there as a RAID 5 array, a pair of 40GB drives mirrored for the OS and a CD ROM drive for installation. (I hope the 550W power supply can drive it all.)

Then I'm going to install the latest release of OpenFiler on it.

I'm also going to borrow a pair of dual-channel IDE crontroller cards from Nate (thanks again Nate!) and build up the RAID arrays using the software RAID in the OpenFiler distro (rPath Linux.) I'll have six IDE channels and if the motherboard has multiple PCI busses I should be okay for performance. I have to find the specs for the motherboard still. It's a SuperMicro something or other. (Judging from the pictures and the specs, I think it's a X5DPE-G - but I'll look at the board tonight and find out for sure.)

Then I'll run some performance tests on it. I want to know if it's worth the trouble of backing up my existing filer (with about 200GB on it) to something (don't know what yet) so that I can use the MegaRAID i4 card that I have in my existing OpenFiler.

My gut feeling is that the software RAID will be plenty fast, but I don't know for sure.

Then after I get my new OpenFiler built I can redirect all the Windows users My Documents folders to it and start doing real backups to my DLT drive! How awesome will that be?

I figured out how to do the redirection with a group policy so no matter which machine you log into you get your My Documents folder. It's a pretty cool Windows hack, plus the end user doesn't even have to know it's happening, so they don't have to be told to 'save your data to the H drive' or anything. It just happens!

And how geeky will it be to be able to state that I have One Terabyte in my basement? (Remember when a terabyte was an unimaginable amount of space?)

Here is a great tip about a program to keep your iTunes synced with your music folders. I've already had some trouble with this on my OpenFiler share, so I'll be trying this out at home!

Nate at work turned me on to MT-Daap - a Daap server for Linux.

Daap is the protocol that iTunes uses for sharing your libraries.

MT-Daap runs as a daemon on your linux box and shares out your files (mp3, mp4, aac, etc...) as an iTunes library share.

One of the cool features that MT-Daap has is that you can setup smart playlists in a config file and they will work just like the iTunes ones.

Slick!

I am installing the nightly build now - has a bunch of changes and new features, so we'll see how that works out.

Wow. That's way cooler than the stable build! The web interface is much better - you can actually build smart playlists with it! And there is a Java applet that plays your mp3s in your browser!

It's so cool I'm laughing!

I'm starting to rip my CDs onto the Openfiler box and I think I have a problem.

• I have about 130GB of space available for MP3s.
  
• I have about 500 CDs.
  
• Each CD is between 300 and 500 MB when ripped.
  
• That means that I need about 250GB of storage space.
  

Hmmm.

Well, I guess I'll just rip what I can for now. I think I'll start at A and go until I run out of space.

My dream would be to buy five 250GB drives. They are pretty cheap after rebate, but of course you are only supposed to get a rebate on one....

So I would take one of the drives and copy all the data that's currently on the RAID to it, then build a new RAID with four of the new drives, giving me a quarter of a terabyte of space. Whee.

Then I'd copy the data back onto the RAID and add the fifth drive as a hot spare.

But it's all a dream until I can buy five 250GB drives, which isn't going to happen soon.

I soooo have to win that lottery.

Updated: Oops. My face is red. I should know better. As Nate points out in his comment, the CDs should not be that large when ripped.

And they aren't.

I was looking at the CD size - unripped.

I am ripping at 192K so the CDs rip down to 40 to 90MB, not 400MB.

So if I have 500 CDs, they should require 42GB? Is that right? Well, I'm going to rip until I run out space, or out of CDs.

But it still would be cool to have three-quarters of a terabyte in my basement.

So I have this OpenFiler NAS in the basement now and I want to put my MP3s on it and serve them to multiple computers around the house. (See this, this and this.)

I had been ripping my CDs into iTunes onto the drive in my laptop and then copying the directory structure to the OpenFiler. This is sub-optimal for several reasons:

1. It's difficult to keep them synced up.
   
2. I have about 18GB of space for MP3s on my laptop and 135GB of space on the OpenFiler.
   
3. iTunes doesn't know about the files on the OpenFiler, so they don't show up in my laptop's library after I copy them over.
   

So that's no good.

Number 1 could be solved with rsync - and I have just installed Cygwin so I could use their rsync utility.

Number 2 is not a big issue, really.

Number 3 was the show-stopper as far as I was concerned.

Then I used my head, searched with Google and found Libra.

Libra is a very simple application that allows you to create, use and manage multiple iTunes music libraries. Normally, iTunes has only one library, and you can create playlists and groups within this unique library. However, some users have reported that iTunes slows down noticeably when the library becomes large, and other users would like to simply categorize their music in separate sets. This is what Libra allows you to do safely.

Libra is shareware, not free, but it's only $10. It seems to work okay so far, and it's just nagware, so I'll run it a bit before I send in the money, but if it works well, it's worth it.

So the new plan (which I am executing as I type this) is to create a new library on the OpenFiler, make it the active library and import all the MP3s into it.

After that's done, I can change back to the library on the laptop drive and delete a bunch of songs to make more disc space.

I can also rip my CDs directly into the library that is on the OpenFiler from now on and I should be good to go!

Pretty slick.

I should also be able to export or copy the iTunes Library files to other machines, assuming I mount the OpenFiler on the same drive letter. We'll have to see how that goes later.

Well, I have tested all six Western Digital WD800 drives using the WD DLGDiags tool and the one Samsung SP8004H drive with the Samsung HUTIL tool.

Of the six WD drives, two are dead. They don't even pass the quick test. Heck, they can't even start the quick test. When you tell the sofware to quick test them, it just hangs for about 5 minutes, then spits out a page with all kinds of parameters, showing you the errored ones. It would be cool, if it didn't mean the drives were dead.

On the other hand, the four remaining WD drives all passed the extended media test with zero errors. This is a good thing, I guess, but of course provides no indication of how long they will last before they also die.

Guess how many of the drives are still under warranty?

---> ZERO <---
That's right, none of them. Grrrr.

So now I have a quandry.

• Do I build a RAID 5 array using the four good WD drives and the one Samsung drive? Thus creating a RAID array that I'm sure I would have no possibility of fixing if it broke?
  
• Do I try and talk Liz into letting me go out and buy two 200BG drives (lots of rebates would bring it down to about $100 for the pair) and build two mirrors - one 80GB for OS and some storage, one 200GB for storage alone?
  
• Do I give up the idea of having an Openfiler box for a while and wait until I can afford to buy a fistful of larger drives?
  

What to do, what to do?

---

Okay, I have a plan.

Since I don't really have any spare cash right now, and Christmas is coming, high heating bills, etc., I'm going to do the following:

I'm going to rebuild the Openfiler box using the four WD800 drives. I'll end up with about 240GB of space in a RAID 5 configuration.

I'm only going to put my MP3s on it for now. If it goes away, well, so be it. I might take some DLT backups of it.

I also might run out and buy one of those cheap 200GB drives to put into my main server - rebuilding it into a busted mirror until I can afford to buy a second one to mirror it onto. CompUSA has one that is $29.95 after rebates, assuming they have any left. The current drive in my main server is a WD800 of unknown vintage - it would be nice to replace that before it cacks.

So that's my plan.

For now.

Update: Well, of course, by the time I got to CompUSA (admittedly, it was 7:00 PM) they were out of the 200GB drives. The clerk said they ran out about 1:00 PM.

C'est la Vie.

So, just because I didn't have anything else to do this morning I did the following to my Openfiler box:

• Replaced the power supply (an old 'Sparkle' power suppy that I'm not sure the fan even worked on) with an old (but should be better than the 'Sparkle') PC Power & Cooling supply. Still only 235 Watts, but should be a much higher quality supply.
  
• Added a huge fan to the front of the case to pull air over the hard drives. The drives had been running a little warm, not hot, but warm none-the-less, so I figure more cooling can't hurt. I have a pair of 5 inch fans from an old Sun box, so I mounted one on the front of the case. I had to take the door off, but who cares? It's not too loud, but there is definitely more air moving.
  
• Started testing all the hard drives with DLGDiags downloaded from Western Digital. So far the first drive that I swapped in has passed and one of the existing drives has passed. It takes about 45 minutes per drive to do the full test. I'm guessing that all the drives will pass, including the one that was originally failed out. If that's the case, what else should I test them with? Full write tests? Does Spinrite still exist? (Apparently yes.)
  
• Verified that all the fans are working - PS fan, CPU fans, case fans. They all run.
  

So. What else to do?

I'm going to test all the drives with DLGDiags. If even the failed drive passes, then I'll have to try and test them with something better.

Then I'll rebuild the filer from scratch. Wheee.