In a previous post I detailed building a NAS from a Supermicro 1U server, using a X8DTL-3F motherboard and a Xeon E5645 CPU.
I’ve made some big changes since then, so I decided that this warranted a new post rather than just an update to that post.
Obviously, one change is that FreeNAS is now called TrueNAS, but that isn’t the reason for my post.
As mentioned previously, one of the big limitations of the X8DTL-3F motherboard is the ancient LSI SAS1068E controller which has a hard limit of 2TB drives. I have always meant to add a more modern HBA but have always thought it would be nice if it were onboard. I know that the X8DTL-6F motherboard exists, which is effectively a X8DTL-3F but with a LSI SAS2008 onboard instead of the SAS1068E, and I know TrueNAS plays very well with the SAS2008.
X8DTL-6F system
Recently an X8DTL-6F system came up for sale on eBay at a price not too much more than the cost of a standalone HBA. I figured I could repeat my trick of selling off the bits I didn’t need, and maybe make a small profit.
(Update: Sadly those days seem to be over now, and nothing is selling).
Anyway, I performed the motherboard swap, which went without a hitch, and booted up. I knew that it would be ideal to run the controller in IT mode but wanted to see it all running first which, as you will soon learn, was a bad move.
All seemed well, but after a while the pool attached to the SAS controller started to go degraded. I thought one of the drives was failing and instigated a replace using a spare drive, but then all the drives started degrading.
I wondered if perhaps it was the controller being in IR mode that was causing the issue, because I know ZFS really doesn’t like it. So I sought out the IT firmware.
Flashing to IT mode was incredibly straightforward, as you can read all about in this separate post.
Unfortunately, there were more woes to come, as when I booted back into TrueNAS I found that every drive on the pool connected to my SAS 2008 controller was still showing as degraded, and the pool was showing as unusable. But, fortunately for me, it is a pool I just use for backups so I just dropped the pool and re-created it, and it is working very happily now.
Fortunately my important data was on a pool connected to the Intel ICH10R controller.
So, with hindsight, I should have flashed the SAS2008 to IT mode before I ever connected the HDDs to that controller. But hindsight is a wonderful thing, and is also why I write these blog posts. I’m sure nobody reads them, but if I help even one person then it makes it worthwhile.
I noticed that I was only getting 32GB of RAM rather than 48GB, and after testing each bank in turn I found that Slot 1 of CPU1 was non-functional. However, I solved this problem by moving the CPU over to the CPU2 slot, and the three Quad Rank DIMMs to the slots for CPU2, and now I have my 48GB back. I also decided to change the CPU air cooler to one of my Cooler Master Hyper 212 EVO coolers which are now spare as I am now using water cooling on my workstation as I felt the Arctic is a little bulky.
Extra cooling
Talking of the drive bays, I felt that things were a little toasty in the case and that it would benefit from some extra cooling. The only thing I could think of that did not involve a case mod was to add an extra exhaust fan, which I did by mounting a 92mm PWM fan to the unused expansion card slots using rubber shock mounts.
Another issue I had was I was getting corruption on the boot pool, which is a 240GB SSD drive. This had previously been a 160GB HDD which I had cloned onto the SSD drive with Clonezilla and I don’t think TrueNAS has ever been happy with this as it complains that the block size is incorrect. So I exported the config, did a fresh install of the same version of TrueNAS onto the SSD, and then loaded the config. Amazingly this worked without a hitch.
So, there we are. It’s all back up and running again.
Finished system
When funds allow, I will probably retire the 5 x 2TB array as it generates a lot of heat and uses electricity. Plus floor to ceiling drive bays do restrict inlet airflow. Since this array is used for backups, I think a 2 x 8TB mirrored array would make more sense. However, this is more money than I feel I can justify right now, and I may as well just continue running with what I have. Which, of course, means I could have just kept on with the X8DTL-3F
Isn’t hindsight a wonderful thing? 😉
