15Bay NAS (15x2TB HDD) – SAS Expander

I was looking for a fast and easy way to expand my NAS. I wasn’t interested in building an entirely new server. I only wanted to get more disks connected and without incurring the additional cost of running another server. My current setup consists of 8x2TB HDDs in an external enclosure connected to the server via e-SATA (here) this was working fine but it didn’t allow for expanding to a much larger number of disks. SAS Expanders turn out to be a good choice in terms of technology for achieving the goal of connecting many hard disks to a server, being able to scale via daisy chaining expanders, and operating without a full motherboard/cpu/memory. The enclosure I was currently using costs about $240 for 8 bays. I will show you how to get 15bays for not much more and it allows for you to scale. What do you get for less than $1000? 15x2TB of storage with possibilities to scale into the future.

Notes: 

  • These particular parts I selected will operate with 2TB hard disks but will not work with any larger disk sizes (3TB, 4TB, etc). This is because most of these parts are older generation and use SAS1 technology. If you want to support larger disks then you would be looking for SAS2 parts and backplanes. But the costs of the SAS2 backplane and supporting chassis as well as the difference in cost between 2TB and 4TB hard disks makes it more cost efficient to use older 2TB hard disks.
  • I didn’t want an entire server, so this build doesn’t have a motherboard or any other components. I ended up selling the parts that came with the SuperMicro 933T. This cuts down on power usage and improves airflow through the case. It also allows me to add more disks to the inside of the case (the SAS expander supports up to 24 hard disks) if needed.
  • Why not just a tower case with lots of internal bays? I thought about using something like that because it would probably be less costly than purchasing a 15bay server. The trade off is with the 933T chassis you get redundant power supplies (dual psu with one backup) and easy access to external swappable drive bays, making the 933T a more attractive solution for me.
  • Without a motherboard you need to bridge the ATX motherboard pins to get the power supply to turn on, check out this tutorial (green + black). I will write another post about how I re-enable the power button on the front panel and control the chassis fans using an Arduino + temperature sensor!

The Parts 

  • $4 Molex Power Cable Extension 12″ – here
  • $5 PCI-E Molex Power – here
  • $10 SAS RAID Controller – here
  • $13 Small Flat Hard Disk Screws – here
  • $14 Mini SAS to 4-SATA SFF-8087 Forward Breakout Cable – here
  • $25 External Mini SAS SFF-8088 Male to Internal Mini SAS SFF-8087 Male – here
  • $39 HP Proliant 24 Bay 3Gbps SAS PCI E Expander Card – here
  • $245 15 Bay SuperMicro 933T – here
  • $510 15 2TB Hard Disks- here

Some Assembly Required

  1. Remove the motherboard with CPU and Memory.
  2. Attach the HP SAS Expander to the PCI-E Molex Power adapter.
  3. Screw the SAS Expander to the back of the case.
  4. Attach the Chassis Backplane to the SAS Expander (4×4 Forward Breakouts).
  5. Connect an empty Molex to the PCI-E Power Adapter using the Molex extension cable.
  6. Screw in the 15x2TB hard disks to the disk caddies that come with the SuperMicro 933T.
  7. Pop all of the hard disks into the chassis.
  8. Attach the SAS RAID Controller to the server that will be attached to this array of disks.
  9. Connect the SAS RAID Controller in the other server to the SAS Expander in the NAS.
  10. That’s it! Power up the NAS by connecting the green and black pins in the ATX connector.

Interested in taking this build to 24bays instead of 15bays? I chose 15bays because the costs of additional hard disks for 24bays would make this project more than $1000. If you want to do 24bays then you just need more hard disks and a chassis that will take 24bays like here. The SAS Expander already supports 24bays and the other parts would be the same!

Some Photos

All of the Hard Disks!

Hooking up the SAS Expander and Chassis Backplane.

Notice the Molex extension cable to connect to the PCI-E Power adapter.

You will also probably notice the chassis fans are currently not hooked up. You can hook these up to the ATX motherboard pins or another spare molex but in my next post I will detail the use of an arduino and temperature sensor to drive the chassis fans.

Dell Precision T5600 – Gaming/VR Desktop

 

t5600-from-the-right

I wanted to build a desktop for gaming, VR, handle lots of VMs, and experiment with parallel programming (CUDA/OpenCL). I decided my build will have plenty of CPU cores, lots of memory, and top of the line graphics cards! All of this for less than $2500.

Gaming/VR – This function without a doubt needs a high end graphics card. I decided to go with the GTX 1080 SC because it also has lots of (2560) CUDA cores. I got two of them for a total of 5120 CUDA cores. The CPU for this build has a relatively low clock speed compared to the top of the line gaming desktops but the trade off for me was many more cores. Optionally as the LGA2011 socket server processors get cheaper I can pick up a 3.0+ Ghz processor later on. I have the HTC Vive running on this machine.

Lots of VMs – 16 cores/32 threads and lots of memory. With this many cores and the option to go up to 128GB of RAM, this should keep me satisfied in this category for a long while.

Parallel Programming – Two GTX 1080 SC with a combined 5120 CUDA cores along with dual socket E5 v2670 for 16 cores/32 threads. This should be plenty to play with.

Parts List

  • Dell Precision T5600
  • 2x Dell T5600 CPU Fans (should come with desktop)
  • 2x Intel Xeon E5 v2670
  • 4x 8 GB DDR3 ECC RAM
  • Dell 825W PSU (CVMY8 or DR5JD)
  • 2x GTX 1080 SC
  • Hard Drive SSD

Get the Parts

  • Desktop – Getting the T5600. I recommend that you get the cheapest one you can find. We will be replacing the parts in it. It’s important to get a system with dual CPUs because we will need that second CPU fan. The parts we are keeping will be the case, motherboard, cpu fans, and maybe the memory. All of the other parts such as RAID controllers, CPU, hard drives, and Power Supply can be sold to recoup some of the cost. Here are some listings, you can also submit a lower offer for some of them:
    • $475 –   8 cores 2x4core 2.4GHz, 24 GB mem: here
    • $449 – 12 cores 2x6core 2.0Ghz, 16 GB mem: here
  • CPU – If you are happy with the CPUs that came with the T5600 you bought or you’d like to wait to upgrade the CPU then skip this part. Want better CPUs? I have the E5 v2670, they are decently priced and will give you 16 cores @2.6Ghz (32 threads with hyperthreading). Here are some from EBay, I recommend submitted your best offer (lower than their asking price).
  • Memory – If you are happy with the memory that came with the T5600 you bought or you’d like to wait to upgrade the memory then skip this part. Want more memory? This Desktop will take up to 128GB of DDR3 RAM! 8x16GB. Any 8GB/16GB DDR3 ECC memory will do. It must be 1333Mhz or 1600Mhz and ECC. Here are some I picked out from EBay:
    • $87 – Hynix (4x8GB) 1600Mhz: here
    • $83 – Samsung (4x8GB) 1600Mhz: here
    • $76 – MMB (4x8GB) 1600Mhz: here
  • Power Supply – Theoretically a 625W PSU should be enough to power just one GTX 1080 SC so if you’re happy with one then skip this part. We want to power two of them so we are going to need a 825W PSU. I would contact Dell here via their chat support to figure out which one is compatible. From my personal experience with talking with them the models CVMY8 and DR5JD would fit the case. They have specific guide rails which prevent other 825W by Dell from being used. I was able to find one on EBay for $90, I apparently overpaid.
  • Graphics Card – You can’t do without this component. Every gamer knows the most important part of a gaming desktop is the graphics cards! I was able to pick one up for around $550 new, I had to submit quite a few offers on EBay to find a seller willing to give me one at this price. You can also try your luck bidding for one as well. Be patient here and you will get it for what you want to pay.
    • Submit best offer: here
    • Submit best offer: here

What do you get?

  • 32 Threads (8 cores per CPU, 16 Cores, 32 Threads with Hyperthreading)
  • 32 GB DDR3 ECC RAM
  • 5120 CUDA Cores from the two GTX 1080 SC
  • VR Ready and more

Down sides to this build?

  • Decent clock speeds on CPU but lots of cores. For gaming it’s typically better to have higher clock speeds. High end gaming machines will be up in the 3.8Ghz range. You can buy a faster Intel Xeon processor if this is a concern for you, I haven’t ran into any issues yet.
  • Tight fit, the components just fit into the case.

Here is a picture of the cards in the PC

 

 

Powering the Dual GTX 1080 SC

The 825W PSU has enough power for the whole system. The tricky part is how to connect the GTX 1080 SC to the power supply. Connecting both GPUs to the one PCI-E slot on the PSU board draws too much current and makes the system unstable. There are four connectors from the Dell PSU Board (see the photo below).

 

To get around this issue I connect the first GPU card to the PCI-E power pins using an 8-pin to 8-pin PCI-E cable. And the other GPU card to a SATA to 8-pin adapter. The power becomes split between the PCI-E and the Motherboard/SATA power connectors and this runs stable for me.

  • 8-pin male to 8-pin male PCI-E power cable: here or here or here
  • 15-pin SATA to 8-pin male PCI-E power cable: here or here or here

That should get your dual GTX 1080 SC happily powered.

Next Update [I will append to this post] (Getting SLI to work on the Dell Precision T5600)

See the SLI HB Bridge below! Get it, here or here or here