The ASRock X99 Extreme11 Review: Eighteen SATA Ports with Haswell-Eby Ian Cutress on March 11, 2015 8:00 AM EST
- Posted in
If there is one thing I like about ASRock, it is their ability to do something different in an increasingly difficult market to differentiate. One of these elements is the Extreme11 series, using an LSI RAID controller to provide more SAS/SATA ports on the high end model. Today we have the X99 Extreme11 in for review.
ASRock X99 Extreme11 Overview
Our last review of an Extreme11 model was back in the X79 era, featuring the six SATA ports from the PCH and eight from the bundled LSI 3008 onboard controller. Our sample back then used eight PCIe lanes for the controller and achieved 4 GBps maximum read and write sequential speeds when using an eight drive SSD RAID-0 SF-2281 array. Between the X79 and the X99 model came the Z87 Extreme11/ac which used the same LSI controller but bundled it with a port multiplier, giving sixteen SAS/SATA ports plus the six from the chipset for 22 total. When we come to the X99 Extreme11 in this review, we get the same 3008 controller without the multiplier) which adds eight ports to the ten from the PCH, giving eighteen in total.
One of the criticisms from the range is the lack of useful hardware RAID modes with the LSI 3008. It only gives RAID 0 and 1 (also 1E and 10) with no scope for RAID 5/6. This is partly because the controller comes without any cache (or albeit a very small one) which cannot help with managing such an array. ASRock's line on this is partly due to controller cost and complexity of implementation, suggesting that users who require these modes should use a software RAID solution. Users who want a hardware solution will have to buy a controller card that supports it, and ASRock is keen to point out that the Extreme11 range has plenty of PCIe bandwidth to handle it.
The amount of PCIe bandwidth brings up another interesting element to the Extreme11 range. ASRock feels that their high end motherboard range must support four-way GPU configurations, preferably in x16/16/16/16 lane allocation. In order to do this, along with having enough lanes for the LSI 3008 controller that needs eight, for the X99 Extreme11 there are two PLX8747 PCIe switches on board. We covered the PLX8747 during its prominent use during Z77, but a base summary is that due in part to its FIFO buffer it can multiplex 8 or 16 PCIe lanes into 32. Thus for the X99 Extreme11 and its dual PLX8747 arrangement, each PLX switch takes 16 lanes from the CPU to give two PCIe 3.0 x16 slots, totaling four PCIe 3.0 x16 slots overall. The final eight lanes from the CPU go to the LSI controller, accounting for 40 lanes from the processor. (28 lane CPUs behave a little differently, see the review below.)
As you might imagine, two PLX8747 switches and an LSI controller onboard does not come cheap, and that is why the Extreme11 is one of the most expensive X99 Motherboards on the market at $630+, only to be bested in this competition by the ASRock X99 WS-E/10G which comes with a dual port 10GBase-T controller for $670. Aside from the four PCIe 3.0 x16 slots and 18 SATA ports, the Extreme11 also comes with support for 128GB of RDIMMs, LGA2011-3 Xeon compatibility, dual Intel network ports, upgraded audio and dual PCIe 3.0 x4 M.2 slots. The market ASRock aims for with this board needs high storage and compute requirements in their workstation - typically with these builds the motherboard cost is not that important, but the feature set is. That makes the X99 Extreme11 an entertaining product in an interesting market segment.
With the extra SATA ports and controller chips onboard, the Extreme11 expands into the EATX form factor, which means an extra inch or so horizontally for motherboard dimensions. Aside from the big block of SATA ports, nothing looks untoward on the board, giving an extended heatsink around the power delivery down to the chipset heatsink which has an added fan to deal with the two PLX8747 chips and the LSI 3008 controller.
The socket area is fairly crammed up to Intel’s specifications, with ASRock’s Super Alloy based power delivery packing in twelve phases in an example of over engineering. The DRAM slots are color coded for the black slots to be occupied first. Within the socket area there are four fan headers to use – two CPU headers in the top right (4-pin and 3-pin), a 3-pin header just below the bottom left of the socket (above the PCIe slot) and another 3-pin near the top of the SATA ports. The other two fan headers at the bottom of the board are one 4-pin and another 3-pin, with the final fan header provided for the chipset fan. This can be disabled if required by removing the cable.
The bottom right of the motherboard next to the SATA ports and under the chipset heatsink hides the important and costly controller chips. Combining the two PLX8747 on the left, the LSI RAID controller and the chipset comes north of 30W in total for power use, hence the extra fan on the chipset.
Each PLX8747 PCIe switch can take in eight or sixteen PCIe 2.0 or PCIe 3.0 lanes, then by using a combination of a FIFO buffer and multiplexing output 32 PCIe 3.0 lanes. Sometimes this sounds like magic, but it is best to think of it as a switching FPGA – between the PCIe slots, we have full PCIe 3.0 x16 bandwidth, but if we go up the pipe back to the CPU, we are still limited by that 8/16 lane input. The benefit of the FIFO buffer is a fill twice/pour once scenario, coalescing commands and shooting them up the data path together rather than performing a one in/one out. In our previous testing the PLX8747 gave a sub 1% performance deficit in gaming, but aids compute users that need inter-GPU bandwidth. It also surpasses the SLI fixed limitation of needing eight PCIe lanes, ensuring that the NVIDIA configurations are happy.
The LSI 3008 is a little long in the tooth having been on the X79 and Z87 Extreme11 products, but it does what ASRock wants it to do – provide extra storage ports for those that need it. In order to get a case that can support 18 drives is another matter – we often see companies like Lian Li do them at Computex, and some cost as much as the motherboard. The next cost is all the drives, but I probably would not say no to an 18*6 TB system. The lack of RAID 5/6 for redundancy offerings is still a limitation, as is the lack of a cache. Moving up the LSI stack to a controller that does offer RAID 5/6 would add further cost to the product, and at this point ASRock has little competition in this space.
On the back of the motherboard is this interesting IC from Everspin, which turns out to be 1MB of cache for the LSI controller. There is scope for ASRock to put extra cache on the motherboard, allowing for higher up RAID controllers, but the cost/competition scenario falls into play again.
The final part of the RAID controller is this MXIC chip, which looks to be a 128Mbit flash memory IC with 110ns latency.
Aside from the fancier features, the motherboard has two USB 3.0 headers above the SATA ports (both from the PCH), power/reset buttons, a two digit debug display, two BIOS chips with a selector switch, two USB 2.0 headers, a COM header, and the usual front panel/audio headers. Bang in the middle of the board, between the PCIe slots and the DRAM slots, there is a 4-pin molex to provide extra power to the PCIe slots when multiple hungry GPUs are in play. There is also another power connector below the PCIe slots, but ASRock has told us that only one is needed to be occupied at any time. I have mentioned to ASRock that the molex connector is falling out of favor with PSU manufacturers and very few users actually need one in 2015, as well as the fact that these connectors are both in fairly awkward places. The response was that the molex is the easiest to apply (compared to SATA power or 6-pin PCIe power), and the one in the middle of the board is for users that have smaller cases. I have a feeling that ASRock won’t shift much on this design philosophy unless they develop a custom connector.
The PCIe slots give x16/x16/x16/x16, with the middle slot using eight PCIe 3.0 lanes when in use causing the slot underneath to split causing an x8/x8 arrangement. With sufficiently sized cards, this gives five cards in total possible. Normally we see the potential for a seven card setup, but ASRock has decided to implement two PCIe 3.0 x4 M.2 slots in-between a couple of the PCIe slots. The bandwidth for these slots comes from the CPUs PCIe lanes, and thus do not get hardware RAID capabilities. However, given the PM951 is about to be released, two of them in a software RAID for 2800 MBps+ sequentials along with an 18*6 TB setup would be a super storage platform.
For users wanting to purchase the 28-lane i7-5820K for this motherboard, the PCIe allocation is a little harder to explain. The CPU gives 8 lanes each to the PLX controllers, giving a full x16/x16/x16/x16 solution still applies, with another 8 lanes for the LSI controller. The first M.2 x4 port gets the last four lanes and the second M.2 slot is disabled.
The rear panel gives four USB 2.0 ports, a combination PS/2 port, a Clear CMOS button, two eSATA ports, two USB 3.0 from the PCH, two USB 3.0 from an ASMedia controller, an Intel I211-AT network port, an Intel I218-V network port and audio jacks from the Realtek ALC1150 audio codec.
|ASRock X99 Extreme11|
|Memory Slots||Eight DDR4 DIMM slots, up to Quad Channel 1600-3200 MHz
Supporting up to 64 GB UDIMM
Supporting up to 128 GB RDIMM
|Network Connectivity||Intel I211-AT
|Onboard Audio||Realtek ALC1150 (via Purity Sound 2)|
|Expansion Slots||4 x PCIe 3.0 x16
1 x PCIe 3.0 x8
|Onboard Storage||6 x SATA 6 Gbps, RAID 0/1/5/10
4 x S_SATA 6 Gbps, no RAID
8 x SAS 12 Gbps/SATA 6 Gbps via LSI 3008
2 x PCIe 3.0 x4 M.2 up to 22110
|USB 3.0||6 x USB 3.0 via PCH (2 headers, 2 rear ports)
2 x USB 3.0 via ASMedia ASM1042 (2 rear ports)
|Onboard||18 x SATA 6 Gbps Ports
2 x USB 3.0 Headers
2 x USB 2.0 Headers
7 x Fan Headers
HDD Saver Header
Front Panel Audio Header
Front Panel Header
Two-Digit Debug LED
BIOS Selection Switch
|Power Connectors||1 x 24-pin ATX
1 x 8-pin CPU
2 x Molex for PCIe
|Fan Headers||2 x CPU (4-pin, 3-pin)
3 x CHA (4-pin, 2 x 3-pin)
1 x PWR (3-pin)
1 x SB (3-pin)
|IO Panel||1 x PS/2 Combination Port
2 x eSATA Ports
4 x USB 2.0
2 x USB 3.0 via PCH
2 x USB 3.0 via ASMedia
1 x Intel I211-AT Network Port
1 x Intel I218-V Network Port
Clear CMOS Button
|Warranty Period||3 Years|
Post Your CommentPlease log in or sign up to comment.
View All Comments
wyewye - Friday, March 13, 2015 - link"this system is just a confused jumble of parts slapped together"
This is the best conclusion for this mobo.
I think they hope marketing/sales guys will be able to bamboozle dummies to sell this as a 18 port raid server mobo. Anyone who spends 600$ on a high-end mobo without reading a review, deserves whats coming to them.
swaaye - Wednesday, March 11, 2015 - linkThat chipset fan is cheesy. They most definitely could have come up with a better cooling solution.
Hairs_ - Wednesday, March 11, 2015 - linkAs pointed out above, this board doesn't answer a single question any user is asking, and it doesn't fulfill any logical useage case.
I'm struggling to see why it was reviewed other than the possible reason "reviewer only wants to review weird expensive stuff". Getting in a board whose supposed only reason to exist is the number of storage ports, then not yet the storage, and say " in sure it's the same as the one I reviewed a few years ago " is... Troubling. What was the point of getting it in for review at all??
ClockHound - Wednesday, March 11, 2015 - linkWhat's the point?
It's the new Anandtech, where the point is clicks! Catchy headlines with dubious content, it's how Purch is improving a once great review site. Thanks, Purch!
ap90033 - Friday, March 13, 2015 - linkI think you may be on to something! Sad to see..
Stylex - Wednesday, March 11, 2015 - linkI don't understand how motherboards still have usb2 ports. Did it seriously take this long for it to transition from usb1.1 to usb2?
DanNeely - Wednesday, March 11, 2015 - linkAt the USB1.1-2.0 transition time, Intel chipsets had at most 6 ports; and since the new standard didn't need any more IO pins so they could cut over all at once. Not needing any more IO pins is important because it's been the the limiting factor for chipset cost for a number of years; with the die size being determined by the number of output pins added.
The bottleneck for USB3 has been the chipsets. Pre-IVB they had no USB3. IVB added support for 4 ports, haswell bumped it to 6, the 9x series chipsets that were supported to launch with broadwell were essentially unchanged from the previous model. As a result, mobo makers who wanted to add more USB3 have had to spend extra money on 3rd party chips to do so. Initially it was on USB3 controllers which generally ate a PCIe lane for every pair of ports added. More recent designs are using 4 port USB hub chips; which give better bang for the buck but still drive prices up.
When skylake launches later this year, the situation should improve; its higher end versions will offer up to 10 ports. That might be enough ports to make all USB3 configurations possible in the mid range without either using a very small total number of ports or driving the board price up with extra controller chips. High end boards will probably still have some ports attached to a controller though, because Intel's expanding it's use of flexible IO ports and native USB3 will be competing with PCIe storage for IO pins. In both cases though, I suspect a number of boards will also expose the 4 remaining 2.0 ports; probably 1 internal header and 2 external ports. That won't be just a case of 'gotta use them all'; older OSes (eg win7) without native support for USB3 are easier to install if you've got a few 2.0 ports available; and there will be residual demand for 2.0 headers from people with older cases or internal card readers.
The situation is similar with AMD chipsets. But due to their being only able to compete in the value segment of the market, they're behind Intel; topping out at 4 native USB3 ports.
Stylex - Friday, March 13, 2015 - linkAh, the pin count makes a lot of sense, thanks for that insight!
But still, how much more could it possibly drive up the price to use a third party controller, $5-10? I'd pay that for all usb3, especially on a board like this one.
DanNeely - Sunday, March 15, 2015 - linkEstimates I've seen over the last few years put a 2 port USB3-PCIe controller as adding $10 to the retail price of a board; a 4 port USB3 hub chip added $5. The caveats are that hubs only add ports not total bandwidth; which is fine if you're only interested in being able to plug a USB3 device into any port but use sufficiently few of them that sticking multiple high speed devices on the same hub isn't a problem. Controllers don't have that problem; but do need PCIe lanes. Those tend to be in short supply on intels 8x/9x chipsets. Using a 4-8 lane PLX on the chipset relieves that pressure somewhat but is another $10 or $20 to the board price. The situation there will be better for skylake due to the 100 series chipets having 28 high speed IO lanes instead of only 18; but that's partially counter balanced by m2/SataExpress connections needing several lanes each.
The lack of native 3.1 support means that the next generation of mobos will probably go the controller route; not hubs to bump up the port count. With Intel rarely doing any major updates on the Tock versions of the chipset, it will probably be at least 2017 before external USB3 controllers mostly go way.
darkfalz - Thursday, March 12, 2015 - linkYou have a keyboard, mouse or gamepad that requires 100 MB/sec bandwidth, do you?