GIGABYTE Server GA-7PESH3 Motherboard Reviewby Ian Cutress on September 4, 2014 2:00 PM EST
- Posted in
A dual processor system sounds awesome to the home user but in reality it is almost entirely a professional market. The prosumer has to use Xeons at JEDEC memory speeds and then ensure that the software is NUMA aware, especially if it decides searching for data in the other processor's L3 cache. However now GIGABYTE Server is selling to the prosumer via Newegg, and they sent us the $640 GA-7PESH3 for review.
For most users, a dual processor system affords several issues, aside from the cost. Performance with a 2P system is very dependent on the software in use. With a single processor system, each core can ‘snoop’ into the other core cache in order to see how data is updated. In a 2P system, the latency of talking between the two CPUs is at least an order of magnitude higher. This means that memory accesses can be delayed causing branched and locked code to be slow. In order to get around this, the software has to be NUMA aware, and the majority of regular applications are not.
There is no overclocking with a 2P system, and single thread speeds can be lower. As a result, gaming often sees a hit in performance, as well as basic tasks. The optimal use case scenario, for most software that is not aware of dual processor architecture, is any workload that needs as few memory accesses as possible and is ‘embarrassingly parallel’ such as ray tracing, video editing, virtualization or particular types of scientific compute. 2P motherboards, particularly those built by server teams, also often come with system management tools not seen in the consumer space, allowing users to access their system as it processes data and monitor progress as well.
GIGABYTE GA-7PESH3 Overview
This motherboard is built with storage and GPUs/coprocessors in mind. Although each CPU socket has four DRAM slots (rather than a more usual complement of eight each), the PCIe is a full array of PCIe 3.0 slots offering x16/-/x16/-/x16/-/x16 or x16/x8/x8/x8/x8/x8/x8 whether the system is using four PCIe devices or seven. This means that the PCIe lane allocation comes from both CPUs, which in itself can produce issues for CrossfireX and SLI setups, although compute does not rely on those technologies and should suffer no issues. Interestingly GIGABYTE does not supply any extra power connectors for these PCIe slots, indicating that not each port might not be able to provide 75W when all are populated (e.g. seven graphics cards).
In terms of storage, the system has eight SAS 6 Gbps ports in white on the right hand side, rather than the mini-SAS connectors we saw on other C60x motherboards. For SATA storage there are two SATA 6 Gbps ports in red and four SATA 3 Gbps in black, each at the bottom of the board. The motherboard also has two USB 3.0 type A ports sticking out of the motherboard – this feature is useful for users with software that requires USB dongle verification.
We tested the motherboard in both dual E5-2697 v2 (2x 12C/24T, 2x 130W) and dual E5-2687W v2 (2x 8C/16T, 2x 150W), showing where more cores of the former and the higher base frequency of the latter pay off in our CPU benchmarking suite. This especially shows through in our rendering benchmarks, although not so much in the x264 video conversion. In these configurations, idle power consumption with a single GTX 770 installed gave 115-130W at idle and 300W-320W with 100% CPU load. The Realtek ALC892 codec is more a token gesture for basic audio; other server motherboards tend to eschew audio, but as this is more a workstation outfit then basic audio should be sufficient with dedicated audio professionals using their own PCIe sound cards. On the audio note, the DPC Latency was clocked in as 300+ microseconds, similar to other server products, but ultimately quite high compared to mainstream motherboards.
Server motherboards are designed to be left on for extended periods of time, and thus not a lot of effort is made in POST times – this is indicated by the SAS and two Intel NICs having their own OPROM POST sequences, making the total power on to Windows loading screen over 60 seconds. This reduces to 38 seconds when the SAS and NICs are disabled.
Our gaming benchmarks were limited by the motherboard not displaying an image when more than one GPU was used, but the single GPU benchmarks all showed a regression in performance against an i7-4770K setup. One could suggest that this is not a platform for gaming, although users who want to have this system in a dual work/home role should have no problems matching mainstream system performance.
Motherboards in this bracket are all about the feature set, and the GA-7PESH3 oozes capability, especially when pairing two 12 or 15 core CPUs with a four-way co-processor setup matched with a SAS RAID. The only downside is the cost to get there. Server motherboards are often limited in BIOS and Software, and the 7PESH3 does not move things into a more useful role there, indicating a potential gap in the server board market.
For those that are more used to the consumer sized motherboards, the SSI CEB standard is both longer and uses different fixing points to the case, so users who invest should ensure that their case supports the standard as most do not. One of the poignant features the GIGABYTE Server team pointed out to me was the socket implementation which on this board has the sockets side by side and level, compared to others (such as the 7PESH1 we reviewed) which are often offset due to PCB layout.
Because of this orientation, equipping each CPU with eight DRAM slots is a little unwieldy unless concessions are made, so GIGABYTE went with one DIMM per channel for each CPU. While this limits the system to eight slots, this does afford 64GB of UDIMMs or 256GB of quad-rank RDIMMs both at 1866 MHz rather than the 1600 MHz standard. Given how Xeon server and workstations are often memory speed limited, this might be a good positive step.
Each CPU needs its own 8-pin CPU power, and because there is no overclocking on this platform each of the power delivery segments is built to server specifications: several high-rated power phases built for 24/7 operation.
While there are six fan headers on board, the placement does seem to detract from any liquid cooler setups by providing each of the sockets only one header to play with. The other four headers are located in pairs – one pair above the SAS ports and the other pair below the PCIe sockets.
For storage the motherboard uses SAS ports directly on the PCB rather than through a mini-SAS interface. These eight ports are in white, with the SATA ports in red and black at the bottom of the board. The red ports are SATA 6 Gbps while the black ports are SATA 3 Gbps. The heatsinks above these ports are for the chipset and a controller. Next to the two black SATA ports at the bottom are two USB 3.0 type A ports (powered by an Etron controller) directly in the motherboard. These ports allow for permanent USB storage to be attached while in a case, or as is often the case with workstation builds, suitable for USB verification dongles for professional software that is sold on a per-system license. My experience is that these dongles are being phased out due to the prevalence of a central license server or the web, but this is a feature that makes it on to some consumer motherboards.
The GA-7PESH3 has a maximum complement of full-length PCIe slots, although the naming configuration is odd. The bottom one is labeled PCIE_1 and the top is PCIE_7, this is because PCIE_1/3/5/7 are all x16 (two from each CPU) with PCIE_2/4/6 being x8. When something is installed into an x8 slot, the slot underneath reduces down to x8 mode, ending up with two possible extremes:
Four PCIe devices: x16/-/x16/-/x16/-/x16
Seven PCIe devices: x16/x8/x8/x8/x8/x8/x8
For users with single slot RAID cards or single slot compute GPUs, such as AMD’s FirePro V7900 or S7000, each element of bandwidth is more than welcome. However for users interested in SLI or Crossfire, the card that gets the default video output is often the one at the bottom. It should be noted that there is no additional power delivery on the motherboard for the PCIe slots. This is a little worrying - should a user put in several devices that consume ~70W each and not require extra power plugs, the motherboard might have a hard time delivering 7x70W and cause the 24-pin ATX connector to burn out. This comes across as a design flaw, perhaps suggesting that while the system should be geared towards 4-way GPUs or 7-way Compute, only devices that have external power should be used. Even then, this might cause issues for any combination of cards requiring >200-250W. from the 24-pin power connector.
Like other motherboards in this segment, the 7PESH3 uses an Aspeed IC to provide system management tools. This allows the user to log in to the system via a dedicated network port, turn the machine on, and access the graphical output. The Aspeed also provides the VGA output on the back if the system is under a desk but filled with compute performance. The base design of an AST2300 is a 400 MHz ARM9 core, which GIGABYTE has paired with a small amount of onboard memory.
The rear panel on the motherboard contains a PS/2 combination port, four USB 2.0 ports, four USB 3.0 ports (from an Etron controller), a VGA port, a COM port, two Intel 82574L NICs, a management NIC from the Aspeed IC and audio jacks from the Realtek ALC892 audio codec.
|CPU Interface||2x LGA2011-0|
Eight DDR3 DIMM slots
Supporting up to 64 GB UDIMM / 256 GB RDIMM
Up to Quad Channel, 1066-1866 MHz, ECC and non-ECC
|Video Outputs||VGA via Aspeed 2300|
2x Intel 82574L
1 x Aspeed Management NIC
|Onboard Audio||Realtek ALC892|
7 x PCIe 3.0 x16
2 x SATA 6 Gbps
4 x SATA 3 Gbps
8 x SAS 6 Gbps
SATA RAID 0/1/10/5
SAS RAID via LSI SAS2008, RAID 0/1/1E/10
4 x USB 3.0 on Rear Panel
2 x USB 3.0 Type-A on Motherboard
1 x USB 3.0 Header
2 x SATA 6 Gbps
4 x SATA 3 Gbps
8 x SAS 6 Gbps
6 x Fan Headers
1 x SATA SGPIO Header
1 x LSI RAID Key Header
2 x USB 3.0 Type-A
1 x USB 3.0 Header
1 x USB 2.0 Header
1 x HDD Back Plane Board Header
1 x BIOS Recovert Jumper
Front Audio Header
Front Panel Server Header
1 x 24-pin ATX
2 x 8-pin CPU
2 x CPU (4-pin)
4 x SYS (4-pin)
4 x USB 2.0
4 x USB 3.0
2 x Intel NIC
1 x PS/2 Combination Port
1 x VGA
1 x Serial
1 x Management NIC
|Warranty Period||3 Years|
In The Box
Workstation and server motherboards are in that weird nexus where system admins might not want a lot of extras. Their systems are planned out and all the separate items are ordered separately. Now that the GIGABYTE Server division is selling to end-users, the packages have to represent what a home/prosumer might need in with the box. As a result, we get the following:
Driver DVD (Manual on Disk)
Quick Reference Guide
Rear IO Shield
3.5-inch USB 3.0 Front Panel with 2.5” bracket
Six SAS/SATA Cables
USB 2.0 Rear Bracket
2/3/4 way SLI Connectors
Three Crossfire Connectors
This is a nice amount of kit to get with a motherboard, especially with a server/workstation based product. One could levy an argument that for non-multi GPU implementations, the price could be reduced by $25 by removing all the additional SLI/Crossfire Connectors, but items like the USB 3.0 front panel are well received.
Post Your CommentPlease log in or sign up to comment.
View All Comments
ddriver - Thursday, September 4, 2014 - linkWhy include gaming benches, considering this product is not intended nor optimal for the task, while at the same time prosumer and enterprise tests are completely absent?
ATC9001 - Thursday, September 4, 2014 - linkThe gaming benches are just for good FYI, considering most of the audience are gamers/ enthusiasts. However, I do agree I would like to see more prosumer workload benchmarks!
ddriver - Thursday, September 4, 2014 - linkWhere is 3D MAX / MAYA 3d rendering? Where is AfterEffects / Premiere / Nuke / Fusion video rendering? Where is Cubase / ProTools / Sonar audio rendering? Where are multiphysics simulations? Where are all the practical workloads that would justify the purchase of a dual socket workstation and that can actually scale up so many cores?
Instead we got games, everyday office and multimedia applications and 2 completely impractical synthetic tests. And it is not just that review, this is becoming a common practice at reviewing workstation class mobos. Almost as ridiculous as using different JS engine implementations to measure raw CPU power in mobile devices...
cen - Thursday, September 4, 2014 - linkI completely agree. Anandtech used to have a server division but apparently this is gone now. I have noticed several times that server grade products get non-server related treatment, rendering the whole review completely pointless.
xyzzy1954 - Friday, September 5, 2014 - linkI'll second that. I just bought a 'gamer' setup, but for video and audio rendering. I also do a lot of OLAP analytics. I'd really enjoy a review / comparison of 97 to 99 systems for these purposes. Threads matter, RAM > 32 GB matters. I don't get all this hardware from manufacturers for review purposes...so I can't afford to do the comparisons. I feel confident that AnandTech would publish the comparisons if they had all the relevant tests 'off the shelf' (like they already do for game-based testing). Maybe we should crowd source some tests for them....
wintermute000 - Saturday, September 6, 2014 - linkyeah I second that. SQL, virt, photoshop, blender, cubase etc. should be the kind of tests not games and office apps!!!!
Kevin G - Sunday, September 7, 2014 - linkI personally don't mind the office and gaming benchmarks as they provide continuity between reviews of consumer class hardware. It is irritating is they're the only benchmarks to be had though. A workstation motherboard needs some workstation tests!
duttyfoot - Tuesday, September 9, 2014 - linki feel the same, if you review this type of system you should at least add cinebench along with all the other applications mentioned.
MikeMurphy - Wednesday, September 10, 2014 - linkI really enjoyed the gaming benchmarks. I've always wondered how these boards would perform in those circumstances.
Samus - Thursday, September 4, 2014 - linkTo show this thing is a monster, except when gaming ;)