Supermicro crams 18 GPUs into a 3U box • The Register

GPU-enhanced servers can usually pack as much as eight of the accelerators, however Supermicro has constructed a field that manages to suit 18 of them inside an air-cooled chassis that’ll eat up simply 3U of rack area.

The delightfully named SYS-322GB-NR sports activities 20 PCIe slots, with the expectation that is the place you will join GPUs. That is an uncommon association nowadays: most AI servers supply Nvidia’s SXM socket or use the Open Accelerator Module spec, as each supply extra inter-chip bandwidth than PCIe.

However this field is not designed to do the heavy lifting required of different AI servers. Supermicro suggests this machine for jobs like operating machine studying and AI inference workloads on the edge, as a part of automated manufacturing techniques that require information to be processed from digicam feeds or sensors at very low latencies. One other recommended function is utilizing GPUs devoted to graphics fairly than AI and connecting as much as 64 displays – the kind of factor that will get visualization wonks excited on the prospect of constructing 46,080 x 12,960 pixel shows.

In the back of the system is room for 18 single-slot GPUs or ten dual-slot playing cards. Or a minimum of that is what the press launch claims – the advertising and marketing imagery appears to point eight twin slot playing cards, although that could be much less of a bodily restrict and extra an influence and cooling one.

Supermicro would not say which playing cards it will assist – maybe as we’re between main releases from Nvidia and AMD – however does observe that accelerators from each distributors are on the menu.

For edge AI, we suspect Nvidia’s diminutive L4 accelerators might be a well-liked configuration. In the meantime, for those who want a bit of further grunt, a financial institution of ten Nvidia L40S GUPs churning out 3.6 petaFLOPS of dense FP16 efficiency could be the ticket – assuming the PSU can provide roughly 5.5kW of energy we estimate such a configuration would wish below load.

Supporting all these GPUs is not trivial both. On the coronary heart of the system are a pair of Intel 6900-series Xeons with assist for as much as 128 cores, 256 threads, and 96 lanes of PCIe 5 a bit, which feed 20 PCIe slots on the motherboard. The observant amongst you’ll observe that even with 192 PCIe lanes, that is nonetheless not almost sufficient for 18 – not to mention 20 – PCIe x16 slots.

It is not clear if Supermicro solely helps eight lanes per slot when absolutely populated or if it is utilizing a PCIe swap to beat the limitation. If we needed to guess, it is most likely the previous. Except the GPUs have to shuffle information between each other, eight lanes per slot might be positive. And in the event that they do, Supermicro sells techniques higher suited to that use case. In any case, we have reached out for remark concerning energy and PCIe bandwidth and can let you realize what we discover out.

Past the sheer variety of PCIe slots at your disposal, the system is in any other case a vanilla server that helps as much as 6TB of DDR5 or, in the event you favor one thing speedier, 8,800 MT/sec MRDIMMs. Storage can be fairly commonplace, with assist on your alternative of both 14 E1.S or 6 U.2 NVMe drives.

Oh, and if GPUs aren’t your factor however dense memory-packed servers are, Gigabyte just lately announced a twin socket Epyc system with a whopping 48 DIMM slots. ®