Well-used supercomputer auctioned off for 480,000 US dollars

Contents

The US government has sold the "Cheyenne" supercomputer at auction for around USD 480,000. The system was in operation until the end of 2023 and must be picked up by the unknown buyer at the NWSC data center in Cheyenne, which is the capital of the US state of Wyoming.

The process is interesting because the handling of the sale, the data and the now historical significance of the machine offer an insight into the changes to supercomputers. As a rule, these are first decommissioned and later sold piece by piece or used in parts for other tasks - and sometimes simply scrapped by the operators. When Cheyenne was built in 2016, it was ranked 20th in the top 500 list of the world's fastest computers. It was mainly used for research projects at various universities and other scientific institutions.

No longer economical to operate

With a service life of seven years – originally only five were planned – Cheyenne has achieved a fairly long service life for these machines. Due to increasing computing power in chip development combined with decreasing power consumption, longer use than typically two to four years is not economical.

At its peak, Cheyenne required 1.7 megawatts for a computing power of 5.4 petaflops (Rmax). Today, the booster module of the "Juwels" machine at the German research center Jülich, for example, can achieve around 71 petaflops for roughly the same electrical power, i.e. over thirteen times as much performance. The comparison is not entirely fair because Juwels - like most of today's fastest supercomputers - mainly computes with Nvidia GPUs, in this case 3744 H100 modules from Nvidia. Not all applications can be parallelized so well that running them on GPUs is worthwhile.

Who needs 8064 old Xeons?

However, the US computer now being auctioned dates back to the last years of the era of x86 CPUs with many cores for supercomputers. Cheyenne calculates with 8064 E5-2697v4 Xeon CPUs, each of which has 18 Broadwell cores. Based on Intel's list price - which research institutions generally do not have to pay anywhere near - the processors alone were worth over 21 million US dollars in 2016.

However, using such technically obsolete hardware from a supercomputer for continuous computing today makes little economic sense, especially as the components can hardly be tested for reliability at silicon level. This also applies to the 16 GByte DIMMs of DDR4 memory that comprise the 18 or 9 terabytes of RAM per rack at Cheyenne. Even ECC memory can become unreliable over the years due to constant continuous load, the errors are only directly noticeable there and do not only manifest themselves through crashes or calculation errors.

The price for Cheyenne therefore seems quite high and is likely to be significantly higher than the scrap value of the components, even if a lot of copper is used for cables and heat sinks due to the water cooling. According to the operators, the constantly leaking cooling system is also a reason for selling the machine. If you want to put it back into operation, even if only partially, you also need a data center that is designed for water cooling. A conversion to air cooling is hardly possible with the racks and blades built by SGI, because it involves a high density of computing power: 288 Xeons are in a rack that weighs over a ton.

51 racks have to go

For this reason, the removal of Cheyenne, as the sellers expressly emphasize, can only be handled by specialists - of course, the machine has to be picked up on site. The computer consists of 28 compute racks weighing 1087 kilograms each, 14 racks with cooling units weighing 635 kilograms each, 7 racks for water distribution (374 kilograms) and 2 racks for I/O weighing around 500 kilograms each. And anyone who is now perhaps looking at the scrap metal price will also be disappointed: the data center states that Cheyenne has been completely separated from cables, fiber optics and water pipes, these parts are not part of the purchase.

Cheyenne's successor is called Derecho, is located in the same data center and has already been put into operation. With 19.9 petaflops, not yet rated in the top 500 lists, it is three and a half times as fast; no information is yet available on its energy requirements. Derecho consists of 2488 compute units with AMD's Epyc CPUs (Milan), which are supported by 82 GPU units with a total of 328 A100 GPUs from Nvidia.

(nie)