Argonne National Laboratory’s supercomputer, MIRA, is an IBM BlueGene/Q system and ranked the #3 supercomputer in the world. MIRA performs at 8.15 petaflops or 8.15 quadrillion floating-point operations per second. It can conduct 10 quadrillion calculations per second. Argonne’s goal is to have MIRA handle more than 5 billion computing a year when the system is in full production.
The design/build project, for which IMEG served as Engineer of Record, implemented an earlier study performed by IMEG of the existing infrastructure system to determine expansion needs to support the new water-cooled supercomputing system within the Theory and Computing Science (TCS) Supercomputing Support Facility (SSF). Each of the 50 computer racks added to the existing data center contributed 100 KW of heat gain, approximately 90% of which was rejected to water. Air cooling was provided to accommodate the remaining 10% along with the corresponding I/O and storage racks.
Specific infrastructure improvements included:
- Adding two 50,000 CFM recirculating air handling units to accommodate the anticipated increase in air cooled computer load
- Adding stainless steel medium temperature cooling pumps and piping infrastructure to support the water cooled computer load
- A central plant tie-in for the existing low temperature cooling loop, which provides additional capacity and redundancy
- Humidification for the existing data center
- Electrical feed and distribution improvements necessary to support the required power
- Addition of an emergency generator
- Modifying the VESDA fire suppression system to require two smoke inputs prior to water release
IMEG also worked with Argonne for budgeting and infrastructure conceptual planning for the next supercomputing system, anticipated to arrive in 2016. Since these supercomputers do not exist when planning begins, initial planning is based heavily on assumptions and input from the super computer vendor community. Planning begins prior to Argonne knowing which computer vendor they will select. After the vendor is selected, these assumptions are confirmed in an iterative process by the vendor. At each stage, IMEG updates the infrastructure planning and budgeting. This process typically requires several iterations. Construction of the infrastructure improvements begins prior to the computer being fully designed and tested so that the infrastructure is in place and ready for connection when the computers arrive.