LUMI – The Most Powerful Supercomputer In Europe

LUMI (Large Unified Modern Infrastructure) is the most powerful supercomputing centre on the European continent and fifth in the world, according to the Top500 list of June 2024.

The supercomputer is located in the small town of Kajaani, in central Finland and just under 600 kilometres north of the Finnish capital, Helsinki.

Silicon was invited to view first-hand these facilities and all the details that have allowed LUMI to become a supercomputing monster, comparable to the combined power of 1.5 million state-of-the-art laptops in a space the size of two tennis courts.

LUMI – project origins

LUMI is part of the EuroHPC JU (European High-Performance Computing Joint Undertaking), a joint European Union effort, the CSC (Sunkiah Science Centre) and several member states (Finland, Belgium, Denmark, Estonia, Norway, Poland, Czech Republic, Sweden, Switzerland, the Netherlands and Iceland) to build a world-class supercomputing infrastructure in Europe.

The objective of EuroHPC is to establish Europe as a global benchmark for supercomputing, promoting scientific progress and industrial competitiveness.

In 2019 Finland was selected to host one of the most advanced supercomputers in Europe within the EuroHPC project. Kajaani was chosen as the ideal place for the project due to its geographical and energy advantages, such as cold weather, which helps reduce cooling costs, and the availability of 100 percent renewable hydroelectric power, making LUMI one of the world’s greenest supercomputers.

Indeed, the city is surrounded by lakes with various hydroelectric power plants that generate energy for the entire region.

Construction began in 2020 in an industrial building that had previously functioned as a paper factory, which was closed down in 2008. Shortly after its closure, the building was adopted by the CSC to house its data centre, which began the process for the eventual construction of LUMI.

PICTURE – The former paper factory that houses the LUMI supercomputer has space for future expansions

The facility was completed in 2021 and LUMI became operational at the end of that year in its first phase. Currently, the supercomputing centre is in its third phase of deployment and is achieving full performance in multiple applications.

LUMI – technical specifications

During the visit to the LUMI supercomputing centre, Silicon had the pleasure of meeting Pekka Manninen, director of LUMI Leadership Computing Facility, who acted as our guide to this remarkable facility.

LUMI itself is built using the HPE Cray EX architecture – a system specialising in high performance computing. Its configuration is based on AMD GPUs and CPUs, the only manufacturer that develops both units for this type of workload.

The guide Manninen explained that the selected AMD MI250X GPUs are unique in their class due to their technical supremacy and watt performance.

Specifically, the GPU partition (LUMI-G) consists of 2,978 nodes, each with an AMD Trent CPU 64 cores and four AMD MI250X GPUs, which translates into a total of 11,912 AMD GPUs.

For its part, the CPU partition (LUMI-C) has 2,048 double socket CPU nodes with 64-generation AMD EPYC chips and between 256GB and 1024 GB of memory. In total, more than 262,000 CPU cores.

The system has an additional 32TB memory partition. In the storage section, LUMI consists of different levels depending on workloads. There are 10 Flash storage PB for short-term short-term fast access, 80 PB storage on traditional hard drives for longer term storage, and 30 PB to share and store data over the life of each project.

All partitions (CPU, GPU and storage) are connected via 200 Gbit/s Cray Slingshot connections.

These and other more complex specifications, have allowed LUMI to be placed in fifth place in the Top500 list, with a sustained speed of 379,70 PFlops/s and capable of reaching peaks of 531,51 PFlops/s.

As Silicon readers will know, the Flop/s measure refers to the Operations in Coma Floating per Second that a computer is able to make. This measure has become the benchmark for measuring the performance of high-performance computing systems.

The industry has been watching the GigaFLOPS (GFlop/s), the TeraFLOPS (TFlop/s), the PetaFLOPS (PFlop/s) pass and there are already supercomputers who have broken the barrier of the ExaFLOPS (EFlop/s). However, LUMI is able to carry out more than 379 PFlops/s, or what is the same: 379 quadrillions of Operations in Floating Coma per Second on a sustained basis.

LUMI – applications

Supercomputing centres such as this are designed to solve the most complex calculations that humans face.

Pending quantum computing becoming a reality, high-performance computing (HPC) is currently allowing for advances in multiple fields such as scientific research, health and biomedicine, as well as artificial intelligence and machine learning.

LUMI is designed to help solve the most complex problems currently facing modern science. It allows researchers to perform large-scale climate simulations, model the behaviour of subatomic particles, and explore new frontiers in theoretical physics.

It also allows molecular interactions to be modelled, as well as simulations that accelerate the discovery of drugs, in order to aid in the early detection of cancer and more efficient treatments to reduce mortality rates.

For example, during the Covid-19 pandemic, LUMI played an important role in modelling the spread of the virus and in research on possible treatments.

More recently, ICEYE uses LUMI’s computing capability to analyse in real time radar-generated data from its microsatellite system and turn it into images of the explored terrain, allowing fires, flooding or any other environmental disaster to be detected independently of local weather conditions.

It should be noted that ‘Destination Earth Climate Adaption Digital Twin’ is a case of particularly relevant use that is already running in the supercomputing centre. Basically, this is a new type of climate information system that can be used to assess the effects of climate change and the various adaptation strategies at local and regional levels over several decades. It is a digital earth twin in which all kinds of circumstances are simulated and analysed with unprecedented resolution, which will allow scientists to predict virtually any kind of natural disaster.

Other LUMI use cases that were publicly disclosed during the visit are closely related to artificial intelligence, such as the development of a large and open language model for the scientific community, called OLMo. Because it is an open model, scientists around the world can collaborate and exploit the potential of a language model that already has 70 billion parameters since its first version was released earlier in 2024.

The role of the Finnish CSC

As mentioned above, the Finnish CSC is the organisation responsible for the maintenance, cooling and updates of the LUMI supercomputer, but also enables the scientific research and any other use that may be given to the supercomputer. As a scientific institution, the CSC aims to facilitate its use to researchers, academic institutions and companies from the countries previously mentioned.

In this way, the CSC ensures that LUMI is available for projects of various kinds and other projects that require high-performance computing capabilities.

This process is carried out through competitive applications, where the most promising projects that can benefit from the use of the supercomputer are selected.

Moreover, among the functions of the CSC are to provide technical support to researchers – helping them to make the most of LUMI’s calculation power.

The heart of LUMI – AMD’s open technology

For the past two or three years, especially since ChatGPT broke into everyday life as the most popular generative AI system, the world has not stopped talking and writing about artificial intelligence.

The entire IT ecosystem, from manufacturers to independent software developers, to integrators and distributors, have sought to adopt AI without hesitation. AI services are being developed that could change the lives of millions of people, just like smartphones did in the 2000s.

Artificial intelligence is not something new. It has been under development for several decades, but what is happening now is that complex algorithms and large language models require large amounts of computing capacity, which is typically executed via the cloud and powerful supercomputers such as LUMI.

In all these AI systems that are processing huge amounts of information and algorithms, the common component are GPUs, that efficiently support and manage all these processes and resolves the typical input and output requests of the CPU.

While it is true that GPU manufacturer Nvidia has become something of an AI benchmark, it should be noted that AMD is not far behind.

AMD’s strengths include the ability to design and deliver both GPUs and CPUs, its efficiency per watt consumed, and its strategy to develop and support an open software ecosystem through AMD ROCm, which allows developers to optimise AI and HPC workloads on AMD GPUs.

“In the areas of AI and High Performance Computing you need to think of all the elements as a whole and not separately,” Alexander Troshin, AMD’s EMEA HPC and Enterprise Product Marketing Manager told Silicon. “Developing GPUs and CPUs and using an open ecosystem to get the most performance and efficiency out of them is critical to succeed in these complex projects.”

The result of this AMD strategy greatly facilitates versatile and efficient AI implementations, both at the machine learning and inference levels – the two main tasks in these workloads.

And this is not just AMD saying this, as the Top500 list reveals that two of the five most powerful supercomputers worldwide are built with AMD technology. The fifth is LUMI, while the first on this list, the Frontier system, was the first to break the ExaFLOP barrier (1.2 EFlop/s), which is a milestone in the history of supercomputing.

This is just the beginning. Over the coming months, the world can expect to see combinations that will break new ground with the combination of 5th generation AMD EPYC CPUs, AMD Instinct MI350 GPUs (expected in 2025) and connectivity for UALink and Ultra Ethernet HPC environments.