Germany's High-Tech Treasure

At LRZ in Munich, a Massive New SGI Supercomputer Ushers In a New Future for Scientists throughout Europe

For Scientists at the Leibniz Computing Center (LRZ) in Munich, theoretical computing performance is not nearly the whole story. What matters to them — and to hundreds of users throughout Bavaria, Germany and Europe — is how Germany's new national supercomputer will help them solve today's most daunting scientific problems.

So LRZ's phase I installation of its massive SGI® Altix® 4700 supercomputer came as a welcome milestone to Europe's scientific community. Nothing short of a national treasure in Germany, the new 64-bit Linux® blade system is packed with 4,096 Intel® Itanium® 2 processors and features an impressive 17 trillion bytes (Terabytes) of globally addressable memory.

"The key criterion in choosing this system was not an abstract peak performance quantity," notes Dr. Horst-Dieter Steinhöfer, head of the high-performance computing (HPC) department at LRZ. "The top criterion was sustained performance while running a realistic load mix of programs for solving significant scientific problems that overload smaller systems. On those terms, we can expect our new LRZ national supercomputer to be one of the most powerful systems in the world."

To researchers working with increasingly complex simulations in physics and astrophysics, materials research, fluid dynamics, chemistry, geosciences and biological sciences, the new system is about saving time: time-to-results, time-to-solution and time-to-innovation.

Even so, the LRZ supercomputer ranked 18th in the Top 500 of the world's supercomputers with a peak performance of 26.2 TeraFLOPS in the November 2006 rating.

Flexible, Powerful and User-Friendly
The system's users will achieve those time savings in large part because of the SGI Altix system's remarkable flexibility. The Altix system can run programs:

• Written for distributed memory architectures (MPI codes), allowing users access to up to 4,000 processors;
• Written for shared-memory architectures (OpenMP codes), with up to 256 processors accessible today in shared-memory mode, and increases planned as needs warrant; and
• Written as hybrid codes, leveraging a flexible combination of both architectures

LRZ administrators expect the Altix system will help researchers achieve innovative breakthroughs in all application areas, with programming of parallel application codes growing significantly easier.

For example, consider astrophysicists and fluid dynamics researchers. They have used supercomputers since their inception, generating some of the most complex parallel processing tasks ever seen, and thus have grown familiar with how to get the most from a supercomputer. But for world-class HPC resources to be also widely used in other scientific disciplines, users require a user-friendly supercomputing environment.

Fortunately, the shared-memory SGI Altix platform provides that kind of usability. Access to a massive main memory pool, for example, gives genome and proteome researchers the ability to hold massive databases entirely in memory. Just that feature alone can result in an immense acceleration of the computation task, because the system need not access disk storage for data.

While LRZ's SGI Altix 4700 system is comprised of nodes totaling 256 processors each, the system can scale to up to 1,024 processors operating under a single instance of Linux. A new record for Linux scalability, this feat was in fact achieved by SGI on LRZ's own system running SuSE® Linux Enterprise Server 10 from Novell. As users demand larger single system image configurations, LRZ plans to scale one or more nodes to 512 and, perhaps one day, 1,024 processors.

A Varied Waiting List
For now, however, a growing list of challenging projects is queuing up for their turn on the LRZ's national supercomputer system.

The new Altix resource is poised to help scientists and researchers investigate the nature of turbulences and their flow through porous media, the interaction of flows and deformable structures, the generation and propagation of acoustic waves, high-temperature superconductivity, the investigation of materials exhibiting memory effects with regard to their form, the analysis of chemical reactions involved in combustion and catalytic processes, the propagation of seismic waves and earthquakes, and the discussion of proteomics, which investigates the relationship between the sequence, structure and function of proteins.

All of these projects have undergone a careful peer review among candidates applying to use the LRZ system. These researchers will leverage an exceptionally equipped resource. In addition to its 4,096 processors and 17TB of memory, the system features 300TB of FiberChannel disk storage that is directly attached to the partitions for fast repeated storing and access of data common to simulation runs. A 10 Gigabit Ethernet network connects the system to the German scientific network and to the DEISA group of European national supercomputing centers with appropriately high network bandwidth.

While the phase I implementation runs on single-core Intel Itanium 2 processors, Phase 2 improvements (slated for summer 2007) include an upgrade to dual-core Intel Itanium 2 processors, an increase to 40TB of memory, and a doubling of disk storage capacity. As a result, real-world application performance is expected to nearly double after Phase 2 is completed.

While deployment of the SGI system represents a coup for the state of Bavaria and for the nation itself, LRZ views the supercomputer as an international resource.

"The new supercomputer at LRZ provides a significant advance over the previously existing HPC infrastructure with respect to its architecture, its versatile employability and its user-oriented capabilities," said Prof. Dr. H.-G. Hegering, chairman of LRZ's directorate. "It enables LRZ to deliver a uniform computing environment from the desktop up to the top-level HPC system. This powerful new resource will improve the region's competitiveness by delivering world-class compute performance to Germany. In fact, with this new SGI Altix system, we can finally enter the coveted competition for hosting the European supercomputing system."

LRZ's Real-World Proving Ground
In 2004, Germany's top computing experts set out to select the computer that would someday become the country's national supercomputer. The group, led by LRZ, realized that theoretical peaks in performance had little bearing on the day-to-day reality that system users would face.

So LRZ established a suite of benchmark tests to provide a foundation for its supercomputer selection process. Designed to represent the mix of the applications likely to run on the system, the tests were conceived to reliably predict the sustained performance that real-world users could expect from each system being evaluated by LRZ.

The test suite included a Large-Eddy simulation (Fluid dynamics, CFD, with classical discretization methods): a Lattice-Boltzmann application (CFD, new approach using particle collision): a quantum chromo dynamics application and three program codes from astrophysics, high energy physics and quantum chemistry; various kernel benchmarks (such as Fourier analysis and matrix multiplications) that test loop structures and standard library routines heavily used in parallel applications. Tests were geared toward applications written for both distributed memory architectures using MPI and shared-memory architectures using OpenMP.

When the tests were complete, one system proved superior over all others. "SGI's NUMAflex™ platform yielded the highest performance of all tested systems in five out of six application benchmarks, and led the field for the aggregate total performance," recalls LRZ's Dr. Matthias Brehm. "We also checked the internal communication and I/O capacities. In these areas, the SGI Altix again proved itself the leading platform. The SGI machine led the competition with respect to the entire benchmark mix. It scales well and allows very flexible usage possibilities."

---

Images courtesy of Helmut Payer and Kai Hamann, produced by gsiCom.