Skip to main content




In almost all fields of physics, such as atomic and molecular physics, solid state physics, plasma physics, hydrodynamics, electrodynamics, quantum mechanics, quantum chromodynamics, high-energy physics, astro- and geophysics, fundamental new results were achieved by meands of High Performance Computing. It provides the possibility of analyzing complex systems by the aid of simulation. This procedure is as important as theoretical investigations and closely interacts with the latter. Numerical simulation is more and more becoming a pillar equivalent to the two classical pillars of gaining knowledge, namely the theoretical investigation and the experiment. Corresponding to the problems involved and the methods and tools used, “Wissenschaftliches Rechten” — the German designation — has a strongly interdisciplinary character by integrating contributions from different fields of natural sciences, applied and numerical mathematics as well as informatics. In the following, out of numerous projects in the field of physics currently running at the HLRS, 14 projects were selected to illustrate the scientific progress which can be achieved with the supercomputer resources of the HLRS with examples from geophysics, astrophysics, molecular dynamics and solid state physics.

Werner Hanke, Hans Ruder

Finite difference modelling of elastic wave propagation in the Earth’s uppermost mantle

Most models of the Earth’s upper mantle had previously assumed a homogeneous elastic structure. In contrast many seismological data sets show conclusive evidence for strong scattering. For instance scattering within the uppermost mantle is prominently documented in the so-called high-frequency teleseismic P n phase, generated by mantle velocity fluctuations. This phase, and its correspondence S n , are seen globally in data sets from active and passive seismology.We demonstrate that a wave guide, which is caused by random fluctuations of the mantle’s elastic properties can explain the main features of the teleseismic P n . We focus on the statistical properties of these fluctuations acting as scatterei. To test the hypothesis of an upper mantle scattering wave guide we calculate synthetic seismograms and compare them with observations. To compute realistic seismograms we solved the elastic wave equation numerically. Using a 2D finite difference scheme we calculate synthetic seismograms for a variety of very large models (larger than 1000 wave length). The size of the models employed and the number of time steps computed are unprecedented so far and inconceivable without modern high-performance computing. We developed and optimized an efficient code with High Performance FORTRAN (HPF) for a massive parallel computer system. We discuss the influence of the vertical and horizontal correlation length, RMS velocity fluctuations, and thickness of the heterogeneous layer on the scattering properties of the upper mantle and on the propagation mechanism of the teleseismic P n .

Trond Ryberg, Marc Tittgemeyer, Friedemann Wenzel

Direct Simulation of Seismic Wave Propagation

Full wave form modeling techniques in 3D complex heterogeneous elastic media axe computationally expensive, even on today’s largest supercomputers. In contrast asymptotic techniques only compute a small subset of the wave field, but can be very fast. Using asymptotic methods in combination with full wave form techniques can speed up the overall computation. I use a particular asymptotic method to compute dynamically the progressingly active computational domain for a parallel full wave form finite difference technique. This leads to an overall decreased computational runtime. This computational technique is applied to simulation in a industrially oriented oil/gas bearing subsruface model as well as to simulation of earthquake related scientific scenarios.

Martin Karrenbach

Summary of Project 11172

The project 11172 is embedded in the Collaborative Research Center 381’Charakterisierung des Schädigungsverlaufes in Faserverbundwerkstoffen mittels zerstörungsfreier Prüfung’ (SFB 381). The SFB 381 consists of theoretical and experimental groups in Stuttgart and one theoretical group in Karlsruhe. With help of the CrayT3E we perform large-scale numerical solutions of the elastodynamic wave equation in 2-D and 3-D. These solutions are required to get a better understanding of complex wave phenomena and hence to develop new non-destructive testing methods. Our work on the CrayT3E is based on the program Ultimod [1,4]. The parallel HPF-implementation has been done in the Cray-projects PARSEP and ERDMOD in Stuttgart described in [2,3]. In one part of our project we have modified this program in order to solve our specific problems. This is described in the article ’Modeling of elastic waves in fractured media’ in this report. The other three articles are related to SFB 381 topics that are in connection with our Cray-project and rely very much on synthetic data generated within this project.The resource requirements of our program strongly depend on the given problem (model-size, timesteps to calculate, …). Until now, we restrict ourselves to relatively small jobs (N64med-queue), but with our new approach — high contrast 3D-FD-modeling — we are going to calculate very large-scale problems.

Erik H. Saenger, Peter Hubral, Andreas Kirchner, Kai-Uwe Vieth, Tobias Müller, Serge A. Shapiro

Development and Astrophysical Applications of a Parallel Smoothed Particle Hydrodynamics Code with MPI

Smoothed Particle Hydrodynamics (SPH) is a particle method to simulate compressible fluids. First we present a brief introduction to SPH, where we focus on an approach to treat the physical viscosity. Then we describe in detail the basic principles of our parallel implementation of the SPH method. The efficiency of the code on Cray T3E and IBM SP2 is discussed. In the last part we present a short introduction to accretion disks in interacting binary stars and report on some new results in that field achieved with our code.

Stefan Kunze, Erik Schnetter, Roland Speith

Collisional dynamics around black hole binaries in galactic centres

We present the first results from simulations applying a hybrid “self consistent field” (SCF)1 and direct AarsethN-body integrator (NBODY6)2 which synthesises the advantages of the direct force calculation with the efficiency of the field method. The code is aimed for use on parallel architectures and is therefore applicable for collisional TV-body integrations with extraordinarily large particle numbers (> 105). It opens the perspective to simulate the dynamics of globular clusters with realistic collisional relaxation, as well as stellar systems surrounding a supermassive black hole in galactic nuclei.

Marc Hemsendorf

IMD — A Massively Parallel Molecular Dynamics Package for Classical Simulations in Condensed Matter Physics

We describe the current development status of IMD (ITAP Molecular Dynamics), a software package for classical molecular dynamics simulations on massively-parallel computers. IMD is a general purpose program which can be used for all kinds of two —and three-dimensional studies in condensed matter physics, in addition to the usual MD features it contains a number of special routines for simulation of mechanical properties of solids, analysis and visualization.

Johannes Roth, Jörg Stadler, Marco Brunelli, Dietmar Bunz, Franz Gähler, Jutta Hahn, Martin Hohl, Christof Horn, Jutta Kaiser, Ralf Mikulla, Gunther Schaaf, Joachim Stelzer, Hans-Rainer Trebin

Symmetric diblock copolymers confined into thin films: A Monte Carlo investigation on the CRAY T3E

We present the results of large scale computer simulations targeted at investigating the phase stability and the structure of symmetric AB diblock copolymers in thin films. The connectivity of the two different monomer species A and B in the diblock copolymer prevents macrophage separation and the molecules assemble into A-rich and B-rich domains on the scale of the molecule’s extension. This large length scale of the ordering phenomena makes these polymeric systems a promising candidate for revealing the universal features of self-assembling in amphiphilic molecules. However, the widely spread length and time scales impart protracted long relaxation times to the systems and pose a challenge for computer simulations. Parallel supercomputers like the CRAY T3E are necessary for equilibrating systems with structure on very different length scales. In order to investigate the stability of different phases we quench several systems with identical parameters from the disordered state to a temperature in the intermediate segregation regime and monitor the morphology. Three film thickness are investigated and we find lamellar phases which are parallel and perpendicular oriented lamellar phases. The results are compared to analytical calculations, and the influence of capillary waves on the profiles in the thin film are discussed.

T. Geisinger, M. Müller, K. Binder

Molecular Dynamics of Covalent Crystals

A molecular mechanics-like two and three body potential is implemented in the ITAP molecular dynamics program IMD1. First results on the simulation of covalent crystals are presented.

J. Hahn, H.-R. Trebin

Simulation of random copolymers at selective interfaces and of cross-linked polymer blends

We have investigated two complex statistical systems where the interfacial properties of a two-phase polymer system can be controlled. Using the bond fluctuation model (BFM) we have first studied the adsorption behavior of random copolymer chains at a single selective interface separating two phases. We found a new regime where the adsorbed chains extend in a brush-like fashion perpendicular to the interface as predicted from scaling arguments. We have extended our simulation model to investigate the influence of cross-linking on the phase behavior of polymer blends consisting of A- and B- chains. Here very large lattices are necessary to avoid finite size effects due to the network superstructure on large scale. A large cubic lattice consisting of 400 x 400 x 400 sites was sliced into 64 parts to allow for parallelization. Here we equilibrated 40 000 chains of length N = 100 before starting cross-linking under conditions without AB-interactions with different cross-link densities. Investigating the network topology we found an effective chemical dimension larger than three in agreement with previous investigations on smaller systems. This differs strongly from a simple homogeneous structure such as a diamond topology. Switching on the AB-interaction between the different chains within the network results in the formation of A- and B- enriched components, i.e. to microphase separation. The sizes of these regions axe much bigger than expected from a homogeneous network approach. The structure factor S(k) shows at intermediate wave numbers k a k-4-behavior, in contrast to the k-2-behavior expected from simplified models.

Jens-Uwe Sommer, Stefan Lay, Alexander Blumen

Towards the Limits of present-day Supercomputers: Exact Diagonalization of Strongly Correlated Electron-Phonon Systems

The problem of describing lattice dynamical effects in electronically low-dimensional highly correlated systems, such as, e.g., the quasi-1D charge-density wave and spin-Peierls materials or the quasi-2D high-Tc superconductors, has been a challenge to solid state physicists for a long time. At present, perhaps the only realible results come from purely numerical finite-lattice calculations. Employing the Lanczos algorithm in combination with a kernel polynomial moment expansion and the maximum entropy method on massive parallel computers, we show a way of calculating ground-state and spectral properties for models of electrons or spins strongly interacting with quantum phonons.

Gerhard Wellein, Holger Fehske

The Metal-Insulator Transition in the Hubbard Model

The Mott-Hubbard metal-insulator transition is studied in the two-dimensional Hubbard Model with and without next-nearest neighbor hopping at half-filling by a combination of Quantum-Monte-Carlo and exact diagonalization techniques. In the case without next-nearest neighbor hopping, antiferromagnetic correlations are suppressed by large fluctuations due to a relatively high tempera-ture. The single particle spectral function and the spin— and charge-excitations of the metallic state below the critical Hubbard interaction Uc, where the transtion occurs, are similar to the tight-binding result for U = 0. Above Uc, a non-isotropic gap in the spectral function opens, and flat sidebands appear in the spectrum. The gap formation is accompanied by pronounced spin— and charge-modes. In the case including next-nearest neighbor hopping, both finite-temperature Quantum- Monte-Carlo and exact diagonalization (T = 0) results yield a metal-insulator transition at about the same critical interaction Uc. However, whereas the finite- temperature transition is again of the Mott-Hubbard type, the T = 0 transition is of Mott-Heisenberg type induced by long-range antiferromagnetic correlations.

M. G. Zacher, A. Dorneich, C. Gröber, R. Eder, W. Hanke

Vibronic studies of adsorbate-covered semiconductor surfaces with the help of HPC

The technologically relevant adsorbate systems S:Ge(001)-(1 x 1) and Se:Ge(001)-(1 x 1) are prototypes for the passivation of semiconductor surfaces. Over the last years, their electronic and structural properties have been studied by ab-initio methods in detail, while the description of vibronic properties was limited to semi-empirical methods. The use of high performance computing (HPC) allows for the first time the calculation of surface phonons for these particular systems from first principles within a reasonable time, for example with the help of density functional perturbation theory. This theory, its implementation on a parallel computer and results concerning the vibronic properties of the above-mentioned surfaces are presented in this article.

Ulrich Freking, Albert Mazur, Johannes Pollmann

Computational Methods in Chemistry and Molecular Biology

In 1998, the Nobel prize in chemistry was awarded to two outstanding figures in the field of quantum molecular science, namely John A. Pople and Walter Kohn. Pople and Kohn had both made important contributions to the field over a few decades, and thus a substantial share of the work which has led to the state-of-the-art in Theoretical Chemistry and Computational Chemistry can be traced back to their influence.

Bernd Artur Hess

The multi-reference configuration interaction method on massively parallel architectures

We report on a scalable implementation of the configuration- selecting multi-reference configuration interaction method for massively parallel architectures with distributed memory. Based on a residue driven evaluation of the matrix elements this approach allows the routine treatment of Hilbert spaces of well over 109 determinants as well as the selective treatment of triple and quadruple excitations with respect to the reference space. We demonstrate the scalability of the method for up to 128 nodes on the IBM-SP2. We elaborate on the specific adaptation of the transition residue-based matrix element evaluation scheme that ensures the scalability and load-balancing of the method and discuss the projects using this methodology that axe presently under investigation: the elucidation of electronic dynamics near the conical intersection of NO2, the calculation of potential energy surfaces and electronic spectra of transition metal compounds such as Cr2 and the first row transition metal dihalides, the investigation of the electronic structure of aromatic compounds such as benzene, naphtalene and anthrazene and their derivatives and the ab-initio determination of microscopic spin- and spin-phonon coupling constants in quasi-onedimensional transition metal compounds, such as CuGeO3 and NaV2O5 in the solid state.

P. Stampfuß, W. Wenzel

Quantum Chemical Studies on Heterocyclic Rearrangements in Benzofuroxans: Reaction Paths, Vibrational Spectra, and Rate Constants

Quantum chemical calculations at the coupled-cluster level were used to investigate the reaction mechanism of the Boulton-Katritzky rearrangement and the ring chain tautomerism in benzofuroxans. The well known sensitivity of this molecular class to electron correlation effects could be overcome by the inclusion of triple excitations within the coupled-cluster approach. The computational results support experimental findings and reject a discussed intermediate in the Boulton-Katritzky rearrangement. The reactions of two prototype components were investigated, namely 4-nitrobenzofuroxan and 5-methyl-4-nitrobenzofuroxan.

Guntram Rauhut, Frank Eckert

High Level Quantum-Chemical Computations on the Cyclizations of Enyne Allenes

The ring-closure reactions of the enediyne and enyne-allene moieties of the natural antitumor antibiotics neocarzinostatin (1), calicheamicin (4), and dynemicin (5) are held responsible for the DNA-cleavage abilities of these potent pharmacophors. Neocarzinostatin, first isolated from Streptomyces carzinostaticus in 1961 [1], consists of the key chromophore 1 (Fig. 1.1) bound to a 113-amino acid apoprotein. [2,3] The enediyne moiety of 1 is activated by a thiol nucleophile, rearranges to enyne-cumulene 2, and cyclizes to the highly reactive biradical 3 (Fig. 1.1), a reaction akin to the Myers-Saito cyclization. [4] Since 1 binds to the minor groove of DNA, [5–;7] 3 is able to abstract hydrogens from adenine or thymine moieties [8] leading to cell damage. While calicheamicin 4 and dynemicin 5 (Fig. 1.1) undergo Bergman cyclizations to give rise to 1,4-didehydrobenzene biradicals, the neocarzinostatin chromophore 1 reacts differently.

Matthias Prall, Peter R. Schreiner

MD Simulation of a Phospholipid Bilayer

We report first results of a molecular dynamics simulation of a fully hydrated dipalmitoyl-sn-glycero-phosphatidylcholin (DPPC) bilayer using the NγT-ensemble. Because of the large size of the simulated system, a parallel version of the simulation package MOSCITO [1] has been developed, which employs a dynamic loadbalancing algorithm to ensure uniform workload among the processors.

Ralf Schmelter, Alfons Geiger

Three-Dimensional Organization of Chromosome Territories and the Human Cell Nucleus

Despite the successful linear sequencing of the human genome its three-dimensional structure is widely unknown. However, the regulation of genes — their transcription and replication — has been shown to be closely connected to the three-dimensional organization of the genome and the cell nucleus. On the bases of polymer physics we have recently developed detailed and quantitative structural models for the folding of the 30 nm chromatin fiber within the human interphase cell nucleus. A quantitative test of several plausible theories resulted in a best agreement between computer simulations of chromosomes, cell nuclei and experiments for the so called Multi-Loop-Subcompartment (MLS) model. Results concern the following properties: overlap of chromosome territories, -arms, -bands, 3D spatial distances between genomic markers as function of their genomic separation in base pairs, fractal analyis of simulations, mass distribution of chromatin in cell nuclei and the fragmentation distribution of cellular DNA after irradiation with carbon ions. Thus in an anology to the Bauhaus principle that “form follows function”, analyzing in which form DNA is organized might help us to understand genomic function.

Tobias A. Knoch, Christian Münkel, Jörg Langowski

Computational Fluid Dynamics (CFD)

Many products in industry, e.g. space vehicles, airplanes, helicopters, trains, cars, engines, single crystals, machines in process engineering and nuclear engineering, are extremely complex systems that combine and integrate various highly advanced technologies. Their definitions and developments are carried out in complex and time consuming processes requiring a lot of high skilled people.

S. Wagner

Parallel Computation of Interface Dynamics in Incompressible Two-Phase Flows

A numerical method for the direct numerical simulation of incompressible flows with interfaces between two fluids with totally different density and viscosity is developed. The method is applied to binary droplet collisions and to the droplet impact on liquid films. Corresponding simulation running times are reported. The single-processor performance and the parallel performance of the code on NEC SX-4 and Cray T3E/512-900 at HLRS are analyzed. It is concluded, that simulations with this code should be run on the Cray T3E/512-900 only for problems, which are too large for the NEC SX-4.

Martin Rieber, Arnold Frohn

Numerical Simulation of Fluid Flow and Heat Transfer in an Industrial Czochralski Melt Using a Parallel-Vector Supercomputer

In this paper, the fluid flow and heat transfer in an industrial Czochralski melt was analyzed by solving the time-dependent three-dimensional Navier-Stokes equations on curvilinear boundary-fitted grids in a rotating frame of reference. In order to represent the ellipsoidal crucible, a grid with 720,896 control volumes was generated in six blocks. Using the natural advantage of block-structuring, computations were carried out on a parallel-vector machine (NEC SX-4) with an optimal load-balancing efficiency of 100% using four processors. Simulations of the flow field were performed with and without the k — ε turbulence model. It was seen that the turbulence model suppresses the fluid mechanical instabilities leading to an axisymmetric flow and thermal field, while the simulations without the turbulence model were found to predict the three-dimensional time-dependent features of the melt flow well. A total performance of 2.95 and 2.81 GFlops on four processors was reached for the simulation with and without turbulence model, respectively.

Sven Enger, Michael Breuer, Biswajit Basu

Numerical flow simulation in cylindrical geometries

The CFD-code MGLET, which is optimized for direct and large-eddy simulation on high performance computers, has been extended by a cylindrical version allowing to compute flow problems in cylindrical geometries. Laminar pipe flow and cylindrical Couette flow have been simulated to test the implementation. Preliminary results of Taylor-Couette flow and turbulent boundary layer flow along cylinders are shown to demonstrate the applicability of the new code-version.

T. J. Hüttl, M. Smieszek, M. Fröhlich, M. Manhart, R. J. D. Schmidt, R. Friedrich

DNS of Laminar-Turbulent Transition in Separation Bubbles

A Laminar Separation Bubble (LSB) is created under the influence of an adverse (positive) pressure gradient along a wall by laminar separation, laminar-turbulent transition and turbulent re-attachment of the flow to the wall. Direct Numerical Simulations (DNS) of the flow in an airfoil boundary layer which are based on solving the full Navier-Stokes equations exhibit good quantitative agreement of the mean-flow data with wind-tunnel experiments, but only examinations of the unsteady results are able to reveal the underlying physics. Thus, a hitherto unknown temporal amplification of three-dimensional small-amplitude disturbances is observed and explained by the entrainment of three-dimensional fluctuations by the roll-up of the detached boundary layer. Once the three-dimensional disturbances become saturated this mechanism leads to a rapid breakdown of the laminar flow into regions of small-scale turbulence which are organized in a quasi two-dimensional coherent manner.

U. Maucher, U. Rist, M. Kloker, S. Wagner

Numerical Simulation of Supersonic Hydrogen-Air Combustion

In this investigation, unsteady, three-dimensional, supersonic flow with nonequilibrium chemistry in a square channel with transverse hydrogen injection was numericaly analyzed. To this end, the concepts of large eddy simulation (LES) were applied to a model supersonic combustion chamber using the three-dimensional solver of the compressible Navier-Stokes equations with chemical reactions called “ACHIEVE”. The time accurate computation was accelerated by an implicit method.

Ernst von Lavante, Michael Kallenberg, Dirk Zeitz

Computation of Turbulent Flows with Separation by Coherent Structure Capturing

Within the scope of technical application it is necessary to simulate flows with separation not only at “academic” Reynolds numbers by direct numerical simulation (DNS), but also at higher Reynolds numbers by means of large eddy simulation (LES) or very large eddy simulation (VLES). The flow around a sphere at Re=50 000 has been calculated on unstructured grids with the Smagorinsky subgrid model and data is compared with available measurements. It is necessary to evaluate existing models and test them on simple geometries for the future application on real problems. Results indicate that the agreement between simulation and experiment is within the range of the experimental error.

M. Schmid, M. Perić

Large Eddy Simulation of the Flow around a Circular Cylinder

Large-Eddy Simulation (LES) is a promissing concept for modelling turbulent flows. The principle is to solve the three-dimensional, unsteady Navier Stokes equations on a grid which is too coarse to capture all vortical motions. The unresolved part of the flow field, constituted by fine-scale motion, is accounted for by an additional so-called subgrid-scale model term in the equations. This approach allows to reduce the computational cost for the same Reynolds number or to compute the flow at a much higher Reynolds number.

Jochen Fröhlich, Wolfgang Rodi

Direct Numerical Simulations of an Adverse Pressure Gradient Turbulent Boundary Layer on High Performance Computers

Direct Numerical Simulations (DNS) of developing turbulent boundary layers have been performed using a Finite Volume method with a locally refined grid near the wall. In the case of a zero pressure gradient boundary layer it is shown that the use of local grid refinement saves a considerable amount of computational resources. The method is then applied to an adverse pressure gradient boundary layer. The comparison of the results with an experiment performed by Watmuff [10] shows that the method is well suited for strongly developing turbulent flows.

Michael Manhart

Aeroelastic Analysis of a Helicopter Rotor in Forward Flight

Flow fields around rotary wings, especially around helicopter rotors, are extremely complex due to strong compressibility and three-dimensional effects. The solution of the governing equations for these flow problems is very difficult and can usually only be achieved with the help of numerical methods.

B. Buchtala, K.-H. Hierholz, S. Wagner

Flow with chemical reaction

The numerical simulation of flows with chemical reactions, in particular with turbulent combustions, is still one of the most difficult challenge in scientific computing. Because of the limitation of CPU time and memory resources the resolution of all length and time scales for realistic applications is still impossible. But due to the extensive use of parallel computers the investigations in this area are growing up rapidly and the results become more and more profitable. While some years ago the main goal was the adaption of the software to the architecture of parallel computers and to develop new software for it, we have today more reliable results which are in good agreement with measurements and which sometimes can be used already for design and development. Most of the computations have been performed in 3D. Three of the following projects have been run on the CRAY T3E, three on the NEC, and 1 of them on both machines. Most of the groups have used MPI.

Dietmar Kröner

Investigation of Chemistry-Turbulence Interactions Using DNS on the Cray T3E

We use direct numerical simulation (DNS) to study the influence of turbulence in chemically reacting flows. First a short description of the governing equations, their numerical solution, and performance results on the Cray T3E are given. Then two applications are presented: The temporal evolution of a turbulent premixed methane-air flame is simulated using a chemical mechanism including 15 species and 84 elementary reactions. Isolated pockets of cold fresh gas are formed propagating into the burnt side of the flame. Examples are given for the combined influence of curvature and preferential diffusion on the chemical composition in the flame front. A correlation analysis shows that it is possible to use the formyl Radical as an indicator for the reaction intensity. The second application is the DNS of autoignition processes in turbulent hydrogen-air mixing layers. A reduction of ignition-delay time compared to the laminar case is observed and a recent result is presented concerning the locations where the first ignition spots occur.

Marc Lange, Jürgen Warnatz

Multigrid Convergence Acceleration for Non-Reactive and Reactive Flows

The numerical simulation of chemically reacting flows requires extremely long CPU times if finite-rate chemistry is employed. Multigrid methods belong to the most promising methods for convergence acceleration and are widely used for non-reactive subsonic and transonic flows. This paper presents the application of an implicit multigrid algorithm to non-reactive and reactive turbulent high speed flows. Special care is required for the treatment of shock waves as well as chemical and turbulent source terms to enable convergence. CPU time reductions by a factor of at least 4 are presented for a non-reactive flow with streamwise air injection and a lifted methane-air flame. All calculatione are performed on the NEC SX-4.

Peter Gerlinger, Dieter Brüggemann

Quasi-Particles in a Three-Dimensional Three-Component Reaction-Diffusion System

We investigate a reaction-diffusion system which consists of a set of three partial differential equations. Due to the reaction kinetics the system can be referred to as a 1-activator-2-inhibitor system. We show, that such systems axe capable of supporting localized moving structures, so called quasi-particles. For certain parameters it is possible to predict the propagation speed of these solutions as well as their behaviour in scattering processes. In more general cases we have carried out simulations which reveal different scattering results depending on the parameters. We find annihilation, reflection and merging of particles.

C. P. Schenk, A. W. Liehr, M. Bode, H.-G. Purwins

Upwind Relaxation Algorithm for Re-entry Nonequilibrium Flows

The development of the Navier-Stokes solver URANUS (Upwind Relaxation Algorithm for Nonequilibrium Flows of the University of Stuttgart) will be described.Advanced gas-phase and gas-surface interaction modelings allow an accurate prediction of thermochemical relaxations in high-temperature nonequilibrium flows around re-entry vehicles and the associated aerothermal surface loads.A fully coupled, fully implicit numerical algorithm, which solves the Newton-linearized equation system iteratively with arbitrary accuracy allows for large CFL numbers. Therefore, Newton-like convergence rates and a sufficient convergence grade are obtained for accurate computation of sensitive flow quantities like skin friction coefficient and Stanton number. The favorable performance of the algorithm will be demonstrated for solutions of the 2D and 3D sequential nonequilibrium codes on NEC SX-4.Due to the large memory requirement and the large computational work of the 3D algorithm, a 3D Parallel-Multiblock version is developed which allows to compute nonequilibrium flows around re-entry vehicles with about one million mesh points on CRAY T3E to be computed.

H.-H. Frühauf, M. Fertig, F. Olawsky, T. Bönisch

3D Simulation of instationary turbulent flow and combustion in internal combustion engines

This paper describes the advances of internal combustion engine simulations achieved in the parice3d project. The aim of the project was to accelerate ICE simulation technology by using massively parallel computers. The paper will only mention the advances in pre- and post-processing and focus on the computing part of the simulation.

A. Werner

Numerical prediction of load changes in a coal-fired utility boiler

The present work deals with the numerical transient simulation of load changes in a pulverised coal-fired utility boiler. The boiler is a tangential firing system with an electrical output of 750 MW. The code used for the calculations is the in-house developed 3D combustion code AIOLOS. The implemented solution procedure is based on either the well-known SIMPLEC method (Semi Implicit Method for Pressure Linked Equations Consistent) or the PISO approach (Pressure Implicit with Splitting of Operators). The stability of the procedure is independent of the time step, since an implicit temporal discretisation scheme (Euler Implicit) is used. A validation exercise is presented by comparing the numerical results with temperature measurements available from LSE (Lehrstuhl für Systemtheorie in der Elektrotechnik), Saarland University, Saarbrücken.

Alexander Bundschuh, Benedetto Risio, Uwe Schnell, Klaus R. G. Hein

Structural Mechanics and Electrical Engineering

In this chapter, a selection of engineering applications from structural mechanics and electrical engineering are presented which currently get resources on the HPC platforms at HLRS. While in the past high-end workstations were in most cases sufficient to perform the necessary computations in an appropriate time, today’s problems are becoming more and more complex, taking several weeks or even months of CPU time on a single processor. Both engineering areas are not traditional fields for supercomputer usage. Even so, they have evolved as very interesting fields for the use of high performance computers over the last couple of years, and some research groups have made significant effort to improve their codes. Algorithm development as well as programming methods have been targeted to parallel computing, and today some encouraging results from that work can be presented here.

Wolfgang E. Nagel, Erwin Stein

Design and Application of Object Oriented Parallel Data Structures in Particle and Continuous Systems

The complexity of parallel computing hardware calls for software that eases the development of numerical models for researchers that do not have a deep knowledge of the platforms’ communication strategies and that do not have the time to write efficient code themselves. We show in three examples how two “container” data structures, one for particles, the other an array for arbitrary data types, can address the needs of many potential users of parallel machines that have so far been deterred by the complexity of parallelizing code. These examples comprise (i) parallel solutions of partial differential equations, (ii) a molecular dynamics simulation of more than 109 particles and (iii) a simulation of particles sedimenting in a viscous fluid.

Kai Höfler, Matthias Müller, Stefan Schwarzer

Computation of Electromagnetic Fields by the Method of Moments on the CRAY T3E: Iterative Solution Techniques and Large Scale Applications

For certain classes of electromagnetic radiation and scattering problems, fast and efficient techniques have been developed, which enable the solution of these problems on relatively small desktop computers or workstations. However, for validation purposes it is indispensable to have accurate reference data available, which we gain from an application of the method of moments (MoM) as implemented in the computer code FEKO on the CRAY T3E. Iterative solution techniques are presented as a means of reducing run-time and improving parallel scalability, and examples for large scale applications are given.

Ulrich Jakobus

Numerical Treatment of Time Varying Magnetic Fields in Power Transformers by Using the Boundary Element Method (BEM)

It is possible to calculate the leakage inductances of power transformers from their blueprints by using the boundary element method. Empirical values are not needed. The resulting dense systems of linear equations with up to 100000 unknown variables can be solved by using block matrices. Since it is possible to describe the greatest part of all windings of power transformers, the calculated leakage inductances differ only less than two and a half percent from measured values by the manufacturer.

Christian Bollmann

Direct and Inverse Electromagnetic Scattering

The direct and inverse time-harmonic electromagnetic scattering from inhomogeneous media is considered. The physical problem of electromagnetic scattering from known objects is mathematically described by volume integral equations. Being able to master the direct problem is an absolute prerequisite to solving the corresponding inverse problem, which is naturally closely connected. When solving inverse scattering problems one tries to retrieve information about the unknown scatterer from the knowledge of incident probing waves and measured scattering data. We especially investigate methods to reconstruct the geometry and the material properties of inhomogeneous media from scattering data. The objects considered in this context axe either isotropic or anisotropic lossy dielectrics. The objects are assumed to be nonmagnetic. The inverse scattering problem can be formulated as a nonlinear optimization problem which is solved by means of iterative optimization schemes. Numerical examples demonstrate the efficiency of the proposed methods.

Wolfgang Rieger, André Buchau, Günther Lehner, Wolfgang M. Rucker

Computer Science

Strictly speaking, program development for high performance computing is confronted in every case with many areas of computer science. Usually these aspects are not a matter of research but more a question of effective application of methods, tools, and results from computer science.

Christoph Zenger

Fine-Grained Multithreading on the Cray T3E

Fine-grained multithreading can be used to hide long-latency operations encountered in parallel computers during remote memory access. Instead of using special processor hardware, the emulation of fine-grained multithreading on standard processor hardware is investigated. While emulation of coarse-grained multithreading is common in modern operating systems, in the fine-grained case research on emulation has been limited and design of multithreaded processors has been favored. A set of tools was developed to support emulation of multithreading on the Cray T3E parallel computer. Several experiments based on parallel matrix multiplication were performed.

Andreas Grävinghoff, Jörg Keller

ParGrad System: Dynamical Adaptation of the Parallelism Degree of Programs on Cray T3E

Typically, parallel programs are scheduled to use all available processors. However, communication costs can overwhelm the gain through parallelism, causing the program to run slower than with a smaller number of processors. Moreover, allocating many processors to a parallel program may produce poor machine utilization.We show that a parallel program can adapt its parallelism degree automatically to achieve an optimum with respect to criteria such as minimal execution time, achievement of a efficiency threshold or minimal cost-benefit ratio. The adjustment is based on execution time measurements. It only requires programmers to identify the loop, whose parallelism degree is to be adapted, and to set some parameters for the adaptation. The technique has been implemented and tested on the Cray T3E with real applications in the fields of Mathematics, Electrical Engineering and Geophysics. The results concerning four of these applications are presented here. Besides of the extra time needed for the adaptation, the optima were found in all analyzed cases.

Otilia Werner-Kytölä

Comparative Measurements of the Solution of PDE’s on the PARAGON and the SB-PRAM

In the project PARDEAL we want to compare the applicability of two approaches to solve partial differential equations: the SB-PRAM, an experimental shared memory machine and the grid partitioning method on distributed memory architectures. This report refers to speedup measurements on the Paragon of the HLRS.

Andreas Paul

KaHPF: Compiler generated Data Prefetching for HPF

In many parallel applications, network latency causes a dramatic loss in processor utilization. This paper presents KaHPF, an optimizing HPF compiler, that uses software controlled access pipelining (SCAP) to hide communication latency. Various benchmarks of common, but complex communication patterns demonstrate KaHPF’s performance improvement on a Cray T3E. KaHPF’s compiled code is faster than code that uses highly optimized shared-memory system functions by a factor of 1.2 to 3.2. In comparison to PGF’s HPF compiler KaHPF shows an improvement of 3.9 to 30.4.

Matthias M. Müller

A Parallel Object Oriented Framework for Particle Methods

A major goal of the Sonderforschungsbereich 382 is the development of different particle methods for solving physical problems. In this paper, we present two different implementations of parallel Smoothed Particle Hydrodynamics codes and a newly developed method called Finite Mass Methode. We follow up with the results of some test simulations which show the pros and cons of the different approaches.We talk about our experiences with these codes and conclusions for parallelization in general. Our recent work concentrated on object-oriented physical codes and runtime systems. The newly established particle method called Finite Mass Method (FMM), which got developed within the SFB 382 was ported to C++, and parallelized using MPI-1.1. The experiences made with this code led to an object- oriented parallel runtime system, based on message passing and threads. Finally we describe our proposed solutions for higher-level programming abstractions, including design patterns and application-domain specific libraries. Based on this work we currently develop an object-oriented prototype of a QMC simulation.

M. Hipp, S. Hüttemann, M. Konold, M. Klingler, P. Leinen, M. Ritt, W. Rosenstiel, H. Ruder, R. Speith, H. Yserentant

Parallel solution of Partial Differential Equations with Adaptive Multigrid Methods on Unstructured Grids

We present new parallel results for the solution of partial differential equations based on the software platform UG. State-of-the-art numerical methods have been developed and implemented for the efficient and comfortable solution of real-world problems. UG supports distributed unstructured grids, adaptive grid refinement, derefinement/coarsening, robust parallel multigrid methods, various discretizations, dynamic load balancing, mapping and grid partitioning. Here, we give examples for a parallel algebraic multigrid method, for elasto-plastic computations, and for simulations of two-phase flow in porous media.

Peter Bastian, Klaus Birken, Klaus Johannsen, Stefan Lang, Volker Reichenberger, Christian Wieners, Gabriel Wittum, Christian Wrobel

Coupling and Parallelization of Grid-based Numerical Simulation Software

Aims of the GRISSLi project are the parallelization of numerical software for unstructured grids with wide employment of standard tools and the development of the library CI (Coupling Interface) in order to couple different numerical parallel programs based on various kinds of discretization, like structured or unstructured grids. The simulation of steel strip production was chosen as a highly relevant industry application. The chosen simulation uses the block-structured multigrid-solver LiSS and the nonlinear Finite Element package Larstran.

P. Adamidis, A. Beck, U. Becker-Lemgau, Y. Ding, M. Hackenberg, H. Holthoff, M. Laux, A. Müller, M. Münch, B. Steckel, C. Pospiech, R. Tilch


Weitere Informationen