High Performance Computing in Science and Engineering ’02

Strain distributions around Ge quantum dots embedded in a Si matrix are computed by means of classical molecular dynamics simulations using the molecular dynamics code IMD. The Tersoff potential is employed in order to model covalent bonds. Two crystal lattice structures are considered, the cubic and hexagonal diamond structure. The distributions of the planar strain are studied for a large number of system sizes and lattice misorientations. In a second part, the scaling of IMD is analyzed for different parallelization schemes and machine architectures.

Recent results are presented for convection driven dynamos in rotating spherical fluids shells which are obtained from computations carried out at the Stuttgart Supercomputing Center. Studies of the dependence on the Prandtl number P indicate that dynamo action disappears with increasing P unless the magnetic Prandtl number P _m is also increased. Relaxation oscillations of convection coupled to magnetic torsional oscillations are found at low Prandtl numbers and various types of reversals of dipolar dynamos have been identified.

The flow of incompressible fluid inside an ellipsoidal shell with imposed rotation and precession is investigated by direct numerical simulation. The flow becomes unstable and eventually turbulent at large enough precession rates. The mechanisms behind these transitions are relevant for geophysical problems.

We are investigating a three-component reaction-diffusion model, which has been established as phenomenological model for pattern formation processes in direct current semiconductor-gas-discharge systems. Concerning two-dimensional systems we are able to reproduce the experimentally observed phenomena of replication of dissipative solitons by many-particle interaction. In three-dimensional systems these phenomena lead to the formation of complex molecules consisting of single dissipative solitons.

We present simulations of magnetized Herbig-Haro flows of low-mass stars. These slightly overdense jets propagate with about 300 km/s through the inner part of a molecular cloud. The magnetic field is injected from the central star and evolves a complicated time-dependent structure driven by pinch and kink modes. The high spatial resolution achieved on the NEC SX-5 provides for the first time insight into the phenomenon of current filamentation and the complex internal structure of pinch and kink modes.

Strongly interacting matter undergoes a phase transition at some temperature T_c to a deconfined phase where it is believed that the dominant degrees of freedom are quasiparticles with quantum numbers of quarks and gluons contrary to the low temperature phase (T < T_c) where the dominant degrees of freedom are hadrons. The existence of this phase transition was shown using lattice Monte-Carlo simulations of Quantum Chromodynamics (QCD), the theory describing strongly interacting particles, some 20 years ago [1].

The results of the same regional climate simulation executed on different computer systems are compared to assess the influence of computer architecture and compiler characteristics on the results of long-term numerical simulations. The comparisons of temperature and precipitation fields show significant differences at any time. The differences of the monthly mean values of the same fields are, however, much smaller. This indicates that the influence of the computer system selected for the simulation can be neglected for the interpretation of climate values but may lead to non negligible uncertainties in the weather forecast for a specific date.

A 3-D compressible spherical-shell model of the thermal convection in the Earth’s mantle has been investigated with respect to its long-range behavior. In this way, it is possible to describe the thermal evolution of the Earth more realistically than by parameterized convection models. The model is heated mainly from within by a temporally declining heat generation rate per volume and, to a minor degree, from below. The volumetrically averaged temperature, T _a , diminishes as a function of time, as in the real Earth. Therefore, the temperature at the core-mantle boundary, T _CMB,av , has not been kept constant but the heat flow, in accord with Stacey (1992). Therefore,T _CMB,av decreases like T _a . This procedure seems to be reasonable since evidently nobody is able to propose a comprehensible thermostatic mechanism for CMB. First of all, a radial distribution of the starting viscosity has been derived using PREM and solid-state physics. The time dependence of the viscosity is essential for the evolution of the Earth since the viscosity rises with declining temperature. For numerical reasons, the temperature-dependent factor of the model viscosity is limited to four orders of magnitude.The focus of this paper is an investigation of the variation of parameters, especially of the non-dimensional numbers as the Rayleigh number, Ra, the Nusselt number, Nu the reciprocal value of the Urey number, Ror, the viscosity level, r _n , etc. For 0.0 ≤ r _n ≤ +0.3, the authors arrived at Earth-like models. This interval contains the starting model. The quantification of the essential features of the model is provided by the eight plots. Numerical procedure: The differential equations are solved using a fast multigrid solver and a second-order Runge-Kutta procedure with a FE method. On 128 processors, runs with 10649730 grid points need about 50 hours. Figure 11 shows the scalling degree of our code.If the temperature dependence of the viscosity, Eq. (4), is replaced by Eq. (10) then, in the interval 0.0 ≤ r _n ≤ +0.3, reticularly conected thin cold sheet-like downwellings are found from the surface down to 1350km depth. However, the movements along the upper surface are not plate-like.

We present the results of large scale computer simulations in which we investigate the structural and dynamic properties of silicate melts with the compositions (Na₂O)2(SiO₂) and (Al₂O₃)2(Si02). In order to treat such systems on a time scale of several nanoseconds and for system sizes of several thousand atoms it is necessary to use parallel supercomputers like the CRAY T3E. We show that the silicates under consideration exhibit additional intermediate range order as compared to silica (SiO₂) where the characteristic intermediate length scales stem from the tetrahedral network structure. For the sodium silicate system it is demonstrated that the latter structural features are intimately connected with a surprising dynamics in which the one-particle motion of the sodium ions appears on a much smaller time scale than the correlations between different sodium ions.

The single impurity Anderson model (SIAM) in the localized moment regime is used as the generic model to analyze strong correlations in metals. As it turns out the auxiliary particle representation is a convenient way for formulating selfconsistent approximations for SIAMs. We discuss the Non-Crossing Approximation, or NCA, one of the simplest approximations possible and point out why massive parallel machines are necessary when going beyond NCA. The CTMA, an approximation capable of describing the correct groundstate and the SUNCA for impurity systems with finite Coulomb repulsion are discussed and numerical results are presented.

We present a novel approach to solve the Bethe-Salpeter equation for the polarisation function. Rather than from the usual eigenvalue representation, the macroscopic polarisability is obtained from the solution of an initial-value problem. This allows for the first time to calculate excitonic and local-field effects in optical spectra of large and complex systems such as surfaces. As an example we investigate the optical anisotropy of the hydrogen-passivated Si(ll0) surface. It is shown that the electron-hole attraction is largely responsible for the peculiar line shape of the surface optical spectrum.

On the basis of Quantum Monte Carlo (QMC) simulations, we study the influence of specific phonon, i e the oxygen half-breathing (π,0) mode in the three-band Hubbard model as a relevant model for high-temperature superconductors. In the hole-doped case, both the diagonal and the off-diagonal couplings dramatically change the total energy and local spin correlations. However, in the electron-doped case, the diagonal electron-phonon coupling changes the total energy much more than the off-diagonal coupling, and an essential difference from hole doping is that the electron-phonon coupling has negegible effects on local spin-spin correlations.

To understand the interplay of spin, orbital and lattice degrees of freedom in colossal magneto-resistance manganites we numerically diagonalize an SU(2) symmetric spin-orbital model coupled to dynamic Jahn-Teller and Holstein-type phonons. For a four site cluster we demonstrate how the coupling to the lattice changes the order of spins, orbitals and charges, and the correlations between them.

Phase transitions, structures and quantum effects in Nanostructures are studied by path integral Monte Carlo-, Molecular Dynamics- and Car-Parrinello methods. We present results of our computations on pore condensates, reentrance phenomena of systems in external fields, atomic wires and quantum-spin-fluids.

Based on state of the art Quantum Monte Carlo simulations, we investigate the metallic states of the one-dimensional t-J model and of a depleted Kondo lattice model in two dimensions. In the one-dimensional case, it is known that correlation effects invalidate the Fermi liquid picture and that the elementary excitations are spinons and holons carrying separately the charge and spin of the electron. In this dimension we will present new results on the single particle spectral function and discuss the implications of spin-charge separation on this quantity. The Kondo lattice model describes heavy fermion materials which generically have Fermi liquid ground states but with effective masses up to three orders of magnitude larger that the bare electron mass. On the basis of numerical simulations, we will show how this heavy fermion state comes about and set the emphasis on the coherence temperature.

We discuss the localization of the surface exciton at the Si(111)-(2×1) surface due to self-trapping, which leads to a characteristic temperature-dependent linewidth of the optical response and to a significant Stokes shift of the luminescence. Self-trapping results in this case from a structural relaxation in the excited state, caused by the interplay between electronic and geometric degrees of freedom. The most significant contribution to this effect comes from one single geometric deformation mode which is driven by the internal electronic charge transfer in the self-trapped exciton. To study these mechanisms we employ computational ab-initio techniques designed for excited states (density-functional theory and many-body perturbation theory), combined with tight-binding representations that allows us to simulate enlarged supercells containing several thousand atoms.

Global geometry optimization of pure neutral water clusters using the highly accurate but computationally expensive many-body TTM2-F potential have been performed within reasonable real time, using a parallelized and specialized implementation of evolutionary algorithms. In comparison to previous studies using highly approximate and cheap water potentials that work well for small clusters, qualitatively different structures result for larger clusters, exposing subtle failures of these cheap models. This unexpected result could not have been obtained within reasonable real time on a serial machine.

We report on the progress of our implementation of the configuration-selecting multi-reference configuration interaction method on massively parallel architectures with distributed memory, which now permits the treatment of Hilbert spaces of dimension 0(1012) about 20,000,000 of which can be selected in the variational subspace. We provide scaling data for the CPU time of the code for the IBM/SP3 and the CRAY-T3E. We present benchmark results for two selected applications: the isomers of dinitrosoethylene and the electronic structure of two members of the transition metal dihalide family: VF2 and VCI2.

Multi reference correlation calculations (MR-CI and MR-ACPF) have been performed for small V_nO_m clusters. VO₂ has two doublet states which are so close that it is difficult to predict the symmetry of the ground state. For V₂O₄+ and V₂O₄ we find minimum energy structures of C _2h . symmetry (trans bending of the vanadyl units) in contrast to what has been reported in the literature. The magnetic coupling of the electrons is such that low spin states are favoured (singlet for V₂O₄, doublet for V₂O₄-).

Density Functional Calculations have been performed on the mechanisms of the Iron(III) catalysed Michael Reaction.

Crossflow-induced laminar-turbulent transition and the initial stages of turbulence are investigated by means of spatial direct numerical simulations for the accelerating 3-D flat-plate boundary layer of the “Querströmungsprinzipexperiment” of the DLR-Göttingen [1]. The complete 3-D incompressible Navier-Stokes equations are solved marching in time on the basis of a 4th-order Runge-Kutta scheme. Fourier spectral expansions and 6th-order compact finite differences are used for discretization in space. In accordance with the experiment, both steady and traveling crossflow modes are excited. Attention is focused on the non-linear interaction between vortical structures generated by the primary instabilities. The growth of high-frequency secondary instabilities is also observed. Classical turbulent quantities are evaluated for the region downstream of the laminar breakdown.

Studies on Large-Eddy-Simulations (LES) are presented of turbulent wall bounded flow at Re_r = 590 and Ree = 670 either with and without adverse pressure gradient (AGP). The Simulations of the Kalter and Fernholz Case [6] show the shortcomings of eddy viscosity subgrid scale models to predict pressure induced separation and reattachement.

A direct numerical simulation of a separating and reattaching turbulent boundary layer on a flat plate was conducted on a high performance computer. The separation has been introduced by a stream wise pressure gradient that corresponds to the one of an actually performed experiment (Kalter and Fernholz, [10]). The use of a locally refined grid near the wall is one of the key points that allowed the simulation on the presently available hardware. The comparison between experiment and simulation is fully satisfying considering that the Reynolds number of the experiment could not be achieved in the simulation.

A 3D numerical program for the transient simulation of the dynamic behavior of incompressible two-phase flows has been extended to the computation of heat transfer. In the program the VOF-method with interface reconstruction has been used for the calculation of the disperse phase. The governing equations and the implemented numerical model are described. Numerical results for a transient heat conduction problem of a rigid sphere show good agreement with analytical solutions. The predicted averaged Nusselt numbers for this problem from numerical simulations match well with experimental data from the literature. On the basis of two examples the difference between intermediate and high Reynolds number flow and heat transfer is pointed out. Finally, the influence of different initial droplet velocities on the time dependent temperature evolution is shown. The simulation has been performed on the Cray T3E/512-900 at the HLRS with up to 128 processors.

An implicit LU-SGS (Lower-Upper Symmetrie Gauss-Seidel) algorithm is used for the simulation of reactive and non-reactive three dimensional high speed flows (supersonic combustion). The numerical method is based on an all Mach number preconditioning to enable convergence of the compressible flow solver in the low Mach number limit. The code is fully vectorized and may be used on massively parallel computers using MPI. Parallelization is performed by domain decomposition which causes losses in efficiency of the implicit numerical solver. Therefore overlapping domains are introduced to reduce both losses in convergence rate as well as in robustness. A comparison with non-intersecting grids demonstrates the effectiveness of this method. To investigate both approaches a 3-D turbulent Mach 3.85 supersonic ramp flow with shock wave boundary-layer interaction is chosen. The simulations use up to 256 nodes on a Cray T3E and up to 16384 blocks for the discretization of the computational domain. In addition results for a 3-D supersonic ramp combustor are presented. The finite-rate chemistry reaction mechanism involves 20 reactions and 9 different species.

We present the results of first realistic simulations using our state-of- the-art MHD code on unstructured tetrahedral meshes in 3d. The code incorporates local grid adaption with dynamic load balancing and relies on a recently proposed approximate Riemann solver. We demonstrate that it is absolutely crucial to control the divergence of the magnetic field and that our new hyperbolic divergence cleaning approach works well also in 3d.

The flow in a low-pressure turbine rotor blade with incoming periodic wakes is computed by means of LES based on a dynamic SGS model. The computations are performed by using 32 processors of a parallel super-computer. The optimisation and tuning of the algorithm for the HITACHI-SR8000 environment allowed reducing the CPU time of a factor 2. The simulations identify several relevant phenomena thanks to the good resolution of the problem in both space in time. The computed results are compared with experiments in terms of phase- averaged and mean quantities.

MUFTE-UG, a parallel numerical simulator for multiphase flow and transport processes in porous media, is introduced. The basic PDEs for two-phase flow together with a discretization and solution scheme are presented. Aspects of the implementation of the advanced numerical techniques of UG on parallel hardware are shown and the simulator’s parallel performance is demonstrated using a 2D example.

A two-phase flow of water in a pipe is investigated experimentally and numerically. A three dimensional simulation is performed using an enhanced two-fluid model. To take phenomena like phase change and a possible change of the flow regime into account, adequate inter-phase exchange terms were implemented. The results show that three dimensional phenomena in pipe elements can have a significant influence on the integral parameters of the flow situation.

In next generation nuclear High-Temperature Reactors an annular nuclear core consisting of a central column of graphite spheres and a surrounding ring of fuel pebbles is employed. Due to the complex feeding and shutdown mechanisms three-dimensional effects of heat production, gas flow and heat transport may become important for safety analysis. To simulate flow and heat transport in the core and the surrounding graphite reflector a new code system based on CFX-4 has been developed and run on the NEC-SX4 and NEC-SX5 supercomputers. The simulations are performed with the Heterogeneous Model of porous media. The program has been verified by comparison with two-dimensional simulations of the HTR-MODUL using the well established thermal analysis code THERMIX. A sensitivity study of several models for pressure drop and heat transfer on a simplified model of an HTR-MODUL is performed. Additionally the influence of the variation of the volume porosity near walls on flow and heat transport is analysed. In order to demonstrate the simulation of three-dimensional effects the influence of a package of fuel pebbles located asymmetrically in the central column is investigated. A significant influence on the temperature distribution and the maximum temperature core is found.

The development of the shuttle-like technology demonstrator PHOENIX is intended to proof the feasibility of a future European reusable space transportation system. The present work focuses on aerodynamic solutions for the clean- configuration of PHOENIX by three-dimensional Euler computations. Details on the structured grid generation process using the interactive MegaCads software are discussed and a converged solution for supersonic inviscid flow is presented. The agreement of the obtained aerodynamic coefficients with wind tunnel and design data is favorable. Preliminary work has started on viscous hypersonic flow computations for different flight conditions.

In this paper, two different comprehensive rotor analyses are described and applied to a high speed forward flight test case using the Stuttgart NEC-SX5 HPC facility. Each analysis consists of an unsteady Euler/RANS CFD module coupled with a CSD module based on Timoshenko and Euler-Bernoulli beam theory, respectively. Deformable Chimera grids are applied for spatial discretization and resolution requirements call for the use of state-of-the-art vector super computers. Qualitatively, stand-alone aerodynamic results using a-priori trim data agree fairly well with wind tunnel measurements. Substantial improvement can be achieved by including the time-accurate structural response of the blades. Recent results indicate that the incorporation of an automated trim procedure is beneficial with respect to the agreement of computed and experimental global rotor parameters.

The project NHLRes is concerned with the simulation of aircraft aerodynamics and thus belongs to the research field of computational fluid dynamics (CFD) for aerospace applications. NHLRes comprises the numerical simulation of the viscous compressible flow around transport aircraft high lift configurations. The investigations are based on the solution of the Reynolds-averaged Navier-Stokes equations using a finite volume parallel solution algorithm with an unstructured data concept. The project consists of two parts representing a typical analysis as well as an optimization task for selected three-dimensional high lift flow problems.

A parallel code for the direct numerical simulation (DNS) of reactive flows with detailed models for chemical reactions and molecular transport has been used as a benchmark on two large commodity hardware clusters with current state-of-the-art processor and network technologies. Up to 180 Alpha EV68 833 MHz have been used on the first system and up to 400 AthlonMP 1.4 GHz CPUs have been used on the other one. Both clusters comprise dual-nodes and Myrinet-2000 interconnect. The influence of the dual-nodes on the parallel efficiency is discussed and the performance of these clusters is compared with Cray T3E systems. The other part of the paper deals with the application of DNS to study premixed flames emerging after induced ignition of turbulent mixtures. The discussion focusses on the temporal evolution of the heat-release rate. In addition to the analysis of flames expanding in homogeneous turbulent mixtures, first DNS results of a flame kernel evolving in a turbulent mixture with spatial inhomogeneities of the equivalence ratio are presented.

The increasing computational power and new numerical methods give the possibility for more detailed investigations of complex physical phenomena which are typical in science and engineering. In the first part of this paper, we show how CFD simulations of the flows through packed beds using the lattice Boltzmann approach can be used to get local and detailed information about the physical transport processes. The second part deals with the performance of the lattice Boltzmann method on different hardware platforms. It shows that this modern numerical method also needs adequate computers, i.e. high performance computers, to give best performance.

A Lagrangian particle tracking model is applied to the simulation of the discrete phase in coal-fired furnaces. The interaction of particles with the gas phase leads to additional source terms in the Eulerian transport equations of mass, momentum and enthalpy. The radiative heat exchange of the particles is directly coupled with the discrete ordinates method using an extra source term in the discre- tised radiative transport equation. The routines for the particle model are vectorised using loops over the number of all particles. They are implemented in the 3D combustion code AIOLOS, optimised for vector and parallel computers to achieve high numerical efficiency. The code is applied to the numerical simulation of a utility boiler, showing a high performance and vectorisation rate, combined with good agreement with measurements.

Existing foundations of buildings can be piled subsequently to improve their bearing behaviour. The effort of subsequent piling depends on the initial state of soil as well as the load history. Model tests were carried out to study the influencing quantities, especially the loading history due to piling. The models tests were numerically simulated based on the FEM and on hypoplastic constitutive models for the soil.

In this work, we discuss different granularities for parallel wavelet packet video coding using block-based motion compensation and the performance of the corresponding MPI implementations on the HLRS Cray T3-E. Two inter-frame based parallelization methods (group-of-picture parallelization and frame-by-frame parallelization) are compared against intra-frame parallelization. We highlight the advantages and drawbacks of all three approaches.

This article concentrates on recent work on benchmarking collective operations with SKaMPI. The goal of the SKaMPI project is the creation of a database containing performance measurements of parallel computers in terms of MPI operations. Its data support software developers in creating portable and fast programs. Existing algorithms for measuring the timing of collective operations are discussed and a new algorithm is presented, taking into account the differences of local clocks. Results of measurements on the HLRS Cray T3E/900 are presented and compared with other machines.

The temperature dependence of the internal dynamics of an isolated protein, bovine pancreatic trypsin inhibitor, is examined using normal mode analysis (NMA) and molecular dynamics (MD) simulation. It is found that this model exhibits marked anharmonic dynamics, at temperatures much lower than previously detected in proteins, as evidenced by departure from the harmonic model mean- square displacement. A new method for determining mean-square displacements from elastic incoherent neutron scattering experiments is demonstrated and it is found to be more accurate than the method currently used by experimentalists.