Performance and Scaling of WRF on Three Different Parallel Supercomputers

A WRF weather model configuration with high I/O frequency was tested on three IBM supercomputers for up to 6144 cores. Scalability, overall performance, and I/O throughput was examined for a 12-h forecast with hourly I/O intervals. The three parallel systems tested were the (a) POWER 775 cluster (p7-IH, IBM POWER7, HFI interconnect), (b) the POWER 460 cluster (Pureflex IBM POWER7, Dual QDR Infiniband interconnect) and (c) the iDataPlex cluster (dx360M4 Intel Sandybridge, FDR14 interconnect). MPI traces were obtained on all systems for runs with 128, 256, 512, 1024, 1536, 2048, 3072, 4096 and 6144 cores. I/O quilting was employed, and the number of MPI tasks (including task grids), OpenMP threads and I/O quilting tasks was kept Identical across all three systems. The effects of computation, communication, I/O throughput, load imbalance and simultaneous multithreading (SMT), was examined across all three systems and for all core counts. It was found that all systems performed similarly, despite significant differences in interconnect, processor, I/O subsystems and software/compiler technologies.