Massively Parallel Computational Fluid Dynamics With Large Eddy Simulation in Complex Geometries

To perform complex geometry large eddy simulations in an industrially relevant timeframe, one must reduce the total time to half a day (overnight simulation). Total time includes the time of developing the mesh from the computer-aided design (CAD) model and simulation time. For reducing CAD-to-mesh time, automatic meshing algorithms can generate valid but often non-efficient meshes with often up to an order of magnitude more grid points than a custom-based mesh. These algorithms are acceptable only if paired with high-performance computing (HPC) platforms comprising thousands to millions of cores to significantly reduce computational time. Efficient use of these tools calls for codes that can scale to high processor counts and that can efficiently transport resolved scales over the long distances and times made feasible by HPC. The rapid convergence of high-order discretizations makes them particularly attractive in this context. In this paper we test the combination of automatic hexahedral meshing with a spectral element code for incompressible and low-Mach-number flows, called Nek5000, that has scaled to P >262,000 cores and sustains >70% parallel efficiency with only ≈7000 points/core. For our tests, a simple pipe geometry is used as a basis for comparing with previous fully resolved direct numerical simulations.