Scalable adaptive mantle convection simulation on petascale supercomputers

Carsten Burstedde, Omar Ghattas, Michael Gurnis, Georg Stadler, Eh Tan, Tiankai Tu, Lucas C. Wilcox, Shijie Zhong

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Mantle convection is the principal control on the thermal and geological evolution of the Earth. Mantle convection modeling involves solution of the mass, momentum, and energy equations for a viscous, creeping, incompressible non-Newtonian fluid at high Rayleigh and Peclet numbers. Our goal is to conduct global mantle convection simulations that can resolve faulted plate boundaries, down to 1 km scales. However, uniform resolution at these scales would result in meshes with a trillion elements, which would elude even sustained petaflops supercomputers. Thus parallel adaptive mesh refinement and coarsening (AMR) is essential. We present RHEA, a new generation mantle convection code designed to scale to hundreds of thousands of cores. RHEA is built on ALPS, a parallel octree-based adaptive mesh finite element library that provides new distributed data structures and parallel algorithms for dynamic coarsening, refinement, rebalancing, and repartitioning of the mesh. ALPS currently supports low order continuous Lagrange elements, and arbitrary order discontinuous Galerkin spectral elements, on octree meshes. A forest-ofoctrees implementation permits nearly arbitrary geometries to be accommodated. Using TACC's 579 teraflops Ranger supercomputer, we demonstrate excellent weak and strong scalability of parallel AMR on up to 62,464 cores for problems with up to 12.4 billion elements. With RHEA's adaptive capabilities, we have been able to reduce the number of elements by over three orders of magnitude, thus enabling us to simulate large-scale mantle convection with finest local resolution of 1.5 km.

Original languageEnglish (US)
Title of host publication2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008
DOIs
StatePublished - 2008
Event2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008 - Austin, TX, United States
Duration: Nov 15 2008Nov 21 2008

Other

Other2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008
CountryUnited States
CityAustin, TX
Period11/15/0811/21/08

Fingerprint

Supercomputers
Coarsening
Peclet number
Parallel algorithms
Data structures
Scalability
Momentum
Earth (planet)
Convection
Fluids
Geometry

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software

Cite this

Burstedde, C., Ghattas, O., Gurnis, M., Stadler, G., Tan, E., Tu, T., ... Zhong, S. (2008). Scalable adaptive mantle convection simulation on petascale supercomputers. In 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008 [5214248] https://doi.org/10.1109/SC.2008.5214248

Scalable adaptive mantle convection simulation on petascale supercomputers. / Burstedde, Carsten; Ghattas, Omar; Gurnis, Michael; Stadler, Georg; Tan, Eh; Tu, Tiankai; Wilcox, Lucas C.; Zhong, Shijie.

2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008. 2008. 5214248.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Burstedde, C, Ghattas, O, Gurnis, M, Stadler, G, Tan, E, Tu, T, Wilcox, LC & Zhong, S 2008, Scalable adaptive mantle convection simulation on petascale supercomputers. in 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008., 5214248, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008, Austin, TX, United States, 11/15/08. https://doi.org/10.1109/SC.2008.5214248
Burstedde C, Ghattas O, Gurnis M, Stadler G, Tan E, Tu T et al. Scalable adaptive mantle convection simulation on petascale supercomputers. In 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008. 2008. 5214248 https://doi.org/10.1109/SC.2008.5214248
Burstedde, Carsten ; Ghattas, Omar ; Gurnis, Michael ; Stadler, Georg ; Tan, Eh ; Tu, Tiankai ; Wilcox, Lucas C. ; Zhong, Shijie. / Scalable adaptive mantle convection simulation on petascale supercomputers. 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008. 2008.
@inproceedings{b1e81b16c0454d089f2b100d1233f7ff,
title = "Scalable adaptive mantle convection simulation on petascale supercomputers",
abstract = "Mantle convection is the principal control on the thermal and geological evolution of the Earth. Mantle convection modeling involves solution of the mass, momentum, and energy equations for a viscous, creeping, incompressible non-Newtonian fluid at high Rayleigh and Peclet numbers. Our goal is to conduct global mantle convection simulations that can resolve faulted plate boundaries, down to 1 km scales. However, uniform resolution at these scales would result in meshes with a trillion elements, which would elude even sustained petaflops supercomputers. Thus parallel adaptive mesh refinement and coarsening (AMR) is essential. We present RHEA, a new generation mantle convection code designed to scale to hundreds of thousands of cores. RHEA is built on ALPS, a parallel octree-based adaptive mesh finite element library that provides new distributed data structures and parallel algorithms for dynamic coarsening, refinement, rebalancing, and repartitioning of the mesh. ALPS currently supports low order continuous Lagrange elements, and arbitrary order discontinuous Galerkin spectral elements, on octree meshes. A forest-ofoctrees implementation permits nearly arbitrary geometries to be accommodated. Using TACC's 579 teraflops Ranger supercomputer, we demonstrate excellent weak and strong scalability of parallel AMR on up to 62,464 cores for problems with up to 12.4 billion elements. With RHEA's adaptive capabilities, we have been able to reduce the number of elements by over three orders of magnitude, thus enabling us to simulate large-scale mantle convection with finest local resolution of 1.5 km.",
author = "Carsten Burstedde and Omar Ghattas and Michael Gurnis and Georg Stadler and Eh Tan and Tiankai Tu and Wilcox, {Lucas C.} and Shijie Zhong",
year = "2008",
doi = "10.1109/SC.2008.5214248",
language = "English (US)",
isbn = "9781424428359",
booktitle = "2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008",

}

TY - GEN

T1 - Scalable adaptive mantle convection simulation on petascale supercomputers

AU - Burstedde, Carsten

AU - Ghattas, Omar

AU - Gurnis, Michael

AU - Stadler, Georg

AU - Tan, Eh

AU - Tu, Tiankai

AU - Wilcox, Lucas C.

AU - Zhong, Shijie

PY - 2008

Y1 - 2008

N2 - Mantle convection is the principal control on the thermal and geological evolution of the Earth. Mantle convection modeling involves solution of the mass, momentum, and energy equations for a viscous, creeping, incompressible non-Newtonian fluid at high Rayleigh and Peclet numbers. Our goal is to conduct global mantle convection simulations that can resolve faulted plate boundaries, down to 1 km scales. However, uniform resolution at these scales would result in meshes with a trillion elements, which would elude even sustained petaflops supercomputers. Thus parallel adaptive mesh refinement and coarsening (AMR) is essential. We present RHEA, a new generation mantle convection code designed to scale to hundreds of thousands of cores. RHEA is built on ALPS, a parallel octree-based adaptive mesh finite element library that provides new distributed data structures and parallel algorithms for dynamic coarsening, refinement, rebalancing, and repartitioning of the mesh. ALPS currently supports low order continuous Lagrange elements, and arbitrary order discontinuous Galerkin spectral elements, on octree meshes. A forest-ofoctrees implementation permits nearly arbitrary geometries to be accommodated. Using TACC's 579 teraflops Ranger supercomputer, we demonstrate excellent weak and strong scalability of parallel AMR on up to 62,464 cores for problems with up to 12.4 billion elements. With RHEA's adaptive capabilities, we have been able to reduce the number of elements by over three orders of magnitude, thus enabling us to simulate large-scale mantle convection with finest local resolution of 1.5 km.

AB - Mantle convection is the principal control on the thermal and geological evolution of the Earth. Mantle convection modeling involves solution of the mass, momentum, and energy equations for a viscous, creeping, incompressible non-Newtonian fluid at high Rayleigh and Peclet numbers. Our goal is to conduct global mantle convection simulations that can resolve faulted plate boundaries, down to 1 km scales. However, uniform resolution at these scales would result in meshes with a trillion elements, which would elude even sustained petaflops supercomputers. Thus parallel adaptive mesh refinement and coarsening (AMR) is essential. We present RHEA, a new generation mantle convection code designed to scale to hundreds of thousands of cores. RHEA is built on ALPS, a parallel octree-based adaptive mesh finite element library that provides new distributed data structures and parallel algorithms for dynamic coarsening, refinement, rebalancing, and repartitioning of the mesh. ALPS currently supports low order continuous Lagrange elements, and arbitrary order discontinuous Galerkin spectral elements, on octree meshes. A forest-ofoctrees implementation permits nearly arbitrary geometries to be accommodated. Using TACC's 579 teraflops Ranger supercomputer, we demonstrate excellent weak and strong scalability of parallel AMR on up to 62,464 cores for problems with up to 12.4 billion elements. With RHEA's adaptive capabilities, we have been able to reduce the number of elements by over three orders of magnitude, thus enabling us to simulate large-scale mantle convection with finest local resolution of 1.5 km.

UR - http://www.scopus.com/inward/record.url?scp=70350747438&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=70350747438&partnerID=8YFLogxK

U2 - 10.1109/SC.2008.5214248

DO - 10.1109/SC.2008.5214248

M3 - Conference contribution

SN - 9781424428359

BT - 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008

ER -