Prometheus has solved problems of over 76 million degrees of freedom on 1024 PowerPC processors at LLNL.
We have recently add a new multigrid algorithm to our library: Atlas, an implementation of "smoothed aggregation" multigrid by Vanek, Mandel, and Brezina. Atlas shows promise, being as effective as Prometheus (an more effective for shell problems) while not requiring the rather complex construction of explicit coarse grids.
Prometheus
is publicly available from the Prometheus library page
Prometheus system architecture
Sample FEAP problem solved with Prometheus
Sample fine grid and two coarse grids
We use a parameterized mesh of a "hard" sphere included in a "soft" material for scalability studies. We test problems from 79,679 degrees of freedom to 39,160,959 degrees of freedom, on up to 960 processors of an IBM PowerPC cluster at LLNL. Each version of the problem is run with the number processors required to keep about 40,000 equations on each processor. The interior sphere is composed of 17 layers of alternating "hard" and "soft" materials. The hard material is steel-like with a Poisson ratio of 0.3, and the soft material is rubber-like with a Poisson ratio of 0.49, and elastic modulus 10-4 that of the hard material. The solver is Preconditioned Conjugate gradient (PCG), preconditioned with one "full" multigrid F-cycle, with Prometheus restriction operators. The pre and post multigrid smoother is one applications of PCG, preconditioned with a block Jacobi solver.
Times (sec) of solver on IBM PowerPC cluster with about 40,000 equations
per processor
Times (sec) of full nonlinear solve 10 Newton iterations and about
3500 total multigrid iterations) solver on IBM PowerPC cluster with about
40,000 equations per processor
Scaled speedup for components of one linear solve
Efficiencies for first linear solve of Prometheus/FEAP/PETSc/Epimetheus on IBM PowerPC cluster with about 40,000 equations per processor and upto 960 processors
Mark
AdamsIf you have comments or suggestions, email me at ma2325@columbia.edu
