Ben Schwarz
CS267 HW 0
I'm a PL student working in software security, crypto, and watermarking.
I'd be interested in looking at topics related to parallel
computing for crytography: e.g., random number generation on
parallel processors, parallel cryptanalysis... I have used MPI
and several parallel programming languages in the past.
Earthquake Simulation on a Massively Parallel Computer
Accurately modeling the seismic waves that arise from earthquakes is useful in many realms of science, and is
also of some use in structural engineering. Unfortunately the simulation environment is three dimensions--namely
a mesh of the earth--so decreasing the mesh size is extremely expensive as it results in O(N^3) more work. The application
lends itself well to being carried out on parallel computers. Recently, work has been done to model earthquakes using a
reasonable time interval and mesh size.
One of the main challenges is to model surface and body wave propagation with reasonable time slices: reasonable being
on the order of a few seconds, or less. A recent attempt using a Beowulf cluster consisting of 150 processors was
only able to achieve time slices of 18 seconds. Unfortunately, a lot of important information is lost and the
calculation less precise with such a large time slice. A recent attempt has been made to implement wave propagation on
the Earth Simulator in Japan [2].
The authors of [2] describe their implementation as modeling the full complexity of the 3-dimensional Earth--including
3-dimensional wave speed and density structure, ellipticity, topography and bathymetry. They also model the earth's crust
and the effect that oceans have on surface waves. The simulation uses the spectral-element method which involves
interpolating high-degree Lagrange polynomials on the 3-dimensional mesh [1]. The computation lends itself well to
parallelization, and most of the time is spent computing rather than communicating.
The implementation--on Japan's Earth Simulator--used pure MPI. The Earth Simulator is a distributed-memory massively
parallel architecture with 640 nodes. Each node contains 8 processors and has 16 gigabytes of shared memory. In all,
5120 processors and 10 terabytes of distributed shared memory are available. On the top500, the Earth Simulator is
currently ranked #1. The experiments in [2] were allocated
roughly 38% of the resources on the Earth Simulator. It is not clear what language the implementation of spectral
element method was written in. The investigators were able to achieve a minimum time slice of 5 seconds, which is
considerably better than the previous best of 18 seconds. Although not comparable to modeling the real seismic waves
from earthquakes, it is more practical, reproducible, and reliable.
The reported efficiency of the computation seems average. The authors were able to achieve 5 teraflops using 5.5 billion
grid points, but this was only 30% of the peak performance for the number of resources that were allocated. They claim
that the short vector lengths that are used in the matrix-vector products cannot take advantage of the vector pipes. This
seems like a reasonable area to look for improvement.
The application seems noteworthy because it has repercussions for many areas of science, and the potential to improve
the quality of life for people in earthquake-prone regions of the world. The specific simulation discussed here appears
to be the largest ever run, and consequently has the best results. In the next decade it should be possible to model
the earthquakes using a sub-second time interval, which would be excellent and comparable to physical modeling.
Sources:
[1] E. Faccioli, F. Maggio, R. Paolucci, and A. Quarteroni, "2D and 3D Elastic Wave Propagation by a Pseudo-Spectral Domain
Decomposition Method", J. Seismol., 1:237-251, 1997.
[2] Dimitri Komatitsch, Seiji Tsuboi, Chen Ji and Jeroen Tromp, "A 14.6 billion degrees of freedom, 5 teraflops, 2.5 terabyte
earthquake simulation on the Earth Simulator", Proceedings of the ACM / IEEE Supercomputing SC'2003 conference, 2003.