# U.C. Berkeley CS267

## Applications of Parallel Computers

*Spring 2012*

## Syllabus

##
High-Level Description

This course teaches both graduate and advanced undergraduate students from diverse departments how use parallel computers both efficiently
and productively, i.e. how to write programs that run fast while minimizing
programming effort. The latter is increasingly important since essentially all computers are (becoming) parallel, from supercomputers to laptops.
So beyond teaching the basics about parallel computer architectures and programming languages, we emphasize commonly used patterns that appear in essentially all programs that need to run fast. These patterns include both common computations (eg linear algebra, graph algorithms, structured grids,..)
and ways to easily compose these into larger programs.
We show how to recognize these patterns in a variety of practical
problems, efficient (sometimes optimal) algorithms for implementing them,
how to find existing efficient implementations of these patterns when available,
and how to compose these patterns into larger applications.
We do this in the context of the most important parallel programming models today:
shared memory (eg PThreads and OpenMP on your multicore laptop),
distributed memory (eg MPI and UPC on a supercomputer), GPUs (eg CUDA and OpenCL, which could be both in your laptop and supercomputer), and cloud computing (eg MapReduce and Hadoop). We also present a variety of useful
tools for debugging correctness and performance of parallel programs.
Finally, we have a variety of guest lectures by a variety of experts,
including parallel climate modeling, astrophysics, and other topics.

##
Detailed Schedule of Lectures
(lecturers shown in parentheses)

Jan 17 (Tuesday): Introduction: Why Parallel Computing?
(Jim Demmel)
Jan 19 (Thursday): Single processor machines: Memory hierarchies and processor features
(Jim Demmel)
Jan 24 (Tuesday): Introduction to parallel machines and programming models
(Jim Demmel)
Jan 26 (Thursday): Sources of parallelism and locality in simulation: Part 1
(Jim Demmel)
Jan 31 (Tuesday): Sources of parallelism and locality in simulation: Part 2
(Jim Demmel)
Feb 2 (Thursday): Shared memory machines and programming: OpenMP and Threads; Tricks with Trees
(Jim Demmel)
Feb 7 (Tuesday): Distributed memory machines and programming in MPI (Jim Demmel)
Feb 9 (Thursday): Partitioned Global Address Space Programming with Unified Parallel C (UPC)(Kathy Yelick)
Feb 14 (Thursday): GPUs, and programming with with CUDA and OpenCL (Bryan Catanzaro)
Feb 16 (Tuesday): Performance and Debugging Tools (David Skinner, Richard Gerber)
Feb 21 (Tuesday): Dense Linear Algebra: Part 1 (Jim Demmel)
Feb 23 (Thursday): Dense Linear Algebra: Part 2 (Jim Demmel)
Feb 28 (Tuesday): Graph Partitioning: Part 1 (Jim Demmel)
Mar 1 (Thursday): Graph Partitioning: Part 2, and Sparse-Matrix-Vector-Multiply (Jim Demmel)
Mar 6 (Tuesday): Sparse-Matrix-Vector-Multiply and Autotuning (Jim Demmel)
Mar 8 (Thursday): Particle (N-Body) methods (Jim Demmel); Efficient Data Race Detection for Distributed Memory Parallel Programs (Chang-Seo Park)
Mar 13 (Tuesday): Structured grids and multigrid (Jim Demmel)
Mar 15 (Tuesday): Cloud computing with MapReduce and Hadoop (Matei Zaharia)
Mar 20 (Tuesday): Patterns of Parallel Programming (Kurt Keutzer)
Mar 22 (Thursday): Structured grids(Jim Demmel)
Mar 26-30: Spring Break
Apr 3 (Tuesday): Frameworks for Structured Software Development (John Shalf)
Apr 5 (Thursday): Exascale Computing (Katherine Yelick)
Apr 10 (Tuesday): Parallel Graph Algorithms (Aydin Buluc)
Apr 12 (Thursday): Parallel Climate Modeling (Michael Wehner)
Apr 17 (Tuesday): Parallel Fast Fourier Transform (FFT) (Jim Demmel)
Apr 19 (Thursday): Dynamic Load Balancing (Jim Demmel)
Apr 24 (Tuesday): Big Bang, Big Data, Big Iron: High Performance Computing and the Cosmic Microwave Background (Julian Borrill)
Apr 26 (Thursday): Accelerated Materials Design through High Throughput First-Principles Calculations and Data Mining (Kristin Persson)
May 3 (Thursday): Student Poster Session