U.C. Berkeley CS267 Home Page

Applications of Parallel Computers

Spring 2012

T Th 9:30-11:00, 250 and 254 Sutardja Dai Hall


  • Jim Demmel
  • Offices:
    564 Soda Hall ("Virginia", in ParLab), (510)643-5386
    831 Evans Hall, same phone number
  • Office Hours: W 11-12 and F 11-12, in 564 Soda Hall (subject to change)
  • (send email)
  • Teaching Assistants:

  • Nick Knight
  • Office: 593D Soda Hall (Parlab)
  • Office Hours: T Th 11:40-12:30 in 576 Soda Hall ("Euclid" room in ParLab)
  • (send email)
  • Brian Van Straalen
  • Office: Parlab, 5th floor, Soda Hall, phone?
  • Office Hours: T Th 3:00-4:30 in 643 Soda Hall
  • (send email)
  • Administrative Assistants:

  • Tammy Johnson
  • Office: 565 Soda Hall
  • Phone: (510)643-4816
  • (send email)
  • Roxana Infante
  • Office: 563 Soda Hall
  • Phone: (510)643-1455
  • (send email)
  • Link to webcasting of lectures (Active during lectures only; archived video will be posted here after lecture.)

  • To ask questions during live lectures, you have two options:
  • You can email them to this address, which the teaching assistants will be monitoring during lecture.
  • You can use the chat box at the bottom of the webpage of Class Resources and Homework Assignments.
  • Syllabus and Motivation

    CS267 was originally designed to teach students how to program parallel computers to efficiently solve challenging problems in science and engineering, where very fast computers are required either to perform complex simulations or to analyze enormous datasets. CS267 is intended to be useful for students from many departments and with different backgrounds, although we will assume reasonable programming skills in a conventional (non-parallel) language, as well as enough mathematical skills to understand the problems and algorithmic solutions presented. CS267 satisfies part of the course requirements for a new Designated Emphasis ("graduate minor") in Computational Science and Engineering.

    While this general outline remains, a large change in the computing world has started in the last few years: not only are the fastest computers parallel, but nearly all computers will soon be parallel, because the physics of semiconductor manufacturing will no longer let conventional sequential processors get faster year after year, as they have for so long (roughly doubling in speed every 18 months for many years). So all programs that need to run faster will have to become parallel programs. (It is considered very unlikely that compilers will be able to automatically find enough parallelism in most sequential programs to solve this problem.) For background on this trend toward parallelism, click here.

    This will be a huge change not just for science and engineering but the entire computing industry, which has depended on selling new computers by running their users' programs faster without the users having to reprogram them. Large research activities to address this issue are underway at many computer companies and universities, including Berkeley's ParLab, whose research agenda is outlined here.

    While the ultimate solutions to the parallel programming problem are far from determined, students in CS267 will get the skills to use some of the best existing parallel programming tools, and be exposed to a number of open research questions.

  • Tentative Detailed Syllabus
  • Grading

    There will be several programming assignments to acquaint students with basic issues in memory locality and parallelism needed for high performance. Most of the grade will be based on a final project (in which students are encouraged to work in small interdisciplinary teams), which could involve parallelizing an interesting application, or developing or evaluating a novel parallel computing tool. Students are expected to have identified a likely project by mid semester, so that they can begin working on it. We will provide many suggestions of possible projects as the class proceeds.

    Asking Questions

    Outside of lecture, you are welcome to bring your questions to office hours (posted at the top of this page). If you cannot physically attend office hours, you may contact the instructor team via the instructor email. We encourage you to post your questions to the CS267 Piazza page (sign up first). If you send a question to the instructor email, we may answer your question on Piazza if we think it might help others in the class. During lecture, you can ask questions over the Internet (see Google Chat link at bottom of the resources page). You will submit homeworks via the instructor email - please check with assignment-specific submission instructions first.

    Class Projects

    You are welcome to suggest your own class project, but you may also look at the following sites for ideas:

  • the ParLab webpage,
  • the Computational Research Division and NERSC webpages at LBL,
  • class posters from CS267 in Spring 2010
  • class posters and their brief oral presentations from CS267 in Spring 2009.
  • Announcements

  • (Apr 21) The project poster session will be udring the regular lecture time on Thursday of RRR week, Thursday May 3. We will start in the regular lecture room, 250 SDH, and videorecord your short summaries of each poster, after which we will have a regular poster session in the hallway outside. For the short summaries of each poster, please send me one powerpoint slide before the session (preferably by the day before, or very early in the morning),so I can put them all on my laptop. Final project reports will be due the following Monday, May 7, by midnight.
  • (Mar 2) The link to Richard Gerber's guest Lecture 10 has been repaired.
  • (Feb 15) Details about Homework 2 have been posted here, due Mar 6 by midnight.
  • (Jan 18) Details about Homework 1 (not to be confused with Homework 0) have been posted here, due Feb 14 by midnight. Make sure you have completed the (required) course survey, so we can assign you your teammate. Also, please read the "Asking Questions" section above.
  • (Jan 18) Please read the NERSC Computer Use Policy Form so that you can sign a form saying that you agree to abide by the rules state there.
  • (Jan 17) A corrected pointer to the archived video of the lectures has just been posted (2:36pm).
  • (Jan 17) Please complete the following class survey.
  • (Jan 17) We have been assigned 254 SDH (next to the main classroom 250 SDH) as an overflow room, so that we will be able to accomodate all students on the wait-list.
  • (Jan 17) Homework Assignment 0 has been posted here, due Feb 2 by midnight.
  • (Jan 17) This course satisfies part of the course requirements for a new Designated Emphasis ("graduate minor") in Computational Science and Engineering.
  • (Jan 17) For students who want to try some on-line self-paced courses to improve basic programming skills, click here. You can use this material without having to register. In particular, courses like CS 9C (for programming in C) might be useful.
  • Class Resources and Homework Assignments.

  • This will include, among other things, class handouts, homework assignments, the class roster, information about class accounts, pointers to documentation for machines and software tools we will use, reports and books on supercomputing, pointers to old CS267 class webpages (including old class projects), and pointers to other useful websites.
  • Please read the NERSC Computer Use Policy Form so that you can sign a form saying that you agree to abide by the rules state there.
  • Lecture Notes and Video

  • Live video of the lectures may be seen here (only while the lecture is being given).
  • Archived video, posted after the lectures, may be found here (updated 1/17, 2:36pm)
  • Notes from previous offerings of CS267 are posted on old class webpages available under Class Resources
  • In particular, the web page from the 1996 offering has detailed, textbook-style notes available on-line that are still largely up-to-date in their presentations of parallel algorithms (the slides to be posted during this semester will contain some more recently invented algorithms as well).

  • Lectures from Spr 2012 will be posted here.
  • Lecture 1, Jan 17, Introduction (ppt)
  • Lecture 2, Jan 19, Single Processor Machines: Memory Hierarchies and Processor Features (ppt) (updated Jan 24, 8:45am)
  • Lecture 3, Jan 24, complete Lecture 2 (updated Jan 24, 8:45am) and then Introduction to Parallel Machines and Programming Models(ppt) (updated Jan 26, 8:55am)
  • Lecture 4, Jan 26, complete Lecture 3 (updated Jan 26, 8:55am) and then Sources of Parallelism and Locality in Simulation - Part 1(ppt) (updated Jan 26, 8:55am)
  • Lecture 5, Jan 31, complete Lecture 4, and then Sources of Parallelism and Locality in Simulation - Part 2(ppt)
  • Lecture 6, Feb 2, Shared Memory Programming: Threads and OpenMP (ppt), and Tricks with Trees (ppt)
  • Lecture 7, Feb 7, complete lecture on Tricks with Trees, then Distributed Memory Machines and Programming (ppt)
  • Lecture 8, Feb 9, Partitioned Global Address Space Programming with Unified Parallel C (UPC)(ppt), guest lecture by Kathy Yelick
  • Lecture 9, Feb 14 Introduction to GPUs (pdf) by Bryan Catanzaro
  • Lecture 10, Feb 16. This will be a two part lecture:
  • Tools for Performance Debugging HPC Applications (pptx) by David Skinner
  • Debugging and Optimization Tools (pdf) by Richard Gerber
  • NERSC web site with more documentation and videos about using tools at NERSC
  • Lecture 11, Feb 21, Dense Linear Algebra - Part 1 (ppt)
  • Lecture 12, Feb 23, Dense Linear Algebra - Part 2 (ppt)
  • Lecture 13, Feb 28, Graph Partitioning (ppt)
  • Lecture 14, Mar 1, complete Lecture 13, then begin Automatic Performance Tuning and Sparse-Matrix-Vector-Multiplication (SpMV) (ppt)
  • Lecture 15, Mar 6, complete Lecture 14, updated version here (ppt)
  • Lecture 16, Mar 8,
  • Hierarchical Methods for the N-Body Problem (pptx) (updated 8:40am, Mar 13)
  • Efficient Data Race Detection for Distributed Memory Parallel Programs (pptx) by Chang-Seo Park
  • Lecture 17, Mar 13,
  • Complete Hierarchical Methods for the N-Body Problem (ppt),
  • Begin Structured Grids (ppt), (updated 8am, Mar 22)
  • Lecture 18, Mar 15, Cloud Computing with MapReduce and Hadoop (ppt), by Matei Zaharia
  • Lecture 19, Mar 20, Architecting Parallel Software with Patterns (pptx) (updated 3/20, 12:30pm), by Kurt Keutzer
  • Lecture 20, Mar 22,
  • Complete Lecture 17 on Structured Grids (ppt), (updated 8am, Mar 22)
  • Discuss Class Project Suggestions
  • Lecture 21, Apr 3, Frameworks for Structured Software Development by John Shalf
  • For more information about these and other frameworks, including the software itself and tutorials, see the DOE ACTS Collections
  • Lecture 22, Apr 5, Exascale Computing by Katherine Yelick
  • Lecture 23, Apr 10, Parallel Graph Algorithms by Aydin Buluc
  • Lecture 24, Apr 12, Parallel Climate Modeling by Michael Wehner
  • Lecture 25, Apr 17, Parallel Fast Fourier Transform (FFT) (updated 4/17, 8:05pm)
  • Lecture 26, Apr 19, Dynamic Load Balancing
  • Lecture 27, Apr 24, Big Bang, Big Data, Big Iron: High Performance Computing and the Cosmic Microwave Background by Julian Borrill
  • Lecture 28, Apr 26, Accelerated Materials Design through High Throughput First-Principles Calculations and Data Mining: Part 1 (pdf) and Part 2 (pdf), by Kristin Persson
  • Sharks and Fish

  • "Sharks and Fish" are a collection of simplified simulation programs that illustrate a number of common parallel programming techniques in various programming languages (some current ones, and some old ones no longer in use).
  • Basic problem description, and (partial) code from 1999 class, written in Matlab, CMMD, CMF, Split-C, Sun Threads, and pSather, available here.
  • Code (partial) from 2004 class, written in MPI, pthreads, OpenMP, available here.