James Demmel

Professor of Mathematics and Computer Science
Dr. Richard Carl Dehmel Distinguished Professor
Former Chief Scientist of CITRIS, the Center for Information Technology Research in the Interest of Society

Office:

831 Evans Hall
564 Soda Hall (also known as "Virginia")
Department of Mathematics
Computer Science Division
University of California at Berkeley
Berkeley CA 94720-1776
Email: demmel@cs.berkeley.edu
Office: (510) 643-5386
FAX: (510) 642-3962

Fall 2014 - Teaching
  • Math 221, Matrix Computations

  • Office Hours - T 10:30-11:30 (changed as of Sept 16), Th 1-2, F 1-2 in 564 Soda (subject to change)

    Administrative Assistant:
    Tammy Johnson
    Office: 565 Soda Hall (also known as "Hilgard")
    Email: tamille@eecs.berkeley.edu
    Telephone: (510)643-4816

    Grants Administrator:
    Sherry Liang
    Office: 776 Soda
    Email: liangs@eecs.berkeley.edu
    Telephone: (510)643-1980

    Teaching:

    Graduate Designated Emphasis in Computational Science and Engineering (CSE)


    Teaching for Fall 2014: Math 221, Matrix Computations
    Teaching for Spring 2014: CS267, Applications of Parallel Computers
    Teaching for Fall 2013: Math 16B, Analytic Geometry and Calculus
    Teaching for Spring 2013: CS 267, Applications of Parallel Computers and Math 16B, Analytic Geometry and Calculus
    Teaching for Spring 2012: CS 267, Applications of Parallel Computers
    Teaching for Fall 2011: CS 170, Efficient Algorithms and Intractable Problems,
    Ma 221, Matrix Computations, and
    CS 294, Communication-Avoiding Algorithms
    Teaching for Spring 2011: CS 267, Applications of Parallel Computers and
    CS 70, Discrete Mathematics and Probability Theory
    Teaching for Fall 2010: Math 221, Matrix Computations
    Teaching for Spring 2010: CS 267, Applications of Parallel Computers and CS 170, Efficient Algorithms and Intractable Problems
    Teaching for Fall 2009: Math 221, Matrix Computations
    Teaching for Spring 2009: CS 267, Applications of Parallel Computers
    Teaching for Fall 2008: Math 16A, Analytic Geometry and Calculus
    Guest Lectures for Spring 2008: CS 267, Applications of Parallel Computers
    Teaching for Fall 2007: Math 55, Discrete Mathematics
    Guest Lectures for Spring 2007: CS 267, Applications of Parallel Computers
    Teaching for Spring 2007: CS 170, Efficient Algorithms and Intractable Problems
    Teaching for Spring 2006: CS 267, Applications of Parallel Computers
    Teaching for Fall 2005: Math 110, Linear Algebra
    Teaching for Spring 2005: CS 267, Applications of Parallel Computers
    Teaching for Fall 2004: Math 221, Matrix Computations
    Teaching for Spring 2004: Math 55, Discrete Mathematics
    Teaching for Spring 2002: Math 128a, Numerical Analysis
    Teaching for Spring 2001: CS 170, Efficient Algorithms and Intractable Problems (co-taught by Prof. Jonathan Shewchuk)
    Teaching for Fall 2000: Math 55, Discrete Mathematics
    Teaching for Fall 1999: CS 170, Efficient Algorithms and Intractable Problems
    Teaching for Fall 1999: Math 221, Matrix Computations
    Teaching for Spring 1999: CS 267, Applications of Parallel Computers
    Teaching for Fall 1998: Math 128a, Numerical Analysis
    Teaching for Spring 1998: CS 170, Efficient Algorithms and Intractable Problems
    Teaching for Fall 1997: Math 221, Matrix Computations

    Selected Awards:

    IPDPS Charles Babbage Award, 2013
    AMS Fellow, 2012
    SIAG on Linear Algebra Prize (2012, with G. Ballard, O. Holtz. O. Schwartz; 1991, with W. Kahan; 1988)
    National Academy of Sciences, 2011
    Sidney Fernbach Award, 2010
    SIAM Fellow, 2009
    Distinguished Alumnus Award in Computer Sciences and Engineering, UC Berkeley, 2004
    Invited Speaker, International Congress on Industrial and Applied Mathematics (ICIAM), 2003
    Invited Speaker, International Congress of Mathematicians (ICM), 2002
    IEEE Fellow, 2001
    National Academy of Engineering, 1999
    ACM Fellow, 1999
    NSF-CBMS Lecturer on Parallel Numerical Linear Algebra, San Francisco, 1995
    J. H. Wilkinson Prize in Numerical Analysis and Scientific Computing, 1993
    Presidential Young Investigator Award, 1986
    IBM Faculty Development Award, 1985

    Research Projects and Books:

    Communication-Avoiding Algorithms
  • The following talks (given at U. Wisconsin-Madison) survey our work in this field as of September 2013:
  • Communication Avoiding Algorithms for Linear Algebra and Beyond (pptx)
  • Implementing Communication Avoiding Algorithms (pptx)
  • Communication Lower Bounds and Optimal Algorithms for Programs that Reference Arrays (pdf)
  • Keynote talk at IPDPS'13 (1 hour), 22 May 2013, in pptx
  • The following talks survey our work in this field as of November 2012, in 3 different formats (45 minutes, 1 hour, and 1.5 hours):
  • Invited talk at Supercomputing'12 (45 minutes): in pptx or pdf
  • Presentations at RPI, IBM Yorktown Heights and NYU (1 hour): in pptx or pdf
  • Tutorial and Workshop at Supercomputing'12 (1.5 hours): in pptx or pdf
  • Technical reports and papers
  • Tutorial slides for Supercomputing'11
  • Cornell University presentation (on 12/2/10) (in (ppt)) (in (pdf))
  • Sidney Fernbach Award presentation (pdf)
  • 10 hour short course (slides and video), taught in June 2010 at the Gene Golub SIAM Summer School 2010
  • Power point presentation at the 2009 SIAM Applied Linear Algebra Meeting,
  • Longer version (ppt) presented at ACM Colloquium, Caltech, Nov 9 2009
  • Older presentation (ppt) from Householder Meeting in June 2008
  • The ParLab (Parallel Computing Laboratory) is interdisciplinary research center motivated by the multicore revolution.

    The Complexity of Accurate Floating Point Computation. See below for recent talks.

  • Accurate and efficient expression evaluation and linear algebra, or
    Why it can be easier to compute accurate eigenvalues of a Vandermonde matrix than the accurate sum of 3 numbers

    Slides in pdf. Presented at SNC 2011
  • Accurate and efficient expression evaluation and linear algebra
    Slides in pdf. Presented at SCAN 2010
  • Towards accurate polynomial evaluation, or When can Numerical Linear Algebra be done accurately?
    Slides in pdf. presented at FOCM 2005. (updated version of following talk)
  • The Cost of Accurate Numerical Linear Algebra, or Can we evaluate polynomials accurately?
    Slides in pdf. presented at IWASEP 5.
  • Necessary Conditions for Accurate Evaluation of Polynomials. Slides presented at MSRI Workshop on Applications of Real Algebraic Geometry, April 2004, in postscript or pdf.
  • The Complexity of Accurate Floating Point Computation. Slides presented at ICM 2002, in postscript or pdf. (See below for longer versions of this talk.)
  • The Complexity of Accurate Floating Point Computation. In postscript or pdf. In Proc. ICM 2002, Beijing.
  • Accurate and Efficient Computations with Structured Matrices UC Berkeley PhD Dissertation by Plamen Koev.
  • Proposal to update the LAPACK and ScaLAPACK linear algebra libraries, in pdf or postscript.

  • Presentation in ppt on Sca/LAPACK research and development plans
  • Extra-Precise Iterative Refinement (pdf), a technical report describing the proposed LAPACK implementation of iterative refinement of solutions of linear systems
  • Complete data for numerical experiments summarized in the technical report.
  • CITRIS is the Center for Information Technology Research in the interest of Society, a new $300M multicampus research center with the mandate of creating information technology to tackle society's most critical needs. Initial CITRIS research will focus on energy efficiency, transportation, disaster response, health care, education and environmental monitoring.

  • Long CITRIS Overview Presentation, (updated September 2003) (in PowerPoint)
  • Short CITRIS Overview Presentation, (updated March 2003) (in PowerPoint)
  • Long CITRIS Overview Presentation, (updated February 2003) (in PowerPoint)
  • CITRIS Networking Workshop, 1 March 2002 (in PowerPoint)
  • CITRIS Seminar, 7 Feb 2002 (in PowerPoint)
  • SUGAR is a simulation tool for MEMS (MicroElectroMechanical Systems), tiny sensors and actuators that form a pillar of CITRIS's approach to solving large scale societal problems. SUGAR is used by many researchers in the Berkeley Sensor and Actuator Center (BSAC).

  • BSAC IAB Presentation, 12 March 2002 (in PowerPoint)
  • SUGAR Overview, Feb 2001 (in PowerPoint)
  • BeBOP is Berkeley Benchmarking and OPtimization Group. We work on automatic performance optimization of numerical kernels, tuning them to run as fast as possible for particular architectures and sometimes particular inputs.

  • CS 258 Guest Lecture on Performance Tuning, 13 March, 2002 (in PowerPoint)
  • High Performance Information Retrieval and MEMS CAD, 11 Dec, 2001 (in PowerPoint)
  • Selected Papers and Reports
  • Millennium is a campus-wide project to provide hardward and sofware facilities for high performance computing. My activities are to provide algorithms, software and advice to best facilitate computational science and engineering on campus.

  • SimMillennium: Computer Systems, Computational Science and Engineering in the Large, Aug 1999 (in PowerPoint)
  • Templates for the Solution of Algebraic Eigenvalue Problems: This book gives a unified overview of the theory, algorithms and practical software for eigenvalue problems. It organizes this large body of material to make it accessible for the first time to experts and non-experts who need to choose the best state-of-the-art algorithms and software for their problems.

    LAPACK - Linear Algebra PACKage for high performance workstations and shared memory parallel computers. The LAPACK Manual is available on-line.

    (This and much other useful numerical software is available on Netlib.)

    ScaLAPACK - Scalable Linear Algebra PACKage for high performance distributed memory parallel computers, The ScaLAPACK Manual is available on-line.

    Proposal to update the LAPACK and ScaLAPACK linear algebra libraries.

    SuperLU, implementations of sparse Gaussian Elimination for high performance parallel machines

    Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods is a hyper-text book on iterative methods for solving systems of linear equations. A similar book project for eigenvalue problems is underway.

    Selected Recent Talks:

    Communication-Avoiding Algorithms: Please look above under Research Projects and Books for the most recent talks on this topic.
    Der Zahlenlehrling / The Numerical Apprentice, a poem (in ppt) in memory of Gene Golub, presented at the Householder Symposium 2008
    A Celebration of William Kahan and Beresford Parlett, a speech given at Bay Area Scientific Computing Day 2008, in PowerPower2003 (8.6MB) or pdf (3.6MB).
    The Future of LAPACK and ScaLAPACK (in powerpoint), presented at PARA 06 meeting, Umea, Sweden, June 2006
    Towards Accurate Polynomial Evaluation, or When can Numerical Linear Algebra be done accurately? (in pdf), presented at Banff meeting, October, 2005
    The future of numerical linear algebra: Automatic Performance Tuning of Sparse Matrix Kernels, and The Next LAPACK and ScaLAPACK. In powerpoint.
    The Cost of Accurate Numerical Linear Algebra, or Can we evaluate polynomials accurately?
    In pdf. Slides presented at IWASEP 5.
    The Complexity of Accurate Floating Point Computation, or Can we do Numerical Linear Algebra in Polynomial Time?
    In postscript or pdf. Slides presented at the Householder Symposium XV, 2002 and IWASEP 4. Earlier version presented at ISSAC 2001.
    Scaling in Numerical Linear Algebra, Scaling to New Heights, Pittsburgh Supercomputer Center, 20 May 2002 (in PowerPoint)
    CITRIS Seminar, 7 Feb 2002 (in PowerPoint)
    Guest Lecture on Dense Linear Algebra for CS267 (Fall 2001, in PowerPoint)
    High performance Information Retrieval and MEMS CAD on Intel Itanium (in PowerPoint) UCB EECS Department Talk, Oct 12, 2001. (Version from 11 Dec, 2001)
    CITRIS - Center for Information Technology Research in the Interest of Society (in PowerPoint) UCB EECS Department Colloquium, Oct 10, 2001.
    The Complexity of Accurate Floating Point Computation in postscript or in pdf.
    Recent Progress in High Accuracy and High Performance Linear Algebra Algorithms May 4, 1999. Solving systems of linear equations, and finding eigenvalues of symmetric matrices are standard problems that arise in many mathematical modeling problems. Solutions have existed for a long time, but these solutions are no longer good enough, because the demands of applications keep growing to larger problems, and to more ``sensitive'' problems that require higher accuracy. In addition, the larger problems in turn require large parallel computerswhere communication is much more expensive than computation.
    We highlight several new algorithms under development, all of which have solved linear systems and eigenvalue problems many times larger, more sensitive or more quickly than before. Though these algorithms are all very different, one common mathematical approach distinguishes them from their predecessors: The new algorithms deliberately throw away just enough information about the problem to run both efficiently yet accurately. In contrast, their predecessors would either have to use much higher precision, send many more messages, or do many more operations to get the same answers.
    Recent Progress in High Accuracy and High Performance Linear Algebra Algorithms May 13, 1999. This version of the above talk was an invited presentation at the 1999 SIAM Annual Meeting.
    Survey of Parallel Numerical Linear Algebra Libraries Aug 20, 1997. This survey of dense and sparse parallel numerical linear algebra libraries covered a variety of available software for dense and sparse linear algebra problems on parallel computers, including LAPACK, ScaLAPACK, SuperLU and others. It was presented at a short course for NPACI (at the San Diego Supercomputer Center).

    Current Graduate Students:

    Yozo Hida
    Mark Hoemmen
    Jason Riedy
    Vasily Volkov
    William Kramer

    Undergraduate Research Students:

    Meghana Vishvanath
    Deaglan Halligan
    Ankit Jain
    Weihua Shen
    Berkat Tung
    Michael DeLorimer
    Shoaib Kamil
    Jimmy Iskandar
    Anil Kapur
    Michael Martin
    Brandon Thompson
    Teresa Tung
    Daniel Yoo
    Ben Wanzo
    Suh Kang
    Henry Chen
    Josh Genauer

    Gone but not forgotten:

    Jiawang Nie
    Plamen Koev
    David Garmire
    David Bindel
    Tzu-Yi Chen
    Howard Robinson
    Mark Adams
    David Blackston
    Chee-whye Chin
    Inderjit Dhillon
    Balazs Kralik
    Dominic Lam
    Xiaoye Li
    Huan Ren
    Sharon Smith
    Ken Stanley
    Jinqchong Teo
    Oleg Zakharov
    Melody Ivory
    Andrei Zege

    More on Teaching Activities:

    In Fall 08 we started a New Graduate Designated Emphasis in Computational Science and Engineering (CSE), which operates as a "graduate minor". Over 100 faculty from 20 departments and programs are participating.

    Math 128a is a one semester upper division course on numerical analysis.

    CS 170 is a one semester upper division course on efficient algorithms and intractable problems.

    Math 221 is a one semester course in matrix computations. My text for the course, Applied Numerical Linear Algebra, was published by SIAM in August 1997.

    CS 267 is a one-semester graduate class in Applications of Parallel Computers.

    (Fall 2001 version of CS 267), offered by Prof. Katherine Yelick)

    NSF-CBMS Short Course on Parallel Numerical Linear Algebra, held summer 1995, and based on the Spring 1995 version of CS 267.

    Computational Science Education Project provides an on-line text book on high-performance computing. I co-authored the chapter on Parallel Numerical Linear Algebra

    Math 55 is a one-semester undergraduate course in discrete mathematics, offered Spring semester 1997.

    Biographical Sketch:

    James Demmel received his BS in Mathematics from Caltech in 1975 and his Ph.D. in Computer Science from UC Berkeley in 1983. After spending six years on the faculty of the Courant Institute , New York University , he joined the Computer Science Division and Mathematics Department at Berkeley in 1990, where he holds a joint appointment.