NERSC Bassi System Essentials
[
Connecting |
Storage |
Tools |
MPI |
Jobs |
Repository |
UPC
]
Be sure to visit the
NERSC site; they have quite an extensive
documentation
on using the Bassi system. All of the information here can be found
in more detail there; I just list the most essential items on this page.
The frontend login node is bassi.nersc.gov.
To log on, use a SSH program (such as
OpenSSH and PUTTY;
others are listed at here).
If you need to change your password, login to NIM.
Contact support
if you have problems.
There are two storage provided for you on the Bassi system:
- $HOME. 5 GB of space, avalable from every node.
- $SCRATCH. 250 GB of scratch space, available from every node.
There is a nominal 7-day deletion policy, but files may be deleted
anytime after the job completes, so in your batch script you probably
want to copy back the relevant data to more permanent place, like
$HOME.
Neither of the above is backed up. Usage of your quota can be checked
by the program myquota. Do not use /tmp and
/var/tmp; they are slow and limited in space. Note that some
programs (such as the IBM compilers xlc and xlf)
use /tmp unless the you set the environmental variable TMPDIR.
You can set to $SCRATCH by something like
export TMPDIR=$SCRATCH
Various software on Bassi is managed by
modules.
To load a particular module, you need to run module load module_name.
So for example, to load the GNU C C compiler gcc, you'll need to execute
module load gcc. To see all the available modules, use
module avail. To list the modules which have been loaded, use module list.
Some useful modules include
- gnu. This provides various GNU tools, including gzip,
autoconf, and automake. If you are used to GNU version
of various tools, this is a must!
- gcc. This provides the GNU
compiler collection. Version 3.3.3 is the current default, but
other versions are also available if necessary.
- scalapack. Provides the ScaLAPACK library which implements various
linear algebra routines for parallel machines.
- tau. Provides TAU (Tuning and Analysis Utilities),
tools for analyzing the performance of C, C++, Fortran and Java programs.
See the NERSC
TAU page.
- papi. Provides PAPI, a common interface to various hardware
performance counters. See
PAPI website, as well as
NERSC's PAPI page.
- totalview. Graphical debugger for C/C++/Fortran programs
which has support for parallel programs including MPI and OpenMP.
See NERSC's
totalview page.
More complete list (with links to documentation for various modules)
is available at the NERSC Bassi software page.
To use the MPI library (in module mvapich) on Bassi, prefix the compiler
name with "mp". For example, instead of cc you would use
mpcc. No extra
include paths or library are needed.
See NERSC's MPI page for details.
Please refer to the Bassi page for details.
Supercomputer time is a limited (and precious) resource.
Our class have been given 50,000 processor-hours in repository mp309.
Each person has been allocated some fraction of this with some slack.
If you use a node, all the processors on that node belongs to you,
and hence you will be charged for both the processors. Processor time
quota can be checked with getnim -U<username>.
See NESRC's
accounts
page for more information about how accounts are charged.
To prevent unnecessary charge, please
- Debug before submitting a large job: check serial program, try
a small number of processors (might as well use 2 processors,
since you will be charged for it).
- Set memory and time limits to your batch job: if a bug in your program
causes an infinite loop, the job will be stopped earlier.
To use the UPC compiler load the module upc:
module add upc
You can do something like
upcc -o myprogram foo.upc -lsomelibrary
To run a UPC-compiled program, you can do something like
upcrun -n 32 -c 8 /path/to/myprogram
which runs 32 processes on 4 nodes (with 8 processes per node).
You can use the following command to find the configuration used by the UPC compiler (e.g., the networks supported, C backend compiler used, etc.)
upcc --version
More information can be found at
UPC Homepage, in particular see the
UPC Users' Guide,
the upcc
and upcrun
man pages.