IBM pSeries 690 and pSeries 655

Table of contents

• Overview
• Help Information
• Allocations and Accounting
• File Systems
• Usage policies and batch
• Software
• Compiler notes

Overview 

The Boston University IBM pSeries 690 is a 112-processor system composed of four nodes, named kite.bu.edu (32 processors), pogo.bu.edu (32 processors), frisbee.bu.edu (32 processors), and domino.bu.edu (16 processors). These machines are all now reserved for batch processing. Each p690 node consists of Power4 processors running at 1.3 GHz and sharing 1 GB of memory per processor. There are three levels of cache on each of these machines. Each processor has a 32 KB L1 cache; each pair of processors share a 1.41 MB L2 cache; and each set of eight processors share a 128 MB L3 cache. The combined peak performance of our p690 system is 580 GFLOPS.

The pSeries 655 is a 48-processor system composed of six nodes, named twister.bu.edu, scrabble.bu.edu, marbles.bu.edu, crayon.bu.edu, litebrite.bu.edu and hotwheels.bu.edu. Twister is the interactive machine for the entire set of pSeries machines and is the only one of the machines which users can log in to. The other machines are all reserved for batch processing. Each p655 node consists of 8 Power4 processors running at 1.1 GHz and sharing 16 GB of memory. The p655 caches are configured the same as the p690 caches described above. The combined peak performance of our p655 system is 210 GFLOPS.

Users with accounts on these machines must use ssh to log in to twister.bu.edu. Passwords are shared over the Scientific Computing Facilities, so if you already have an account and password on our other systems, you will have the same login and password on this system.

Help Information 

This page provides only very basic information on the IBM pSeries systems. For additional information, please follow the sidebar links.

• Programming for the IBM pSeries systems (http://scv.bu.edu/computation/pseries/programming.html)
• "Introduction to Scientific Computing on Boston University's IBM p-series Machines" talk (PDF) (http://scv.bu.edu/documentation/presentations/pseries_intro/pseries_intro.pdf)
• p690 links and resources at NCSA (http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/IBMp690/)

Other documents can be found in our documents section or on IBM's web site. You can also access system man pages.

If you are experiencing system problems, please send Email to help@twister.bu.edu.

For more information or help in using or porting applications to the pSeries systems, please contact Doug Sondak (sondak@bu.edu) or Kadin Tseng (kadin@bu.edu).

If you have questions regarding your computer account or resource allocations, please send Email to scfacct@bu.edu.

Allocations and Accounting 

Our allocations policy is outlined in our SCF Users Information document. Each actual CPU hour which you use on the p690 will count as one SU (service units) against your allocation. On the 1.1 GHz IBM p655 machines, you will only be charged 0.85 SUs for use of 1 processor hour. Requests for additional time can be made via the Scientific Computing Facilities Resource Requests pages.

File Systems 

The IBM pSeries systems share a single user filesystem. Home directories are backed up nightly. If you accidentally remove a file, you can request that it be restored by sending email to help@twister.bu.edu. Make sure to specify what machine you were on when you deleted the file(s) and to provide us with the full file pathname(s).

Our file storage section has additional information on what resources are available for storing your files.

Usage policies and batch 

All but one of the nodes on the p690 and p655 systems are reserved for batch processing of jobs. The only interactive node is the p655 system node twister.bu.edu. General interactive login sessions are allowed on this machine only and interactive processes are limited to 10 minutes of CPU usage.

The batch system for the pSeries systems is the Load Sharing Facility (LSF). There are a number of different batch queues for various types of jobs. These include "short" and "long" queues for jobs of different running times and separate queues for 1, 4, 16, and 32 processor jobs. In general, the short queues will run with a higher priority than the long queues. It is very important that you submit your job to the appropriate batch queue and you must always specify a queue name when submitting a job, e.g., bsub -q sp-short progname. Details on the available queues are included in the Scientific Computing Facilities Technical Summary.

All jobs that require more than 10 minutes of CPU time must be submitted through the batch system. A system process monitors the processor consumption of all running jobs and will automatically terminate any job which is not running under the batch system and uses more than 10 minutes of processor time.

There is a very nice X-Windows interface to the batch system using the command xlsbatch. You may also use traditional Unix-style commands. To submit a job, use the command bsub. To see the jobs that are queued, use bjobs. To see the queue parameters use the command bqueues. You may remove or kill a job with the command bkill. For more information on the batch system, see LSF Basics.

Software 

Please consult the packages page to see what software packages are available.

Compiler notes 

• For parallel processing, MPI and OpenMP are available.
• Prefixes, suffixes and other basics
  • Compiler names generally have "xl" prefixes, e.g., xlc, xlf, xlf90, xlf95, xlC
  • Highest level of optimization is -O5
  • To automatically link to the MPI library, prefix the compiler name with "mp," e.g. mpxlc
  • For SMP parallelization use "_r" suffix, e.g. xlf_r
  • OpenMP: use -qsmp=omp flag with "_r" compiler name
  • Automatic parallelization: use -qsmp flag with "_r" compiler name
• To run MPI code do NOT use mpirun, e.g., to run on 4 processors:

  myprog < my.in > my.out -procs 4

• For information on scientific libraries on the IBM systems, see http://scv.bu.edu/computation/pseries/programming.html