Multiprocessing by Message Passing MPI

Compile and run MPI Program on the IBM SP and p690

For C program:

twister% mpcc -o example example.c

For C++ program:

twister% mpCC -o example example.C

IBM is amongst vendors who have not incorporated C++ bindings into MPI-1.2. An mpi2c++ package developed at the University of Notre Dame has been installed on Hal. Instead of using the mpCC compiler script with appropriate -I and -L flags, use the script "balky" provided by "mpi2c++" as shown above. Here is a Hello World! example.

For Fortran 77 program:

twister% mpxlf -o example example.f

For Fortran 90 program:

twister% mpxlf90 -o example example.f90

To compile with 64-bit addressing 

If you have an MPI code that requires more than 2GB of memory or addressing, you will need to compile your code with "-q64". Also, instead of using, say, mpcc, to compile a C code, you must use mpcc_r. For example,
twister % mpcc_r -q64 -o myexample myexample.c -lm

This is what would result if you had used "mpcc" instead:

twister % mpcc -q64 -o myexample myexample.c -lm
myexample.c:
ld: 0711-328 ERROR: Undefined initfini symbol: poe_remote_main
ld: 0711-317 ERROR: Undefined symbol: .MPI_Init
ld: 0711-317 ERROR: Undefined symbol: .MPI_Comm_rank
ld: 0711-317 ERROR: Undefined symbol: .MPI_Comm_size
ld: 0711-317 ERROR: Undefined symbol: .MPI_Finalize
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information.

The use of thread-safe version of a compiler applies to all languages, not just for C. For instance, if your program is written in fortran 90, you would use mpxlf90_r instead of mpxlf90 for a 64-bit MPI compilation.

To run job interactively:

or simply

twister% example -procs 4

With the current system configuration, there is no need to create a host.list file (or equivalent) to specify a node (or nodes). All jobs, interactive or batch, must not request more processors than are available in node. For the pSeries, that would be 32. For batch jobs, the limit is further restricted by what is available for the queue. For communication efficiency, all processors of a job belong to the same node. You will not be able to request processors from different nodes and you will not be able to specify a node through the host.list file. Specifying a node will usually result in an aborted run.

To run job through batch:

There are various batch queues between the IBM SP and p690, please consult the Technical Summary page for durther details. Again, You will not be able to mix processors from different nodes in a batch run via the host.list file. All processors used in a batch job belong to the same node.

twister% bsub -q p4-mp8 "poe example -procs 8"

or

twister% setenv MP_PROCS 8
twister% bsub -q p4-mp8 "poe example"
if you prefer to use the MP_PROCS environment variable to control the size of processors.