Code Porting    

Compiled below is a list of facts you need to be aware of when trying to compile, link, and run, programs on the Blue Gene:

• The Blue Gene is designed for codes that scale well into hundreds or even thousands of processors. The processors in the Blue Gene are slower (in clock speed) than those on the pSeries or the Linux Cluster. Unless your code scales up well and uses a lot of processors, you may be better off running it with fewer processors on Twister.
• In addition, which of the three SCV systems might be more appropriate for you is another issue to consider. As a reference, we have a comparison of the performances of the three SCV systems.
• Runtime limit is 5 hours.(More details ...)
• You can request up to 512 nodes during business hours (9am - 5pm, Monday to Friday) and up to 1024 nodes at other times.(More details ...)
• No virtual memory.
• To compile, must use a cross compiler. (More ...).
• By default, a cross compiler sets -qarch=440d to enable both floating point units (FPUs) in each processor. To disable this feature, manually add -qarch=440. To see how to benefit from using both FPUs, please consult this IBM document.
• Can not link with dynamic libraries.
• Use only MPI for multiprocessing and hence must execute code through mpirun in batch. (More ...)
• Do not support thread-based paradigms such as OpenMP.
• PESSL is not available, but you may be substitute it with SCALAPACK.
• ESSL, IBM's Engineering and Scientific Subroutine Library, is available for in-processor applications.
• MPMD (Multiple Program Multiple Data) paradigm is not supported.
• If your program is written in C++, where necessary you need to include the header file mpi.h. Due to a bug in the current release of MPICH 1.2, mpi.h must be included before stdio.h and iostream.h, if used. In addition, to make C++-relevant definitions in mpi.h accessible, "-DHAVE_MPI_CXX" must be included in compilation
  lee % blrts_xlC -o myexec -DHAVE_MPI_CXX -O3 . . .
• MPI-2 functionalities are not fully supported. For example, MPI_Put and MPI_Get cannot be used. Parallel I/O is not supported either.
• Many system routines, such as fork, exec, and signal are not supported. Consult the list (Chapter 3, System calls supported by Compute Node Kernel) for supported system routines.
• Eager limit (default is 10000) can be set via mpirun
  mpirun . . . -env "BGLMPI_EAGER=1000" . . .
• The Blue Gene supports 32-bit memory, so -q64 cannot be used. However, it supports 64-bit integers.
• Since virtual paging is not supported and relatively small memory (256MB for virtual node mode and 512 MB for COprocessor mode), memory leak can be especially serious. Remember to deallocate memory you allocated when it is no longer needed.
• There is NO /scratch available.
• A Blue Gene node consists of 2 processors. In the default COprocessor mode, 1 processor is used for computation and the other is dedicated to communication. This results in 1 MPI task per node and the node's entire 512 MB of memory is available to the task. In the Virtual Node mode, both processors are used for computations. In this case, there are 2 tasks per node and both tasks must share the node's 512 MB of memory. (More ...)
• During code development phase, it is a good practice to compile your code with the -g (debug) flag. With -g, in the event that your code encounters runtime errors, you will be able to use totalview to assist you with debugging to identify the problem area. Turning on -g may impact negatively the performance of your code. However, the performance degradation is generally small.
• Use IBM XL cross-compilers (such as blrts_xlc) instead of the corresponding GNU compilers for better performance.

For further details on programming for the Blue Gene, read IBM System Blue Gene Solution: Application Development.