Running Multiple MATLAB Tasks

Described in the following is a procedure to run multiple independent MATLAB tasks in a single batch job. Typically, users who want to run MATLAB jobs in this mode do so for parametric studies of the same code under different conditions. These conditions are often controlled by an index or an input file. Since multiple copies of MATLAB are processed concurrently, the computed results from each must be saved in a distinct file. Shown below is a sample batch script that is sufficiently general to accomodate a great variety of situations. This method supercedes an earlier method for the same objective.

1. This batch script is called mpkbatch ("k" stands for Katana to distincguish it from the Linux Cluster version, mplbatch).
   
   mpkbatch:

   #!/bin/csh
   # All SGE batch scheduler commands (for Katana Cluster) start with #$
   # The following requests 4 processors (see scv.bu.edu for details)
   # It can be overridden, for example, with "qsub -pe omp 2 mpkbatch"
   #$ -pe mpi 4
   #  $NSLOTS is the number of processors defined thru -pe above
   unsetenv DISPLAY
   # The Sun Grid Engine batch preserves the current dir in batch. 
   # You could provide your own preferred dir though
   # cd some_dir
   @ rank = 0
   # IMPORTANT: DONOT indent any of the below statements
   while ($rank < $NSLOTS)
   matlab -nojvm  >! myoutput_$rank << MATLAB_ENV &
   myrank = $rank;          % Between MATLAB_ENVs, carry on any MATLAB ops
   outname = ['output.' num2str(myrank)]; % output file name for current proc
   myfunc(myrank, outname); % a function; OK to use $rank instead of myrank
   myscript;                % a script
   exit                     % don't forget to exit
   MATLAB_ENV
   @ rank++
   end
   wait
   

   myfunc.m:

   function myfunc(myrank, outname)
   %function myfunc(myrank, outname)
   disp('This is a MATLAB function')
   if myrank == 0
     disp('Perform a task')
   else
     disp('Perform a different task')
   end
   % can also construct outname here with myrank
   save(outname);    % save data
   

   myscript.m:

   disp('This is a MATLAB script')

   In the above example, four instances of MATLAB (resulting from -pe mpi 4) are launched. The myoutput_* files trap any output sent to the screen by each MATLAB instant.

   Please avoid combining tasks that vary significantly in compute time as the tasks that finish first will wait for the longest task to complete and therefore waste computing resources. Please call or email Kadin Tseng (kadin@bu.edu) if you'd like to explore other possibilities of multiprocessing with MATLAB.

2. Give mpkbatch execute attribute

    cootie:~ % chmod +x mpkbatch

3. Submit batch job to the batch queue requesting four processors and a (default) two hours wallclock time.

   cootie:~ % qsub mpkbatch

4. You can override resource parameters pre-defined in mpkbatch as follows:

   cootie:~ % qsub -pe mpi 8 -l h_rt=24:00:00 mpkbatch