Submitting Jobs from the GUI

CONN knows how to talk to a cluster scheduler by itself. Rather than you writing a job script, CONN splits the work into jobs, submits them, watches the queue, and merges the results when they finish — all from the interface.

This is the path most CONN users on the SCC will want.

The SCC uses the SGE scheduler, but many other clusters — and much of the field — use SLURM. CONN ships a profile for each, and the scheduler-specific blocks below have a toggle so you can read either.

Cluster/HPC Settings

From the CONN main window:

Tools → Cluster/HPC Settings

This opens CONN's parallelization settings, where you choose a profile describing how your cluster accepts jobs. CONN ships profiles for Grid Engine, Slurm, PBS/Torque, LSF, HTCondor, and a "Background process" option that just uses the local machine's cores.

On the SCC, select:

Grid Engine computer cluster
Slurm computer cluster

CONN's Grid Engine profile is known to work here — the toolbox's own source notes it was tested on the BU SCC.

What the profile does

A profile is a set of command templates. When CONN submits a job it fills in the keywords JOBLABEL, STDERR, STDOUT, SCRIPT and OPTS:

qsub -N JOBLABEL -e STDERR -o STDOUT OPTS SCRIPT
sbatch --job-name=JOBLABEL --error=STDERR --output=STDOUT OPTS SCRIPT

OPTS is where your own submission options go, and it is the one field you will almost certainly need to change.

Add Your SCC Project

This is the step that catches people. CONN's built-in Grid Engine profile sets its submit options to only:

-l h_rt=12:00:00
-t 12:00:00

There is no project in there. Every SCC job must be charged to a project, so jobs submitted with the profile as shipped will be rejected.

Edit the submit options field to include yours:

-P your_project -l h_rt=12:00:00
--account=your_project -t 12:00:00

Use Test profile in the same dialog to check the settings before running anything real. Then Save them so you do not have to repeat this — CONN stores per-user settings that override the site defaults.

Running a Step on the Cluster

Watching and Managing Jobs

Further Reading