Job Types on Betzy

Betzy is designed to run highly parallelized jobs. If you need to run medium-sized jobs, than Fram is a better choice, while for serial jobs you shall use Saga.

For a preprocessing or postprocessing job which only needs one or a few CPU cores, use a preproc job.

For development or testing use the devel queue which is limited to small and short jobs.

Here is a more detailed description of the different job types on Betzy:

Normal

  • Allocation units: whole nodes

  • Job Limits:

    • minimum 4 nodes

    • maximum 512 nodes

  • Maximum walltime: 4 days

  • Priority: normal

  • Available resources: 1340 nodes, each with 128 CPU cores and 244 GiB RAM

  • Parameter for sbatch/salloc:

    • None, normal is the default

  • Job Scripts: Normal

This is the default job type. In normal jobs, the queue system hands out complete nodes.

Accel

  • Allocation units: CPUs, Memory and GPUs

  • Job Limits:

  • Maximum walltime: 7 days

  • Priority: Normal

  • Available resources: 4 nodes (max 3 per user), each with 64 CPU cores, 494.5 GiB RAM and 4 x Nvidia A100 GPUs with 40 GiB RAM.

  • Parameter for sbatch/salloc:

    • --partition=accel

    • --gpus=N, --gpus-per-node=N or similar, with N being the number of GPUs

  • Job Scripts: Accel

Can be combined with --qos=devel for shorter development tasks which require GPUs for testing.

Note that accel jobs on Betzy are billed differently than normal jobs. See the accounting page for more information.

Preproc

  • Allocation units: cpus and memory

  • Job Limits:

    • maximum 128 billing units (CPU cores plus memory) per job

    • maximum 1 node per job

    • maximum 16 running jobs per user

    • in total maximum 256 billing units in running jobs per user

  • Maximum walltime: 1 day

  • Priority: normal

  • Available resources: 6 nodes, each with 128 CPU cores and 1 TiB RAM

  • Parameter for sbatch/salloc:

    • --qos=preproc

  • Job Scripts: Preproc

preproc jobs are meant for small preprocessing or postprocessing tasks. Typically, such jobs don’t use many CPUs, so requiring them to use 4 whole nodes would waste resources.

Note that preproc jobs on Betzy are billed differently than normal jobs. The details about how the billing units are calculated can be found in job accounting.

Devel

  • Allocation units: whole nodes

  • Job Limits:

    • minimum 1 node, maximum 4 nodes per job

    • maximum 1 running job per user

  • Maximum walltime: 60 minutes

  • Priority: high

  • Available resources: 4 nodes with 128 CPU cores and 244 GiB RAM

  • Parameter for sbatch/salloc:

    • --qos=devel

  • Job Scripts: Devel

This is meant for small, short development or test jobs.

Can be combined with --partition=accel to increase priority while having max wall time and job limits of devel job.

If you have temporary development needs that cannot be fulfilled by the devel job type, please contact us at support@nris.no.