Job Types on Betzy
Betzy is designed to run highly parallelized jobs. If you need to run medium-sized jobs, than Fram is a better choice, while for serial jobs you shall use Saga.
For a preprocessing or postprocessing job which only needs one or a few CPU cores, use a preproc job.
For development or testing use the devel queue which is limited to small and short jobs.
Here is a more detailed description of the different job types on Betzy:
Normal
Allocation units: whole nodes
Job Limits:
minimum 4 nodes
maximum 512 nodes
Maximum walltime: 4 days
Priority: normal
Available resources: 1340 nodes, each with 128 CPU cores and 244 GiB RAM
Parameter for sbatch/salloc:
None, normal is the default
Job Scripts: Normal
This is the default job type. In normal jobs, the queue system hands out complete nodes.
Accel
Allocation units: CPUs, Memory and GPUs
Job Limits:
Maximum walltime: 7 days
Priority: Normal
Available resources: 4 nodes (max 3 per user), each with 64 CPU cores, 494.5 GiB RAM and 4 x Nvidia A100 GPUs with 40 GiB RAM.
Parameter for sbatch/salloc:
--partition=accel
--gpus=N
,--gpus-per-node=N
or similar, withN
being the number of GPUs
Job Scripts: Accel
Can be combined with --qos=devel
for shorter development tasks which require
GPUs for testing.
Note that accel jobs on Betzy are billed differently than normal jobs. See the accounting page for more information.
Preproc
Allocation units: cpus and memory
Job Limits:
maximum 128 billing units (CPU cores plus memory) per job
maximum 1 node per job
maximum 16 running jobs per user
in total maximum 256 billing units in running jobs per user
Maximum walltime: 1 day
Priority: normal
Available resources: 6 nodes, each with 128 CPU cores and 1 TiB RAM
Parameter for sbatch/salloc:
--qos=preproc
Job Scripts: Preproc
preproc jobs are meant for small preprocessing or postprocessing tasks. Typically, such jobs don’t use many CPUs, so requiring them to use 4 whole nodes would waste resources.
Note that preproc jobs on Betzy are billed differently than normal jobs. The details about how the billing units are calculated can be found in job accounting.
Devel
Allocation units: whole nodes
Job Limits:
minimum 1 node, maximum 4 nodes per job
maximum 1 running job per user
Maximum walltime: 60 minutes
Priority: high
Available resources: 4 nodes with 128 CPU cores and 244 GiB RAM
Parameter for sbatch/salloc:
--qos=devel
Job Scripts: Devel
This is meant for small, short development or test jobs.
Can be combined with --partition=accel
to increase priority while
having max wall time and job limits of devel job.
If you have temporary development needs that cannot be fulfilled by the devel job type, please contact us at support@nris.no.