How to parallelize independent tasks on HPC: (09:00-12:00 CET)

Content: Sometimes you need to run many similar calculations. Instead of running them one after the other and managing many job scripts or writing elaborate shell scripts, we will learn how to run them potentially at the same time using a workflow manager without writing many job scripts. We will demonstrate and practice this using Snakemake. If your calculation is a for-loop in Python or R and it takes too long to run, this session is for you.

Course material: https://training.pages.sigma2.no/tutorials/independent-jobs-in-parallel/

Registration: Closed

Instructor: Ole Saastad

Coordinator

  • Espen Tangen

Practical Information

  • This is an online course via zoom. Participants require access to a computer (not provided by the course organisers) with internet connectivity.

Note

It is not mandatory to have an account on national HPC clusters to attend these courses. Participants can follow the lectures and demos without having an account on national resources since the course content is applicable to all HPC systems.

However - the user experience will probably multifold if you do have access.

If you have access already you may follow lectures using your own account/project and should cope just fine. If you do wish to have user access to our training project, you need to apply for access here.

Link to documentation for the application procedure: https://www.sigma2.no/how-apply-user-account.

When applying for access:

  1. select NN9970K: Training and Outreach as the project.

  2. choose Saga under Resources

  3. set the account start date as the date of application

  4. set the account end date to 2026-04-09.