Access to the compute and storage systems (Betzy, Fram, Saga, NIRD) is permitted only trough secure shell (ssh).
One can use
(secure copy) or
sftp (secure file transfer protocol) and
rsync to upload or download
data. There are graphical frontends for these tools, for instance
WinSCP for MS Windows.
Transferring files between Betzy/Fram/Saga and NIRD
Since NIRD is mounted on the login nodes of Betzy, Fram, and Saga,
one can use regular
mv commands on the cluster login nodes to copy or
move files into or out of the NIRD project areas.
For more information, please check out the Betzy, Fram, and Saga storage page.
scp: secure copy between hosts on a network
Few typical examples when using
Copy single file to home folder on Fram note that folder is omitted, home folder being default:
$ scp my_file.tar.gz <username>@fram.sigma2.no:
Copy a directory to work area:
$ scp -r my_dir/ <username>@fram.sigma2.no:/cluster/work/users/<username>/
Copy a result directory back to my laptop:
$ scp -r <username>@fram.sigma2.no:/cluster/work/users/<username>/my_results /home/some/place
sftp: interactive secure file transfer program (Secure FTP)
In this example we copy all logs with names that start with “out” and
end with “.log” from the
project1 folder to
/nird/projects/project1 on NIRD:
$ sftp <username>@nird.sigma2.no sftp> lcd project1 sftp> cd /nird/projects/project1 sftp> put out*.log
Troubleshooting - Broken pipe error during
Sikt perform daily housekeeping of their DNS and then the connection from
outside to the NRIS services drops that can cause broken pipe error during
file transfer from outside. To avoid this especially while copying large datasets,
it is recommended to open a
screen session from the NRIS cluster/storage login
node and to pull/push the data from there.