You can store your files on your home, work, and/or scratch directory. All directories are mounted on both submit nodes and all compute nodes. Each directory is assigned a
variable when you log in that you can use for quicker access to that space and reference in your scripts.
NOT BACKED UP: No data, anywhere, is backed up. We recommend using the Research File Storage for those purposes.
To see any of the information below, run the command
- Home Directory (
$HOME): Your home directory is a place only you have access to. If you wish to share your data, see
$WORKdirectory below. You have a 100 GB disk quota.
The path to your home directory is (replace
usernamewith your KU Online ID):
Work Directory (
$WORK): Your work directory is a shared space for you to collaborate with your group. The quota is based off how much storage the owner of the group has purchased directory or indirectly from purchasing nodes.
To find your
$WORKdirectory, run the command.
If you do not see a
$WORK, then your owner group has not bought any, and/or bought nodes in 2013, which did not have storage included:
Scratch Directory (
$SCRATCH): Your scratch directory is a place for you to use as a temporary space for your data processing. The quota for scratch is a finite amount, but is set for the whole volume, which CRC staff will maintain.
Files equal to or greater than 60 days old will be deleted
To find your
$SCRATCHdirectory, run the command.
You can use
$SCRATCH in you submit scripts and make it easier to get around the file systems.
cd $WORK #MSUB -o $SCRATCH/job12/out.txt
To determine how much of your quota you are using on the Panasas file system, login and run
This will produce output similar to the following:
$HOME = /home/username <GB> <soft> <hard> : <files> <soft> <hard> : <path to volume> <pan_identity(name)> 63.63 85.00 100.00 : 3949 85000 100000 : /home/username uid:100000000(username) $WORK = /panfs/pfs.local/work/groupname/username Filesystem Size Used Avail Use% Mounted on panfs://pfs.local/work 2.8T 1.7T 1.1T 61% /panfs/pfs.local/work/groupname/username $SCRATCH = /panfs/pfs.local/scratch/groupname/username Filesystem Size Used Avail Use% Mounted on panfs://pfs.local/scratch 55T 37T 18T 68% /panfs/pfs.local/groupname/username
You must be on KU's network or connected to KU Anywhere to access the Data Transfer Node (DTN).
The KU Community Cluster supports SCP, SFTP and Rsync for transferring files:
Host: transfer.crc.ku.edu Port: 22
- SCP:This command-line utility is included with OpenSSH. Basic use is:
scp username@host1:~/file1 username@host2:~/file1_copy
For example, to copy a file from your home directory on your local computer (e.g.,
~/foo.txt) to your home directory on the HPC, on the command line, enter (replace
usernamewith your KU Online ID username):
scp ~/foo.txt email@example.com:~/foo.txt