Having an understanding of the hardware layout of the HPC enables you to understand where your files are, what disks you should use for your data and how your program will run. Below is a schematic of the HPC layout.
Everyone logs into the “login node”. From there you submit your job using a PBS job submission script. The “head node” manages all the jobs distributed over all the “compute nodes”. The PBS scheduler running on the head node will assign your job to one of the compute nodes.
Your home directory under /shared/homes/
is on the Isilon storage system.
It is mounted over the network by the login node and all the compute nodes.
Thus your home directory /shared/homes/XXXXXX
(where XXXXXX is your
staff/student ID) is common across all nodes. However as this is a networked drive
the read and write speed is limited by the network speed.
Each compute node also has several fast local SSD disks (Solid State Drives).
Any files written to, or read from, under the directory /scratch
is usng this
faster filesystem.
This is why it is considerably faster for a compute node to read and write
files under it’s own /scratch
directory than the /shared/homes/
directory.
The drive is faster and the data is not being copied over the network.
This is also why within your PBS submission script you should copy your input
files to the /scratch/XXXXXX
directory before your program runs.
When your program has ended your PBS script should copy your output data back
from /scratch/XXXXXX
to /shared/homes/XXXXXX
.
/shared/homes/XXXXXX
is on the Isilon storage system. Each node has a has a
fast, local SSD disk with a directory /scratch/
. A node can read and write to its local
SSD disk much faster than over the network.
Why do you need to to use /scratch/
for your data reading and writing?