Home > Getting Started > The HPC Hardware Layout

The HPC Hardware Layout

Understanding the HPC Hardware Layout

Having an understanding of the hardware layout of the HPCC enables you to understand where your files are, what disks you should use for your data and how your program will run. Below is a schematic of the HPC layout.

Everyone logs into the “login node”. From there you submit your job using a PBS job submission script. The “head node” manages all the jobs distributed over all the “compute nodes”. The PBS scheduler running on the head node will assign your job to one of the compute nodes.

Your home directory under /shared/homes/ is on the Isilon storage system. It is mounted over the network by the login node and all the compute nodes. Thus your home directory /shared/homes/XXXXXX (where XXXXXX is your staff/student ID) is common across all nodes. Each compute node has a local SSD disk with a directory /scratch/.

Your program running on a compute node can read and write to that nodes local SSD disk (the /scratch/ directory) much faster than over the network to the /shared/homes/ directory which comes from the Isilon storage appliance.

This is why within your PBS submission script you should copy your input files to the /scratch/XXXXXX directory before your program runs. Your program should read or write to that local directory. When your program has ended your PBS script should copy your output data back from /scratch/XXXXXX to /shared/homes/XXXXXX.

graph TB; login(Login Node) head(Head Node) node1(Compute Node 1) node2(Compute Node 2) node3(Compute Node 3) node4(Compute Node 4) node5(Compute Node 5) isilon((Isilon Storage)) switch[Network Switch] login --- switch head --- switch isilon --- switch switch --- node1 switch --- node2 switch --- node3 switch --- node4 switch --- node5 subgraph node5 --- store5((local disk)) end subgraph node4 --- store4((local disk)) end subgraph node3 --- store3((local disk)) end subgraph node2 --- store2((local disk)) end subgraph node1 --- store1((local disk)) end style isilon fill:#eef2db style store1 fill:#eef2db style store2 fill:#eef2db style store3 fill:#eef2db style store4 fill:#eef2db style store5 fill:#eef2db

Above is a schematic of the High Performance Computer Cluster (HPCC). Your home directory /shared/homes/XXXXXX is on the Isilon storage system. Each node has a has a local SSD disk with a directory /scratch/. A node can read and write to its local SSD disk much faster than over the network.

Why do you need to to use /scratch/ for your data reading and writing?

This directory is local to your data so its faster to read and write to than your home directory over the network.
Lots of users reading and writing at GB/sec to the local scratch directory does not cause the network to slow down.
If the network has some short-term connectivity problems it’s likely your run will not be affected. If you are reading/writing to your home directory during any outage your run will be adversely affected.