Loading CUDA

On the GPU nodes we have CUDA installed. To use CUDA you need to load the correct PATH and LD_LIBRARY_PATH ¹. As we have a few versions of CUDA installed we have installed a module file for each CUDA version so that it is easy for you to setup the correct CUDA environment. For more information on modules see Using Modules.

Run module avail to see the available modules, including the CUDA ones:

--------------------- /etc/modulefiles ----------------------------
cuda-11.5  cuda-11.8  cuda 12.1  cuda-latest  openmpi-4.0.4  openmpi-4.1.2

The module cuda-latest will always be the same as the latest CUDA version. Here we load the latest version and we can check the PATH and LD_LIBRARY_PATH has now been set to where this version of CUDA resides.

$ module load cuda-latest

$ echo $PATH
/usr/local/cuda/bin:/usr/share/Modules/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/pbs/bin

$ echo $LD_LIBRARY_PATH
/usr/local/cuda/lib64

Note: Although you can load this module on any node, only on a GPU node will there be these PATHS.

You can unload this module with:

$ module unload cuda-latest

Normally you would place the module load and unload commands in your PBS submission script so that your CUDA code will know where to find the correct CUDA libraries.

Note 1: The LD_LIBRARY_PATH is the path for the default library path which is checked for available dynamic and shared libraries by the dynamic loader.