GROMACS

GROMACS (GROningen MAchine for Chemical Simulations) is a package to perform molecular dynamics simulations, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles. The following description is from the GROMACS Home page: http://www.gromacs.org. – “It is primarily designed for biochemical molecules like proteins, lipids and nucleic acids that have a lot of complicated bonded interactions, but since GROMACS is extremely fast at calculating the nonbonded interactions (that usually dominate simulations) many groups are also using it for research on non-biological systems, e.g. polymers.”

The versions that we have on the cluster are:

  • GROMACS 4.6.7
  • GROMACS 4.6.7 GPU - compiled to run on our GPU nodes.
  • GROMACS 5.1.4 - an earlier build, use 5.1.5 if you can.
  • GROMACS 5.1.5
  • GROMACS 5.1.5 GPU - compiled to run on our GPU nodes.

These versions can be found under /opt. Note: The GPU versions are only installed on the GPU nodes so you will need to login to a GPU node (see Accessing GPU Nodes) to find them under /opt.

You can also compile your own version in your home directory.

Thanks to Sheikh Imamul Hossain for proving some of the data and usage information about GROMACS on this page.

Running GROMACS 5

At the GROMACS site there is a Getting Started Guide. There are several excellent tutorials by Justin Lemkul here: GROMACS Tutorials.

This assumes you will be using a GROMACS version under /opt. Setup GROMAC’s environment:

$ source /opt/gromacs5-5.1.5/bin/GMXRC.bash

These commands should give you a summary of the build information and a help page for the mdrun command:

$ gmx --version
$ gmx mdrun -h

Of course GROMACS runs should be done via a PBS submission script. Here is an example.

#!/bin/bash 

#PBS -N GMACS_Test
#PBS -l walltime=0:30:00
#PBS -l ncpus=16
#PBS -l mem=8GB
#PBS -m abe 
#PBS -M Your.Email@uts.edu.au

# Note: sourcing GMXRC.bash will start a new shell which will change your directory to $HOME. 
# Therefore make sure that the "cd $PBS_O_WORKDIR" comes after the source command. This will 
# change your working directory back to the PBS job directory. 
source /opt/gromacs5-5.1.5/bin/GMXRC.bash
cd $PBS_O_WORKDIR

gmx mdrun -nt $NCPUS -pin on -deffnm npt 

How Many Cores do I Need?

Just asking for more cores will not necessarily give you better performance. Generally for applications that are mult-threaded you will get diminishing speedups as the number of cores requested increases, as can be seen in the real world example for a GROMACS job below. For this user they decided to use 28 cores as there was no significant speedup using any more cores. Requesting less cores also means they have a better chance of their job running sooner when the cluster is very busy.

You should test your applications ability to scale with data sets representative of your work. Perform runs with more cores and note the time required.

GROMACS image

Problems

Note that some GROMACS versions (maybe all) have a bug; Bug #2211 gmx writes normal output to stderr. This means that when you run GROMACS production runs under PBS (which you always need to do) the PBS standard output file (.o<job_id>) will be empty and your PBS standard error file (.e<job_id>) will contain your standard output! The developers need to fix GROMACS.

Build Notes for GROMACS 5.1.4

This notes will assist you if you wish to build your own version of GROMACS under your home directory. This was done for 5.1.4, you should try and use a later version if you can. Read the install notes at: http://www.gromacs.org/Documentation/Installation_Instructions_5.0#quick-and-dirty-installation

Requirements

We need to use a later GCC, at least 4.7, so we need to load devtoolset-3 (see https://hpc.research.uts.edu.au/software_general/gcc/).

$ gcc --version
gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)

$ scl enable devtoolset-3 bash

$ ermdc13 build/$ gcc --version
gcc (GCC) 4.9.2 20150212 (Red Hat 4.9.2-6)

$ which gcc
/opt/rh/devtoolset-3/root/usr/bin/gcc

Good. Now we are using the later gcc.

Download Source Code

This is the site we will download the source from: http://manual.gromacs.org/documentation/5.1.4/download.html. It provides the link to the source code and the MD5 checksum which should be: ba2e34d59b3982603b4935d650c08040. We will install into a directory src.

$ cd src 
src/$ wget http://ftp.gromacs.org/pub/gromacs/gromacs-5.1.4.tar.gz
src/$ md5sum gromacs-5.1.4.tar.gz
ba2e34d59b3982603b4935d650c08040  gromacs-5.1.4.tar.gz
$ 

Good. The checksum matches OK. You do use checksums when they are available? Now we can extract the source.

$ tar xvzf gromacs-5.1.4.tar.gz

The source will now be in src/gromacs-5.1.4

Configure and Build

FFTW: The installation notes say that FFTW should be compiled to work best with GROMACS. You can either compile it yourself (use: -DGMX_FFT_LIBRARY=fftw) or permit the GROMACS installation to download and build FFTW from source (via -DGMX_BUILD_OWN_FFTW=ON). We will use the latter which will tell GROMACS to download and compile it for us.

This example will install into /shared/homes/XXXXXX/gromacs-5.1.4

$ cd gromacs-5.1.4
$ mkdir build
$ cd build
$ cmake .. -DCMAKE_INSTALL_PREFIX=/shared/homes/XXXXXX/gromacs-5.1.4 -DGMX_BUILD_OWN_FFTW=ON 
.....
.....
-- Configuring done
-- Generating done
-- Build files have been written to: /shared/homes/XXXXXX/src/gromacs-5.1.4/build

If the configure step was successfull we can now build. We will use the -j 8 flag to the compiler to use 8 cores which will significantly reduce the build time.

$ make -j 8 

OK, no errors.

$ make check
100% tests passed, 0 tests failed out of 20

Now install:

$ make install

OK, your GROMACS will now be in /shared/homes/XXXXXX/gromacs-5.1.4/. To use this GROMACS you will need to setup it’s environment:

$ source ~/gromacs-5.1.4/bin/GMXRC.bash

If all is working these commands should give you a help page and build information:

$ gmx --version
$ gmx mdrun -h

Build Notes for GROMACS 2018

Coming soon.

This is custom footer