Building Containers

In the examples below if the prompt is shown as local$ then you will need to run the command on a terminal on your own local Linux machine. This is because the command will require root privileges. If the prompt contains hpc$ then you can run the command on the HPC.

Remember: To get help on singularity commands type singularity help and help for the build options with singularity help build.

Building your own Singularity Containers

For this step we need to be on your own local Linux machine! This is what you will see if you try and build an image on the HPC.

hpc$ singularity build centos7.sif centos7.def
FATAL:   You must be the root user, however you can use --remote or --fakeroot to build from a Singularity recipe file.

So for this section login to your own Linux machine or a virtual linux machine (using VirtualBox https://www.virtualbox.org or a Nectar virtual machine https://nectar.org.au ).

Create a singularity definition file called centos7.def like this:

BootStrap: yum
OSVersion: 7
MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/BaseOS/$basearch/os/
Include: yum

%labels
    Maintainer Mike Lake
    Version 1.0

%help
    This is a Centos 7 Singularity container for running fortune and cowsay.

%runscript
    echo "Running the main script from inside the Centos 7 container."
    fortune | cowsay

%setup
    # These are only required if running on the HPC.
    mkdir -p ${SINGULARITY_ROOTFS}/opt
    mkdir -p ${SINGULARITY_ROOTFS}/scratch
    mkdir -p ${SINGULARITY_ROOTFS}/shared

%post
    echo "Installing packages into the Centos 7 container"
    yum -y install epel-release
    yum -y install vim-minimal
    yum -y install perl 
    yum -y install fortune-mod cowsay

The sections you can use (e.g. %runscript) are described here: https://sylabs.io/guides/3.8/user-guide/definition_files.html

Now build your image.

local$ sudo singularity build centos7.sif centos7.def

This will have created a largish file (~ 100 MB) centos7.sif.

Run the containers application

You can perform this step on the HPC. Use scp or your whatever client you use to copy files to the HPC to upload your singularity container image.

To find out what this container was built for we ask for some help. All singularity containers should show a basic help message.

hpc$ singularity run-help centos7.sif 

This is a Centos 7 Singularity container for running fortune and cowsay.

Now run the container’s main run script:

hpc$ singularity run centos7.sif
Hello from inside the singularity of Centos 8.

< Happiness is a hard disk. >
 ---------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

This output came from inside this Singularity container running on the HPC.

You can also check what the operating system is inside your container. All Centos releases have a file /etc/os-release containing release information. Show this file inside the container with:

hpc$ singularity exec centos7.sif cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"

If you ran this from the login node you would get this:

hpc$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="8 (Core)"

Note: see later for how to run a singularity container as a PBS job!

Creating Sandbox Containers

When Singularity builds containers, the output format can be one of multiple formats.

  • a “.sif” file: This is the default format and it is a compressed Singularity read-only image format.
  • a sandbox directory: This is a container within a directory structure, and it is a writable format.

A common workflow is to use the “sandbox” mode for development of the container, and then build the final default Singularity image when done, as the default format is read-only image so cannot be modified.

local$ sudo singularity build --sandbox centos7_devel centos7.def

This will have created a directory centos7_devel. You can create a sandbox without root privileges, but to ensure proper file permissions it is recommended to do so as root.

Note: If you have already created an image and wish to overwrite it, you can do so with the –force option. This option must follow the build command.
$ sudo singularity build --force --sandbox centos7_devel centos7.def

Using the Singularity “shell” command we can shell into this sandbox and modify it.

Using shell to explore and modify containers

You can use the singularity “shell” command to explore your sif file or sandbox. Here we shell into our read-only sif container. We do not need to use sudo. We can use all the normal commands that we would use in Linux:

hpc$ singularity shell centos7.sif
Singularity>
Singularity> ls /usr/
bin  etc  games  include  lib  lib64  libexec  local  sbin  share  src	tmp
Singularity>

Here we shell into our sandbox directory centos7_devel. We do need to use sudo here.

local$ sudo singularity shell centos7_devel
Singularity>

In the above examples, we shell into our sif container or sandbox, but they are still read-only. A sandbox container is only writable if you “shell into it” with the –writable option. It’s usual to do this as root to ensure you have permission to access the files and directories that you want to modify.

local$ sudo singularity shell --writable centos7_devel
Singularity>

In the above you will find that you are root and the container is writable.

Who am I inside the container?

Let’s shell into our singularity container on the HPC.

hpc$ singularity shell centos7.sif
Singularity>

Run the command whoami to see who we are:

Singularity> whoami
mlake
Singularity>

The user remains the same inside and outside of the container. If you enter a container without root privileges, you are unable to obtain root privileges within the container.

Now do this on your local machine where you built your container but invoke it using sudo:

local$ sudo singularity shell centos7.sif
Singularity>

Now we are the root user inside the container. You can check :-)

Singularity> whoami
root
Singularity>

Inspect Containers

Most importantly you can extract container metadata and the definition file that was used to build the container. Run these two commands to see.

hpc$ singularity inspect centos7.sif
  "Maintainer": "Mike Lake",
  "Version": "1.0",
  "org.label-schema.build-date": "Wednesday_3_July_2019_14:59:48_AEST",
  org.label-schema.usage.singularity.version: 3.5.3-1.1.el8

hpc$ singularity inspect --deffile centos7.sif
This is custom footer