Playing with Singularity

In the examples below if the prompt is shown as local$ then you will need to run the command on a terminal on your own local Linux machine. This is because the command will require root privileges. If the prompt contains hpc$ then you can run the command on the HPC.

If you don’t have a local Linux machine to play with then just follow along. Later you can install your own Linux distribution using a virtual machine created with something like “Virtual Box” (https://www.virtualbox.org).

In the section below we will be following an modified version of the tutorial at: https://github.com/NIH-HPC/Singularity-Tutorial.

Fortune and Cowsay

Firstly lets introduce the two programs that we will install into our singularity container; fortune and cowsay. Neither of these programs are installed on the eResearch HPC.

hpc$ fortune
  bash: fortune: command not found
hpc$ 

You can though install these programs on your local Linux machine if you have one.

Each time you run the fortune program it will present to you a useful fortune.

local$ fortune
The following two statements are usually both true:
  There's not enough documentation.
  There's too much documentation.
       -- Larry Wall in <199709020026.RAA08431@wall.org>
local$ 

The program cowsay is a bit different, it requires an input from stdin (the standard input). The input is usually piped into it from an echo command or from the output of another program (like fortune).

local$ echo 'Hi, I am an angus cow' | cowsay
 _______________________
< Hi, I am an angus cow >
 -----------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
local$ 

and here we pipe the output of the fortune program into cowsay:

local$ fortune | cowsay
 ________________________________________
/ Experience is what causes a person to  \
\ make new mistakes instead of old ones. /
 ----------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
local$ 

Now we understand what those programs do we can download and run a container that contains those programs, thus running those two programs on the HPC even though they are not installed there.

Download and Run a Singularity Container

The container that we will use is from the Singularity Hub, which is is a registry for scientific linux containers. Singularity can reference any container there using the “shub://” protocol. We will be using the lolcow container created by David Godlove.

First download that container using the singularity pull command:

hpc$ singularity pull shub://GodloveD/lolcow
Progress |===================================| 100.0% 
hpc$

You will now have a file named lolcow_latest.sif. Now use the singularity command run to actually run the applications inside the container:

hpc$ singularity run lolcow_latest.sif 
 ______________________________________
/ Q: How many Marxists does it take to \
| screw in a light bulb? A: None: The  |
| light bulb contains the seeds of its |
\ own revolution.                      /
 --------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
hpc$

Try this a few times to see different quotes from “cowsay”.

You can also use the singularity run command which will download, build and run the container directly from the Singularity Hub like this: singularity run shub://GodloveD/lolcow. You can run this on our HPC or your local Linux machine. However that might be needlessly pulling down the image each time, its better to download a container once and run it locally.

Remember that the above cow with it’s comments actually came from two programs; fortune and cowsay. The output of fortune is being piped into the cowsay program. We can just run the fortune command only using the exec option as in the command below.

hpc$ singularity exec lolcow_latest.sif fortune
Bank error in your favor. Collect $200.
hpc$

What’s in a Singularity Container

We can also inspect this image to find out some details about it:

hpc$ singularity inspect lolcow_latest.sif 

"org.label-schema.build-date": "Wednesday_3_July_2019_12:55:25_AEST",
"org.label-schema.build-size": "336MB",
"org.label-schema.schema-version": "1.0",
"org.label-schema.usage.singularity.deffile": "Singularity",
"org.label-schema.usage.singularity.deffile.bootstrap": "shub",
"org.label-schema.usage.singularity.deffile.from": "GodloveD/lolcow",
"org.label-schema.usage.singularity.version": "3.1.1-1.1.fc28"

Using inspect --deffile we can see the definition file (we will cover that next) that was used to create the image. Here we can see that the run command is actually the output of the fortune program piped into cowsay and then lolcat (lolcat just colorises the output).

hpc$ singularity inspect --deffile lolcow_latest.sif 
BootStrap: docker
From: ubuntu:16.04

%post
    apt-get -y update
    apt-get -y install fortune cowsay lolcat

%environment
    export LC_ALL=C
    export PATH=/usr/games:$PATH

%runscript
    fortune | cowsay | lolcat

That ends our quick tutorial on downloading and running singularity images. Typing singularity help will give you an summary of all the commands you can use. Typing singularity help <command> will give you more detailed information about running an individual command.

Now proceed to Building Containers to find out how to build your own singularity images.

This is custom footer