In the examples below if the prompt is shown as local$
then you will need to run the
command on a terminal on your own local Linux machine. This is because the command will
require root privileges. If the prompt contains hpc$
then you can run the command on the
HPC.
Remember: To get help on singularity commands type singularity help
and
help for the build options with singularity help build
.
For this step we need to be on your own local Linux machine! This is what you will see if you try and build an image on the HPC.
hpc$ singularity build centos7.sif centos7.def
FATAL: You must be the root user, however you can use --remote or --fakeroot to build from a Singularity recipe file.
So for this section login to your own Linux machine or a virtual linux machine (using VirtualBox https://www.virtualbox.org or a Nectar virtual machine https://nectar.org.au ).
Create a singularity definition file called centos7.def
like this:
BootStrap: yum
OSVersion: 7
MirrorURL: http://mirror.centos.org/centos-%{OSVERSION}/%{OSVERSION}/BaseOS/$basearch/os/
Include: yum
%labels
Maintainer Mike Lake
Version 1.0
%help
This is a Centos 7 Singularity container for running fortune and cowsay.
%runscript
echo "Running the main script from inside the Centos 7 container."
fortune | cowsay
%setup
# These are only required if running on the HPC.
mkdir -p ${SINGULARITY_ROOTFS}/opt
mkdir -p ${SINGULARITY_ROOTFS}/scratch
mkdir -p ${SINGULARITY_ROOTFS}/shared
%post
echo "Installing packages into the Centos 7 container"
yum -y install epel-release
yum -y install vim-minimal
yum -y install perl
yum -y install fortune-mod cowsay
The sections you can use (e.g. %runscript) are described here: https://sylabs.io/guides/3.5/user-guide/definition_files.html
Now build your image.
local$ sudo singularity build centos7.sif centos7.def
This will have created a largish file (~ 100 MB) centos7.sif
.
You can perform this step on the HPC. Use scp
or your whatever client you use to copy
files to the HPC to upload your singularity container image.
To find out what this container was built for we ask for some help. All singularity containers should show a basic help message.
hpc$ singularity run-help centos7.sif
This is a Centos 7 Singularity container for running fortune and cowsay.
Now run the container’s main run script:
hpc$ singularity run centos7.sif
Hello from inside the singularity of Centos 8.
< Happiness is a hard disk. >
---------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
This output came from inside this Singularity container running on the HPC.
You can also check what the operating system is inside your container.
All Centos releases have a file /etc/os-release
containing release information.
Show this file inside the container with:
hpc$ singularity exec centos7.sif cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
If you ran this from the login node you would get this:
hpc$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="8 (Core)"
Note: see later for how to run a singularity container as a PBS job!
When Singularity builds containers, the output format can be one of multiple formats.
A common workflow is to use the “sandbox” mode for development of the container, and then build the final default Singularity image when done, as the default format is read-only image so cannot be modified.
local$ sudo singularity build --sandbox centos7_devel centos7.def
This will have created a directory centos7_devel. You can create a sandbox without root privileges, but to ensure proper file permissions it is recommended to do so as root.
Note: If you have already created an image and wish to overwrite it, you can do so
with the –force option. This option must follow the build command.
$ sudo singularity build --force --sandbox centos7_devel centos7.def
Using the Singularity “shell” command we can shell into this sandbox and modify it.
You can use the singularity “shell” command to explore your sif file or sandbox. Here we shell into our read-only sif container. We do not need to use sudo. We can use all the normal commands that we would use in Linux:
hpc$ singularity shell centos7.sif
Singularity>
Singularity> ls /usr/
bin etc games include lib lib64 libexec local sbin share src tmp
Singularity>
Here we shell into our sandbox directory centos7_devel. We do need to use sudo here.
local$ sudo singularity shell centos7_devel
Singularity>
In the above examples, we shell into our sif container or sandbox, but they are still read-only. A sandbox container is only writable if you “shell into it” with the –writable option. It’s usual to do this as root to ensure you have permission to access the files and directories that you want to modify.
local$ sudo singularity shell --writable centos7_devel
Singularity>
In the above you will find that you are root and the container is writable.
Let’s shell into our singularity container on the HPC.
hpc$ singularity shell centos7.sif
Singularity>
Run the command whoami to see who we are:
Singularity> whoami
mlake
Singularity>
The user remains the same inside and outside of the container. If you enter a container without root privileges, you are unable to obtain root privileges within the container.
Now do this on your local machine where you built your container but invoke it using sudo:
local$ sudo singularity shell centos7.sif
Singularity>
Now we are the root user inside the container. You can check :-)
Singularity> whoami
root
Singularity>
Most importantly you can extract container metadata and the definition file that was used to build the container. Run these two commands to see.
hpc$ singularity inspect centos7.sif
"Maintainer": "Mike Lake",
"Version": "1.0",
"org.label-schema.build-date": "Wednesday_3_July_2019_14:59:48_AEST",
org.label-schema.usage.singularity.version: 3.5.3-1.1.el8
hpc$ singularity inspect --deffile centos7.sif