Setup of HPC Status Integration into eResearch Website¶
THIS CAME FROM GIT REPO eresearch-website. Copied to here on 17th June 2019 Just in case there is something that we need from it.
This describes how to integrate HPC status information into the eResearch website. This install is only required if you wish to have HPC status pages on the website under the URL /pbsweb See also SETUP.md which describes the initial setup of the eResearch website.
This setup is done by the system administrator.
This setup does not need to be done by users.
The server this is being setup on is erdev.itd.uts.edu.au
Software Required for the HPC Status Page¶
This software is required for the URL /pbsweb as that is a WSGI “bottle” application.
Also some httpd configuration is required.
- https://github.com/UTS-eResearch/pbsweb
- swig
- gcc
- openssl-devel
- bottle.py
- mod_wsgi
Note: There is no additional software required for the HPC Usage Page: /hpcusage.
The basic eResearch website will have already been installed into /var/www/eresearch-website
TODO !!! Pbsweb Repo: /var/www/pbsweb
- Note this is outside the web directory.
- Contains git repo git@github.com:UTS-eResearch/pbsweb.git
Web directory: /var/www/html/
Installing the HPC Usage Page - hpcusage¶
$ cd /var/www
$ sudo mkdir hpcusage
Installing the HPC Status Page - pbsweb¶
Install pbsweb¶
Create the directory and install the pbsweb bottle web application:
$ cd /var/www
$ sudo mkdir pbsweb
$ sudo chown mlake:www pbsweb
$ git clone git@github.com:UTS-eResearch/pbsweb.git pbsweb
Install PBS Execution Host Environment¶
$ sudo rpm -ivh pbs-mom-11.3.0.121723-0.x86_64.rpm
Preparing... ################################# [100%]
Updating / installing...
1:pbs-mom-11.3.0.121723-0 ################################# [100%]
*** PBS Installation Summary
*** Creating /etc/pbs.conf.11.3.0.121723
*** Replacing /etc/pbs.conf with /etc/pbs.conf.11.3.0.121723
*** Copying startup script.
*** The /etc/pbs.conf file has been modified.
*** End of /usr/pbs/etc/pbs_postinstall
$
Add the IP address of the PBS server to the /etc/hosts file
138.25.37.57 pbsserver
Try to start the pbs mom …
$ sudo /etc/init.d/pbs start
Starting PBS
PBS Home directory /var/spool/PBS does not exist.
Running /usr/pbs/etc/pbs_habitat to create it.
***
*** PBS_HOME is /var/spool/PBS
*** Creating new file /var/spool/PBS/pbs_environment
*** warning, did not set TZ in /var/spool/PBS/pbs_environment
*** This host does not have the PBS server installed.
*** the default name 'pbsserver' will be used.
***
*** The PBS commands have been installed in /usr/pbs/bin.
*** PBS Mom has been installed in /usr/pbs/sbin.
***
*** End of /usr/pbs/etc/pbs_habitat
Home directory /var/spool/PBS created.
PBS mom
$
You should see the pbs_mom daemon running:
$ ps ax | grep pbs
23409 ? Ss 0:00 /usr/pbs/sbin/pbs_mom
You also need to allow the server “erdev” to connect to the HPC PBS server node.
To do this login to the server hpcnode1, open /etc/sysconfig/iptables for
editing and add the IP address of erdev which is 138.25.37.20. Then restart iptables:
# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
...
-A INPUT -m state --state NEW -s 138.25.37.20 -j ACCEPT
...
$ sudo /etc/init.d iptables restart
Now you can test that qstat returns HPC jobs information:
$ /usr/pbs/bin/qstat
... jobs information ...
Aside: This is the error you will get if you dont have the iptables firewall exception added.
$ /usr/pbs/bin/qstat
[mlake@erdev ~]$ /usr/pbs/bin/qstat
No route to host
qstat: cannot connect to server pbsserver (errno=113)
[mlake@erdev ~]$
Install swig and gcc¶
Swig and gcc to compile it are required to enable the web application pbsweb.py to communicate with the PBS server.
$ sudo yum install mod_wsgi
$ sudo yum install swig
swig and mod_wsgi have no dependencies but gcc has quite a few:
$ sudo yum install gcc
==========================================================================
Package Arch Version Repository Size
==========================================================================
Installing:
gcc x86_64 4.8.3-9.el7 base 16 M
Installing for dependencies:
cpp x86_64 4.8.3-9.el7 base 5.9 M
glibc-devel x86_64 2.17-78.el7 base 1.0 M
glibc-headers x86_64 2.17-78.el7 base 656 k
kernel-headers x86_64 3.10.0-229.7.2.el7 updates 2.3 M
libmpc x86_64 1.0.1-3.el7 base 51 k
mpfr x86_64 3.1.1-4.el7 base 203 k
Updating for dependencies:
glibc x86_64 2.17-78.el7 base 3.6 M
glibc-common x86_64 2.17-78.el7 base 11 M
libgcc x86_64 4.8.3-9.el7 base 92 k
libgomp x86_64 4.8.3-9.el7 base 127 k
Transaction Summary
==========================================================================
Install 1 Package (+6 Dependent packages)
Upgrade ( 4 Dependent packages)
You also need to install openssl-devel
$ sudo yum install openssl-devel
===========================================================================
Package Arch Version Repository Size
===========================================================================
Installing:
openssl-devel x86_64 1:1.0.1e-42.el7.9 updates 1.2 M
Installing for dependencies:
keyutils-libs-devel x86_64 1.5.8-3.el7 base 37 k
krb5-devel x86_64 1.12.2-14.el7 base 640 k
libcom_err-devel x86_64 1.42.9-7.el7 base 30 k
libselinux-devel x86_64 2.2.2-6.el7 base 174 k
libsepol-devel x86_64 2.1.9-3.el7 base 71 k
libverto-devel x86_64 0.2.5-4.el7 base 12 k
pcre-devel x86_64 8.32-14.el7 base 477 k
zlib-devel x86_64 1.2.7-13.el7 base 49 k
Updating for dependencies:
e2fsprogs x86_64 1.42.9-7.el7 base 697 k
e2fsprogs-libs x86_64 1.42.9-7.el7 base 165 k
krb5-libs x86_64 1.12.2-14.el7 base 826 k
libcom_err x86_64 1.42.9-7.el7 base 40 k
libss x86_64 1.42.9-7.el7 base 45 k
openssl x86_64 1:1.0.1e-42.el7.9 updates 711 k
openssl-libs x86_64 1:1.0.1e-42.el7.9 updates 949 k
pcre x86_64 8.32-14.el7 base 418 k
Transaction Summary
===========================================================================
Install 1 Package (+8 Dependent packages)
Upgrade ( 8 Dependent packages)
Aside: If you dont install openssl-devel you will get this error when you run the swig compile script:
pbsweb$ ./swig_compile_pbs.sh
/usr/bin/ld: cannot find -lcrypto
/usr/bin/ld: cannot find -lssl
collect2: error: ld returned 1 exit status
pbsweb$
Run the swig compile script¶
Now run the script ./swig_compile_pbs.sh which will create _pbs.so
pbsweb$ ./swig_compile_pbs.sh
pbsweb$
Test the Bottle Web Application¶
Now that swig has been installed and the swig compile script run we can test the bottle application with its own inbuilt webserver:
$ cd pbsweb
pbsweb$ ./pbsweb.py
Bottle v0.12.7 server starting up (using WSGIRefServer())...
Listening on http://localhost:8080/
Hit Ctrl-C to quit.
^C
$
No errors should be displayed. Nothing more needs to be done for the bottle application. We can now setup the httpd server.
Setup Apache and WSGI¶
Under the VirtualHost we need to add:
Alias /hpcusage /var/www/hpcusage
Alias /static_pbsweb /var/www/pbsweb/static_pbsweb
WSGIDaemonProcess wsgi display-name=%{GROUP} processes=2 threads=15
WSGIProcessGroup wsgi
WSGIScriptAlias /pbsweb "/var/www/pbsweb/pbsweb.wsgi"
WSGIChunkedRequest On
<Directory "/var/www/hpcusage">
Options SymLinksIfOwnerMatch
AllowOverride None
Require all granted
</Directory>
<Directory "/var/www/pbsweb/static_pbsweb">
Options None
AllowOverride None
Require all granted
</Directory>