Install your own Python

This page covers installing your own local version of Python. It will be a more recent version than the “system” Python which is 2.6.6. See also the page Install Miniconda Python.

Your own version of Python would be needed if your application or code requires additional Python packages, a specific version or more up-to-date Python package, or a Python package that is only available in a more recent version of Python.

We will be using two small utility programs: scl and virtualenv. SCL is the Software Collection Library which provides more recent versions of python and virtualenv allows us to clone them into our own home directory. See virtualenv --help, scl --help and man scl.

Installing your own Local Python 2.7 or Python 3.x

First off, if you are writing Python code try to use a Python 3.6 version. This is because during “pip upgrades” the following message appears:

“DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won’t be maintained after that date. A future version of pip will drop support for Python 2.7.”

So let’s create a new Python 3.6 environment. If you absolutely need a Python 2.7 the procedure is similar.

We will first need to use the scl utility. This will show the list of what is available in the Software Collection Library:

$ scl --list
  devtoolset-3
  devtoolset-4
  devtoolset-6
  perl516
  python27
  python33
  rh-python34
  rh-python36      <-- We will install this Python version.
  ruby193
$

What python are we running now? It’s the clusters system python.

$ which python
/usr/bin/python         <-- it's the system Python

$ python -V
Python 2.6.6            <-- it's version 2.6

Use scl to enable a new bash shell with Python 3.6 as the default python:

$ scl enable rh-python36 bash
$ 

$ which python
/opt/rh/rh-python36/root/usr/bin/python   <-- scl has set a Python 3.6 environment for us

$ python -V
Python 3.6.3                              <-- Check it's version 3.6

Use virtualenv to clone this Python environment. I store all my Python virtual environments in a subdirectory “virtualenvs” so I change to that directory first.

$ cd virtualenvs

virtualenvs$ virtualenv mypy36
Using base prefix '/opt/rh/rh-python36/root/usr'
New python executable in ~/virtualenvs/mypy36/bin/python3
Also creating executable in ~/virtualenvs/mypy36/bin/python
Installing setuptools, pip, wheel...done.
virtualenvs$ 

Now you need to “activate” this Python environment. Note: you must use source mypy36/bin/activate from bash, you cannot run activate directly like mypy36/bin/activate. A shortcut for source filename is . filename.

 virtualenvs$ . mypy36/bin/activate 
(mypy36) ~/virtualenvs$ 

This will have added the path to this new Python i.e. ~/mypy36/bin to the start of your existing PATH. It will change your PROMPT so you will know you are in the new Python environment. It will have also added a “VIRTUAL_ENV” environment variable.

(mypy36) $ which python
~/virtualenvs/mypy36/bin/python 

(mypy36) ermdc13 mlake/$ python -V
Python 3.6.3

To exit this local python use “deactivate”. This will return you to the systems Python version 2.6.

(mypy36) ~$ deactivate
$ 

Upgrade pip and Other Packages

The first thing you should do now is upgrade pip.

(mypy)$ pip install --upgrade pip
You are using pip version 7.1.0, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting pip
Using cached .... 
Installing collected packages: pip
Found existing installation: pip 7.1.0
Uninstalling pip-7.1.0:
Successfully uninstalled pip-7.1.0
Successfully installed pip-19.1.1
(mypy)$ 

Now you should consider upgrading all the currently installed packages. You can get a list of Python packages which are outdated with:

(mypy) $ pip list --outdated
Package     Version  Latest 
----------  -------  ------ 
h5py        2.8.0    2.9.0  
numpy       1.15.1   1.16.2 
pip         18.0     19.0.3 
scipy       1.1.0    1.2.1  
setuptools  12.0.5   40.8.0 
six         1.11.0   1.12.0 

You can upgrade a package like “numpy” like this:

$ pip install --upgrade numpy

Another way to do this is to install the pip-review package and use that:

$ pip install pip-review
$ pip-review --interactive

The interactive option will ask you if you wish to upgrade for each package or all the packages at once. You can also use the auto option. See pip-review -h.

Installing New Packages

Installing, upgrading and managing packages is via the “pip” Python package manager. See pip help.   Use pip list and/or pip freeze to list your installed Python packages.

Use pip search and pip show to search for and show infomation on Python packages:

$ pip search rna
...
Thestral (0.5.1)   - A simple single cell RNA-seq analysis package
PyRNA (1.0.0.dev4) - View RNA secondary structure and BLAST search it for identification
...

$ pip show PyRNA
Name: PyRNA
Version: 1.0.0.dev4
Summary: View RNA secondary structure and BLAST search it for identification
Home-page: https://github.com/pypa/sampleproject
Author: Jan Aspan

Use pip install <package_name> and pip uninstall <package_name> to install or remove Python packages. Example: Install the numpy and PyRNA packages:

$ pip install numpy PyRNA
Installing collected packages: numpy, PyRNA
Successfully installed PyRNA-1.0.0.dev4 numpy-1.16.2
$
This is custom footer