Setup python 3 (including mpi4py) without admin rights

The following guide shows how to get a complete python installation without administrator rights:

Step 1: We move to the folder where we want to place our installation:

/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3

Step 2: We download from the python website, compile and install the latest version of python 3:

mkdir python_src
cd python_src
wget https://www.python.org/ftp/python/3.8.0/Python-3.8.0.tar.xz
tar xvf Python-3.8.0.tar.xz
cd Python-3.8.0
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3
make
make install

Possible issues with ssl: In order to be able to use pip at a later stage, python needs to be compiled with ssl support. Please check that this is the case:

/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3
import ssl

In case this gives an exception, please install ssl library and recompile python (in Ubuntu this can be done by sudo apt install libssl-dev).

Possible issue with ctypes and libffi: Check that ctypes is working (will be required for several reasons afterwards):

/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3
import ctypes

If you get an error, it is because the ffi library is not installed. You can download and install the library as follows:

cd /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python_src
wget ftp://sourceware.org/pub/libffi/libffi-3.3.tar.gz
tar -xvf libffi-3.3.tar.gz
cd libffi-3.3/
mkdir /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/libffi
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/libffi
make
make install

The library location needs to be added to our library path:

export LD_LIBRARY_PATH=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/libffi/lib64:$LD_LIBRARY_PATH

The python compilation needs some extra flags:

./configure --prefix=/home/HPC/lusabato/sim_workspace_python_3/python3 LDFLAGS="-L/home/HPC/lusabato/sim_workspace_python_3/libffi/lib/../lib64" CFLAGS="-I/home/HPC/lusabato/sim_workspace_python_3/libffi/include

The continue as normal:

make make install

 




**Possible issues with sqlite3:**
In order to be able to use jupyter notebooks at a later stage, python needs to be compiled with sqlite3 support. Please check that this is the case:

    /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3
    import sqlite3

In case this gives an exception, please install sqlite library and recompile python (in Ubuntu this can be done by sudo apt install libsqlite3-dev).

**Possible issues with tk:**
In order to be able to use ipython --pylab at a later stage, python needs to be compiled with tk support. Please check that this is the case:

    /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_p3/python3/bin/python3
    import _tkinter

In case this gives an exception, please install tk library and recompile python (in Ubuntu this can be done by sudo apt install tklib; sudo apt install tk-dev ).


**Step 3:** We create a virtual environment

    cd /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3
    mkdir venvs
    cd venvs
    /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3 -m venv py3
             

**Step 4:** We activate the virtual environment

source /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/venvs/py3/bin/activate

If we type:
    
    which python
we get:

    /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/venvs/py3/bin/python

**Step 6**: We use pip to install the python modules that we need
 
     pip install numpy scipy cython ipython matplotlib

**Steps 5 and 6 are required only to run parallel simulations using MPI.**

**Step 5 - Check that MPI is available:**

You can use the following commands to check that an MPI installation is available

which mpicc which mpiexec

In case it is not, please activate an MPI installation (Step 5.a) or install a new MPI installation (Step 5.b).

**Step 5.a - Activate MPI:**

On certain machines an MPI installation can be activated using ```module load``` (it is the case for CERN lxplus and CERN HPC cluster) or ```mpi_selector``` (it is the case for the INFN-CNAF cluster)

**Step 5.b - Install MPI:**

If we do not have an MPI installation we need to get one:

     cd /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python_src
     wget https://www.open-mpi.org/software/ompi/v1.10/downloads/openmpi-1.10.2.tar.bz2
     tar jxf openmpi-1.10.2.tar.bz2
     cd openmpi-1.10.2
    ./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/openmpi
     make all install

We set the environment variable for the MPI compiler (pip will use the compiler pointed by your MPICC variable to compile mpi4py):

    export MPICC=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/openmpi/bin/mpicc

At this point be careful not to have other MPI paths in your environment (for example set in your .bashrc)

**Step 6:** Install mpi4py

    pip install mpi4py

_Important:_ Python jobs using mpi4py must be run with the mpiexec corresponding to the MPI compiler that has been used. In our case:

    /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/openmpi/bin/mpiexec

Of course we can add the folder to our PATH or create a shortcut.


**Step 7:** Install h5py:

If you don't need parallel hdf5 I/O you can just:
   
    pip install h5py

Instead, in case you need parallel hdf5 I/O and you want to compile your own hdf5 library (N.B. this is NOT needed for PyPARIS simulations):

    wget https://support.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.10.5.tar.bz2
    tar jxf hdf5-1.10.5.tar.bz2
    cd hdf5-1.10.5/
    ./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/hdf5lib --enable-parallel --enable-shared
    make
    make install
    CC=/usr/bin/mpicc HDF5_MPI="ON" HDF5_DIR=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/hdf5lib pip install --no-binary=h5py h5py

**Step 8:** At this point we are able to download and install the PyECLOUD-PyHEADTAIL suite following the steps described [here](How-to-install-PyECLOUD#install-pyecloud-and-its-friend-libraries).

A couple of references that helped with this task:

[1] [[https://www.open-mpi.org/faq/?category=building#easy-build]]

[2] [[http://stackoverflow.com/questions/5506110/is-it-possible-to-install-another-version-of-python-to-virtualenv]]

[[3]] [[https://chrisbebek.com/blog/?p=97]]