-
Notifications
You must be signed in to change notification settings - Fork 11
Setup python 3 (including mpi4py) without admin rights
The following guide shows how to get a complete python installation without administrator rights:
Step 1: We move to the folder where we want to place our installation:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3
Step 2: We download from the python website, compile and install the latest version of python 3:
mkdir python_src
cd python_src
wget https://www.python.org/ftp/python/3.8.0/Python-3.8.0.tar.xz
tar xvf Python-3.8.0.tar.xz
cd Python-3.8.0
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3
make
make install
Possible issues with ssl: In order to be able to use pip at a later stage, python needs to be compiled with ssl support. Please check that this is the case:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3
import ssl
In case this gives an exception, please install ssl library and recompile python (in Ubuntu this can be done by sudo apt install libssl-dev).
Possible issue with ctypes and libffi: Check that ctypes is working (will be required for several reasons afterwards):
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3
import ctypes
If you get an error, it is because the ffi library is not installed. You can download and install the library as follows:
cd /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python_src
wget ftp://sourceware.org/pub/libffi/libffi-3.3.tar.gz
tar -xvf libffi-3.3.tar.gz
cd libffi-3.3/
mkdir /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/libffi
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/libffi
make
make install
The library location needs to be added to our library path:
export LD_LIBRARY_PATH=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/libffi/lib64:$LD_LIBRARY_PATH
The python compilation needs some extra information:
export PKG_CONFIG_PATH=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/libffi/lib/pkgconfig
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3 LDFLAGS="-L/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/libffi/lib/../lib64" CFLAGS="-I/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/libffi/include"
The continue as normal:
make
make install
Possible issues with sqlite3: In order to be able to use jupyter notebooks at a later stage, python needs to be compiled with sqlite3 support. Please check that this is the case:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3
import sqlite3
In case this gives an exception, please install sqlite library and recompile python (in Ubuntu this can be done by sudo apt install libsqlite3-dev).
Possible issues with tk: In order to be able to use ipython --pylab at a later stage, python needs to be compiled with tk support. Please check that this is the case:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_p3/python3/bin/python3
import _tkinter
In case this gives an exception, please install tk library and recompile python (in Ubuntu this can be done by sudo apt install tklib; sudo apt install tk-dev ).
Step 3: We create a virtual environment
cd /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3
mkdir venvs
cd venvs
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3 -m venv py3
Step 4: We activate the virtual environment
source /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/venvs/py3/bin/activate
If we type:
which python
we get:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/venvs/py3/bin/python
Step 6: We use pip to install the python modules that we need
pip install numpy scipy cython ipython matplotlib
Steps 5 and 6 are required only to run parallel simulations using MPI.
Step 5 - Check that MPI is available:
You can use the following commands to check that an MPI installation is available
which mpicc
which mpiexec
In case it is not, please activate an MPI installation (Step 5.a) or install a new MPI installation (Step 5.b).
Step 5.a - Activate MPI:
On certain machines an MPI installation can be activated using module load (it is the case for CERN lxplus and CERN HPC cluster) or mpi_selector (it is the case for the INFN-CNAF cluster)
Step 5.b - Install MPI:
If we do not have an MPI installation we need to get one:
cd /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python_src
wget https://www.open-mpi.org/software/ompi/v1.10/downloads/openmpi-1.10.2.tar.bz2
tar jxf openmpi-1.10.2.tar.bz2
cd openmpi-1.10.2
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/openmpi
make all install
We set the environment variable for the MPI compiler (pip will use the compiler pointed by your MPICC variable to compile mpi4py):
export MPICC=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/openmpi/bin/mpicc
At this point be careful not to have other MPI paths in your environment (for example set in your .bashrc)
Step 6: Install mpi4py
pip install mpi4py
Important: Python jobs using mpi4py must be run with the mpiexec corresponding to the MPI compiler that has been used. In our case:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/openmpi/bin/mpiexec
Of course we can add the folder to our PATH or create a shortcut.
Step 7: Install h5py:
If you don't need parallel hdf5 I/O you can just:
pip install h5py
Instead, in case you need parallel hdf5 I/O and you want to compile your own hdf5 library (N.B. this is NOT needed for PyPARIS simulations):
wget https://support.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.10.5.tar.bz2
tar jxf hdf5-1.10.5.tar.bz2
cd hdf5-1.10.5/
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/hdf5lib --enable-parallel --enable-shared
make
make install
CC=/usr/bin/mpicc HDF5_MPI="ON" HDF5_DIR=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/hdf5lib pip install --no-binary=h5py h5py
In case you don't have libhdf5 installed, but you don't need parallel IO (PyPARIS case), you can:
wget https://support.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.10.5.tar.bz2
tar jxf hdf5-1.10.5.tar.bz2
cd hdf5-1.10.5/
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/hdf5lib --enable-shared
make
make install
HDF5_DIR=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/hdf5lib pip install --no-binary=h5py h5py
Step 8: At this point we are able to download and install the PyECLOUD-PyHEADTAIL suite following the steps described here.
A couple of references that helped with this task:
[1] https://www.open-mpi.org/faq/?category=building#easy-build