-
Notifications
You must be signed in to change notification settings - Fork 11
Setup python 3 (including mpi4py) without admin rights
The following guide shows how to get a complete python installation without administrator rights:
Step 1: We move to the folder where we want to place our installation:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3
Step 2: We download from the python website, compile and install the latest version of python 3:
mkdir python_src
cd python_src
wget https://www.python.org/ftp/python/3.8.0/Python-3.8.0.tar.xz
tar xvf Python-3.8.0.tar.xz
cd Python-3.8.0
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3
make
make install
Possible issues with ssl: In order to be able to use pip at a later stage, python needs to be compiled with ssl support. Please check that this is the case:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3
import ssl
In case this gives an exception, please install ssl library and recompile python (in Ubuntu this can be done by sudo apt install libssl-dev).
Possible issues with sqlite3: In order to be able to use jupyter notebooks at a later stage, python needs to be compiled with sqlite3 support. Please check that this is the case:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3
import sqlite3
In case this gives an exception, please install sqlite library and recompile python (in Ubuntu this can be done by sudo apt install libsqlite3-dev).
Possible issues with tk: In order to be able to use ipython --pylab at a later stage, python needs to be compiled with tk support. Please check that this is the case:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_p3/python3/bin/python3
import _tkinter
In case this gives an exception, please install tk library and recompile python (in Ubuntu this can be done by sudo apt install tklib; sudo apt install tk-dev ).
Step 3: We create a virtual environment
cd /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3
mkdir venvs
cd venvs
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/python3/bin/python3 -m venv py3
Step 4: We activate the virtual environment
source /afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/venvs/py3/bin/activate
If we type:
which python
we get:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi_py3/venvs/py3/bin/python
Step 6: We use pip to install the python modules that we need
pip install numpy scipy cython ipython matplotlib
Step 5 (option 1) - Activate MPI:
On certain machines an MPI installation can be activated using module load (it is the case for CERN lxplus and CERN HPC cluster) or mpi_selector (it is the case for the INFN-CNAF cluster)
Step 5 (option 2) - Install MPI:
If we do not have an MPI installation we need to get one:
cd /afs/cern.ch/work/g/giadarol/sim_workspace_mpi/python_src
wget https://www.open-mpi.org/software/ompi/v1.10/downloads/openmpi-1.10.2.tar.bz2
tar jxf openmpi-1.10.2.tar.bz2
cd openmpi-1.10.2
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi/openmpi
make all install
We set the environment variable for the MPI compiler (pip will use the compiler pointed by your MPICC variable to compile mpi4py):
export MPICC=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi/openmpi/bin/mpicc
At this point be careful not to have other MPI paths in your environment (for example set in your .bashrc)
Step 6: Install MPI4PY
pip install mpi4py
Important: Python jobs using mpi4py must be run with the mpiexec corresponding to the MPI compiler that has been used. In our case:
/afs/cern.ch/work/g/giadarol/sim_workspace_mpi/openmpi/bin/mpiexec
Of course we can add the folder to our PATH or create a shortcut.
Step 8: Install h5py: If you don't need parallel hdf5 I/O you can just:
pip install h5py
Instead in case you need parallel hdf5 I/O and you want to compile your own hdf5 library:
wget https://support.hdfgroup.org/ftp/HDF5/current/src/hdf5-1.10.5.tar.bz2
tar jxf hdf5-1.10.5.tar.bz2
cd hdf5-1.10.5/
./configure --prefix=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi/hdf5lib --enable-parallel --enable-shared
make
make install
CC=/usr/bin/mpicc HDF5_MPI="ON" HDF5_DIR=/afs/cern.ch/work/g/giadarol/sim_workspace_mpi/hdf5lib pip install --no-binary=h5py h5py
Step 9: At this point we are able to download and install the PyECLOUD-PyHEADTAIL environment:
git clone https://github.com/PyCOMPLETE/PyECLOUD
git clone https://github.com/PyCOMPLETE/PyKLU
git clone https://github.com/PyCOMPLETE/PyHEADTAIL
git clone https://github.com/PyCOMPLETE/PyPIC
git clone https://github.com/PyCOMPLETE/NAFFlib
git clone https://github.com/PyCOMPLETE/PyPARIS
cd PyPIC
make
cd ../PyHEADTAIL
make
cd ..
mv PyHEADTAIL PyHEADTAIL_inst
mv PyHEADTAIL_inst/PyHEADTAIL .
cd PyECLOUD
./setup_pyecloud
cd ../PyKLU
./install
cd ../NAFFlib/NAFFlib
make py2
cd ../..
mv NAFFlib NAFFlib_inst
mv NAFFlib_inst/NAFFlib .
A couple of references that helped with this task:
[1] https://www.open-mpi.org/faq/?category=building#easy-build