problem with python-snappy installation on a virtualenv

[aurelBZH]

I tried to install python-snappy on a virtualenv with python 2.7. It failed

` x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c snappy/snappymodule.cc -o build/temp.linux-x86_64-2.7/snappy/snappymodule.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
snappy/snappymodule.cc:31:22: fatal error: snappy-c.h: Aucun fichier ou dossier de ce type
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

---

Failed building wheel for python-snappy
Running setup.py clean for python-snappy
Successfully built cyvcf2 peddy
Failed to build python-snappy
Installing collected packages: sqlalchemy, numpy, cyvcf2, peddy, geneimpacts, python-snappy
Running setup.py install for geneimpacts ... done
Running setup.py install for python-snappy ... error
Complete output from command /home/aurelien.beliard/.virtualenvs/varapp/bin/python -u -c "import setuptools, tokenize;file='/tmp/pip-build-1c10wm/python-snappy/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-CzAHUl-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/aurelien.beliard/.virtualenvs/varapp/include/site/python2.7/python-snappy:
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'cffi_modules'
warnings.warn(msg)
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/snappy
copying snappy/init.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/snappy_cffi.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/snappy.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/snappy_cffi_builder.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/main.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/snappy_formats.py -> build/lib.linux-x86_64-2.7/snappy
copying snappy/hadoop_snappy.py -> build/lib.linux-x86_64-2.7/snappy
running build_ext
building 'snappy._snappy' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/snappy
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c snappy/snappymodule.cc -o build/temp.linux-x86_64-2.7/snappy/snappymodule.o
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
snappy/snappymodule.cc:31:22: fatal error: snappy-c.h: Aucun fichier ou dossier de ce type
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
`

I tried to reinstall libsnappy-dev to correct it, but i have another problem
I tried to install cffi but there already is a problem:

static struct module_state _state;
                              ^
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c snappy/crc32c.c -o build/temp.linux-x86_64-2.7/snappy/crc32c.o
  c++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-Bsymbolic-functions -Wl,-z,relro -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/snappy/snappymodule.o build/temp.linux-x86_64-2.7/snappy/crc32c.o -lsnappy -o build/lib.linux-x86_64-2.7/snappy/_snappy.so
  c++: error: unrecognized command line option ‘-Wdate-time’
  c++: error: unrecognized command line option ‘-fstack-protector-strong’
  c++: error: unrecognized command line option ‘-Wdate-time’
  c++: error: unrecognized command line option ‘-fstack-protector-strong’
  error: command 'c++' failed with exit status 1

[martindurant]

I'm afraid the error above doesn't mean much to me. The "unrecognized command line option" might mean an old gcc version?

May I suggest that the easy way to get around such issues is to install using conda rather than virtualenv.

[aurelBZH]

I use both. In fact the strange thing is that it work perfectly outside the virtualenv

[martindurant]

Perhaps check the toolchain available in your PATH within and without the virtualenv? I am shooting in the dark...

[aurelBZH]

well outside the virtual env i also have a problem. The install goes well but when i try
python -m snappy --help
i have this error
from _snappy import CompressError, CompressedLengthError, \ ImportError: /home/aurelien.beliard/miniconda3/lib/python2.7/site-packages/../../libstdc++.so.6: versionCXXABI_1.3.9' not found (required by /home/aurelien.beliard/miniconda3/lib/python2.7/site-packages/../../libsnappy.so.1)`

[cclauss]

Can not install snappy on Travis Continuous Integration even with a simple .travis.yml:

language: python
addons:
  apt:
    packages:
    - gcc
install:
  - pip install python-snappy
script:
  - true

Output is https://travis-ci.org/cclauss/snappy-hack/builds/356157907

[martindurant]

@cclauss , this is not surprising, the build requires a full tool chain and the snappy dev libraries. Compare with the conda recipe

[cclauss]

Perfect! The following works as expected:

language: python
addons:
  apt:
    packages:
    - libsnappy-dev
install:
  - pip install python-snappy
script:
  - true

Thanks!

[fireundubh]

Just struggled with this for a bit.

Solution

1. Download the Snappy source from: https://github.com/google/snappy/releases
2. Extract the ZIP or tarball to D:\dev\snappy-1.1.8 (example used below) or wherever you want.

Manual:

3. Download the python-snappy source from: https://github.com/andrix/python-snappy/archive/master.zip
4. Extract the ZIP to wherever you want.
5. Run Developer Command Prompt for VS 2019.
6. Navigate to that folder in the virtualenv/pipenv shell.
7. In the virtualenv/pipenv shell, run: python setup.py install build_ext -n -ID:\dev\snappy-1.1.8

Pip

In the virtualenv/pipenv shell (from Developer Command Prompt for VS 2019), you should be able to run:

3. pip install python-snappy --global-option=build_ext --global-option=-ID:\dev\snappy-1.1.8

But I'm unable to get this method to work. It should be possible though.