gitfs.pygit2 does not work on debian stable (9.x)

[tobiasBora]

Hello,

First, thank you for this formula. The problem is that it fails on my debian 9.

I guess that (one) problem is that debian 9 provides libgit2-24 instead of libgit2-22.

Thank you!

# salt '*' state.apply salt.gitfs.pygit2
pavillon:
----------
          ID: pygit2-git
    Function: pkg.installed
        Name: git
      Result: True
     Comment: All specified packages are already installed
     Started: 02:10:52.522967
    Duration: 1358.19 ms
     Changes:   
----------
          ID: libgit2-22
    Function: pkg.installed
      Result: False
     Comment: Problem encountered installing package(s). Additional info follows:
                                                                                                                                                     
              errors:                                                                                                                                
                  - Running scope as unit: run-r365609e37d4d4f3fafd2de914268826c.scope                                                               
                    E: Unable to locate package libgit2-22                                                                                           
     Started: 02:10:53.881812
    Duration: 5490.401 ms
     Changes:   
----------
          ID: install-pygit2
    Function: pip.installed
        Name: pygit2 == 0.22.1
      Result: False
     Comment: Failed to install packages: pygit2 == 0.22.1. Error: Collecting pygit2==0.22.1
                Downloading pygit2-0.22.1.tar.gz (452kB)                                                                                             
                  Complete output from command python setup.py egg_info:                                                                             
                  c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory                                                             
                   #include <ffi.h>                                                                                                                  
                                   ^                                                                                                                 
                  compilation terminated.                                                                                                            
                  Traceback (most recent call last):                                                                                                 
                    File "<string>", line 1, in <module>                                                                                             
                    File "/tmp/pip-build-zkvzYq/pygit2/setup.py", line 197, in <module>                                                              
                      cmdclass=cmdclass)                                                                                                             
                    File "/usr/lib/python2.7/distutils/core.py", line 111, in setup                                                                  
                      _setup_distribution = dist = klass(attrs)                                                                                      
                    File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 317, in __init__                                                
                      self.fetch_build_eggs(attrs['setup_requires'])                                                                                 
                    File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 372, in fetch_build_eggs                                        
                      replace_conflicting=True,                                                                                                      
                    File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 846, in resolve
                      dist = best[req.key] = env.best_match(req, ws, installer)
                    File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1118, in best_match
                      return self.obtain(req, installer)
                    File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1130, in obtain
                      return installer(requirement)
                    File "/usr/lib/python2.7/dist-packages/setuptools/dist.py", line 440, in fetch_build_egg
                      return cmd.easy_install(req)
                    File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 699, in easy_install
                      return self.install_item(spec, dist.location, tmpdir, deps)
                    File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 725, in install_item
                      dists = self.install_eggs(spec, download, tmpdir)
                    File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 906, in install_eggs
                      return self.build_and_install(setup_script, setup_base)
                    File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1145, in build_and_install
                      self.run_setup(setup_script, setup_base, args)
                    File "/usr/lib/python2.7/dist-packages/setuptools/command/easy_install.py", line 1133, in run_setup
                      raise DistutilsError("Setup script exited with %s" % (v.args[0],))
                  distutils.errors.DistutilsError: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
                  
                  ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-zkvzYq/pygit2/
     Started: 02:11:00.753589
    Duration: 15766.243 ms
     Changes:   

Summary for pavillon
------------
Succeeded: 1
Failed:    2
------------
Total states run:     3
Total run time:  22.615 s

[0xf10e]

The formula probably needs to install `libffi-dev` to get this headerfile onto the minion. That's [what the pkg containing the file is called on Ubuntu](https://packages.ubuntu.com/search?searchon=contents&keywords=ffi.h&mode=exactfilename&suite=artful&arch=any), should be the same across `G@os_family:Debian`.

[colin-stubbs]

Why's the default to install from source? When there's a package available?

[johnkeates]

Probably because the package is too old, incompatible or lacks a required feature.

[javierbertoli]

Also

     ID: libgit2-22
     Function: pkg.installed
    Result: False
   Comment: Problem encountered installing package(s). Additional info follows:
                                                                                                                                               
         errors:                                                                                                                                
             - Running scope as unit: run-r365609e37d4d4f3fafd2de914268826c.scope                                                               
               E: Unable to locate package libgit2-22         

It seems the issue is that the formula is trying to install an version of a libgit2 that does not exist anymore.

I've been using this formula on Debian 9 for a while with no issues, using the system provided packages.

It seems to me that all the 'install from source' is a remnant of other times, when the system didn't provide the required versions?

If you change your pillar to use libgit2: libgit2-24 it will probably be able to proceed. I guess this needs to be updated in the formula.

@tobiasBora , perhaps this pillar will work for you:

salt:
  install_packages: true
  libgit2: libgit2-24
  master:
    gitfs:
      git:
        install_from_package: git
      pygit2:
        install_from_source: false
        version: 0.24.2
        libgit2:
          install_from_source: false

[colin-stubbs]

@johnkeates - python-pygit2 in Debian stable is the same as on EL7 systems, 0.24.2, perfectly fine to use.

https://packages.debian.org/search?keywords=python-pygit2

[johnkeates]

@colin-stubbs Well, my message was a rather generic one, not specific to this thread ;-) If there is no reason at all, then yes, a package manager native package makes much more sense.

[mrichar1]

I believe it is still the case that libgit2 on Debian/Ubuntu is compiled without https support, due to a licensing issue with OpenSSL:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=832798
https://bugs.launchpad.net/ubuntu/+source/libgit2/+bug/1595565

This is probably the reason for building from source?

[reschl]

Has anyone tried to fix this in the formula yet? The problem still exists for Debian 9 and will also be there in Debian 10 (soon). Fixing in the pillar data rather than in the osdefaults seems quite messy to me.

[B1ue-W01f]

Issue still exists, and along with #54165 the general pygit2 setup seems somewhat unusable at least on debian until we get a bug fix. Awaiting 2018.3.5 and 2019.2.1