Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vagrant-libvirt doesn't start VM F30 #11020

Open
mbevc1 opened this issue Aug 12, 2019 · 42 comments
Open

vagrant-libvirt doesn't start VM F30 #11020

mbevc1 opened this issue Aug 12, 2019 · 42 comments

Comments

@mbevc1
Copy link

mbevc1 commented Aug 12, 2019

Vagrant version

2.2.5

Host operating system

Fedora 30

Guest operating system

CentOS 7

Debug output

/opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require': /opt/vagrant/embedded/lib64/libcrypto.so.1.1: version `OPENSSL_1_1_1b' not found (required by /lib64/libssh.so.4) - ~/.vagrant.d/gems/2.4.6/gems/ruby-libvirt-0.7.1/lib/_libvirt.so (LoadError)

Expected behavior

vagrant up server fails

Actual behavior

Should succeed.

Steps to reproduce

  1. Install vagrant-libvirt
  2. Use Fedora 30; updated - uses openssl-1.1.1c
  3. start vagrant up server
@mbevc1
Copy link
Author

mbevc1 commented Aug 12, 2019

vagrant-libvirt/vagrant-libvirt#1031 Doesn't help

@briancain
Copy link
Member

@mbevc1 - Please make sure you've installed the official installer and try again. If that doesn't work, does it work if you install the official 2.2.4 package? Additionally, does it work with other providers that are officially supported, or does this only happen with the vagrant-libvirt provider? Thanks!

@mbevc1
Copy link
Author

mbevc1 commented Aug 12, 2019

I'm using official package from Hashicorp 👍
hm, virtualbox provider seems to work okay.

@briancain
Copy link
Member

@mbevc1 - Does it work with a previously released package of Vagrant? Do you have a full debug log showing the behavior too?

@mbevc1
Copy link
Author

mbevc1 commented Aug 12, 2019

Nope, same error using 2.2.4.

 INFO box: box update check is under the interval threshold
 INFO box: Skipping box update check
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::AfterTriggerAction:0x000000000245a810>
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::BeforeTriggerAction:0x000000000245a798>
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::Call:0x000000000245a748>
 INFO runner: Preparing hooks for middleware sequence...
 INFO runner: 4 hooks defined.
 INFO runner: Running action: machine_action_up #<Vagrant::Action::Builder:0x0000000001e990c0>
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::BeforeTriggerAction:0x0000000001e15450>
 INFO warden: Calling IN action: #<VagrantPlugins::ProviderLibvirt::Action::IsCreated:0x0000000001e15428>
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::AfterTriggerAction:0x0000000001e152c0>
 INFO warden: Calling OUT action: #<Vagrant::Action::Builtin::AfterTriggerAction:0x0000000001e152c0>
 INFO warden: Calling OUT action: #<VagrantPlugins::ProviderLibvirt::Action::IsCreated:0x0000000001e15428>
 INFO warden: Calling OUT action: #<Vagrant::Action::Builtin::BeforeTriggerAction:0x0000000001e15450>
ERROR warden: Error occurred: /opt/vagrant/embedded/lib64/libcrypto.so.1.1: version `OPENSSL_1_1_1b' not found (required by /lib64/libssh.so.4) - /home/xxx/.vagrant.d/gems/2.4.6/gems/ruby-libvirt-0.7.1/lib/_libvirt.so
 INFO warden: Beginning recovery process...
 INFO warden: Calling recover: #<Vagrant::Action::Builtin::Call:0x000000000245a748>
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO warden: Beginning recovery process...
 INFO warden: Recovery complete.
 INFO environment: Released process lock: machine-action-96993d6040bfbf8204caf27e900ef00d
 INFO environment: Running hook: environment_unload
 INFO runner: Preparing hooks for middleware sequence...
 INFO runner: 3 hooks defined.
 INFO runner: Running action: environment_unload #<Vagrant::Action::Builder:0x00007f7dd0908210>
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::BeforeTriggerAction:0x000000000366a1b8>
 INFO warden: Calling IN action: #<VagrantPlugins::Triggers::Action::Trigger:0x000000000366a190>
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::AfterTriggerAction:0x00000000022e9828>
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::BeforeTriggerAction:0x00000000022e97b0>
 INFO warden: Calling IN action: #<VagrantPlugins::Triggers::Action::Trigger:0x00000000022e9788>
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::AfterTriggerAction:0x000000000223a7d8>
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::BeforeTriggerAction:0x000000000223a760>
 INFO warden: Calling IN action: #<VagrantPlugins::Triggers::Action::Trigger:0x000000000223a738>
 INFO warden: Calling IN action: #<Vagrant::Action::Builtin::AfterTriggerAction:0x000000000230bdb0>
 INFO warden: Calling OUT action: #<Vagrant::Action::Builtin::AfterTriggerAction:0x000000000230bdb0>
 INFO warden: Calling OUT action: #<VagrantPlugins::Triggers::Action::Trigger:0x000000000223a738>
 INFO warden: Calling OUT action: #<Vagrant::Action::Builtin::BeforeTriggerAction:0x000000000223a760>
 INFO warden: Calling OUT action: #<Vagrant::Action::Builtin::AfterTriggerAction:0x000000000223a7d8>
 INFO warden: Calling OUT action: #<VagrantPlugins::Triggers::Action::Trigger:0x00000000022e9788>
 INFO warden: Calling OUT action: #<Vagrant::Action::Builtin::BeforeTriggerAction:0x00000000022e97b0>
 INFO warden: Calling OUT action: #<Vagrant::Action::Builtin::AfterTriggerAction:0x00000000022e9828>
 INFO warden: Calling OUT action: #<VagrantPlugins::Triggers::Action::Trigger:0x000000000366a190>
 INFO warden: Calling OUT action: #<Vagrant::Action::Builtin::BeforeTriggerAction:0x000000000366a1b8>
/opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require': /opt/vagrant/embedded/lib64/libcrypto.so.1.1: version `OPENSSL_1_1_1b' not found (required by /lib64/libssh.so.4) - /home/xxx/.vagrant.d/gems/2.4.6/gems/ruby-libvirt-0.7.1/lib/_libvirt.so (LoadError)
	from /opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /home/xxx/.vagrant.d/gems/2.4.6/gems/ruby-libvirt-0.7.1/lib/libvirt.rb:11:in `<top (required)>'
	from /opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /home/xxx/.vagrant.d/gems/2.4.6/gems/fog-libvirt-0.6.0/lib/fog/libvirt.rb:4:in `<top (required)>'
	from /opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /home/xxx/.vagrant.d/gems/2.4.6/gems/vagrant-libvirt-0.0.45/lib/vagrant-libvirt/driver.rb:1:in `<top (required)>'
	from /opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require'
	from /home/xxx/.vagrant.d/gems/2.4.6/gems/vagrant-libvirt-0.0.45/lib/vagrant-libvirt/provider.rb:29:in `driver'
	from /home/xxx/.vagrant.d/gems/2.4.6/gems/vagrant-libvirt-0.0.45/lib/vagrant-libvirt/action/create_networks.rb:26:in `initialize'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:113:in `new'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:113:in `finalize_action'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:36:in `block in initialize'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:36:in `map'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:36:in `initialize'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builder.rb:170:in `new'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builder.rb:170:in `to_app'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/call.rb:52:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/before_trigger.rb:23:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/after_trigger.rb:26:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/box_check_outdated.rb:84:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/before_trigger.rb:23:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/after_trigger.rb:26:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/config_validate.rb:25:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/before_trigger.rb:23:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/after_trigger.rb:26:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /home/xxx/.vagrant.d/gems/2.4.6/gems/vagrant-triggers-0.5.3/lib/vagrant-triggers/action/trigger.rb:17:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/before_trigger.rb:23:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/after_trigger.rb:26:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /home/xxx/.vagrant.d/gems/2.4.6/gems/vagrant-triggers-0.5.3/lib/vagrant-triggers/action/trigger.rb:17:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/before_trigger.rb:23:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/after_trigger.rb:26:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /home/xxx/.vagrant.d/gems/2.4.6/gems/vagrant-triggers-0.5.3/lib/vagrant-triggers/action/trigger.rb:17:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builtin/before_trigger.rb:23:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/warden.rb:50:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/builder.rb:116:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/runner.rb:102:in `block in run'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/util/busy.rb:19:in `busy'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/action/runner.rb:102:in `run'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/machine.rb:238:in `action_raw'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/machine.rb:209:in `block in action'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/environment.rb:613:in `lock'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/machine.rb:195:in `call'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/machine.rb:195:in `action'
	from /opt/vagrant/embedded/gems/2.2.5/gems/vagrant-2.2.5/lib/vagrant/batch_action.rb:86:in `block (2 levels) in run'

@electrofelix
Copy link
Contributor

@briancain I've tried to look at this from the vagrant-libvirt side, and AFAICT the problem stems from needing ruby-libvirt bindings that require linking to the system installed libvirt library. It turns out on Fedora and RHEL 7, libvirt depends on libssh, and it appears that in turn this uses libssl which is provided via openssl.

As both the distro and embedded ruby in vagrant provide openssl, each time they provide different versions there is the potential for a failure to load. As long as the version on the system that is a dependency of the system libvirt library is close enough to the version bundled with vagrant, the ruby-libvirt bindings can be loaded. However once they diverge, you get an error like the above.

It appears that ubuntu/debian have not been building libvirt using libssh, which is why users of these distros have avoided hitting this issue.

One solution would be to bump the version of openssl bundled with the ruby env in the vagrant packages, however I'm not sure if that would cause other conflicts elsewhere. The other is for fedora users to always have to use the distribution provided package of vagrant as that would ensure consistency between the libraries.

Outside of these, there starts to be some questions about whether it's necessary to bundle the entire libvirt client libraries and dependencies with vagrant-libvirt for some distros or rewrite it to use the client command line rather than the API via fog-libvirt.

@mbevc1
Copy link
Author

mbevc1 commented Aug 13, 2019

Thanks for you feedback @electrofelix 👍
@briancain is there anything from your side we can do to rectify this on CentOS/Fedora systems?
Seems like quick fix would be like @electrofelix suggested and rebuild with latest libs, and maybe on the long run try to bundle it without those deps?

@JamesReynolds
Copy link

You can fix this by building libssh.so.* from source using the /opt/vagrant/embedded as the openssl root then installing it into /opt/vagrant/embedded/lib64:

dnf download --source libssh
rpm2cpio libssh-0.9.0-5.fc30.src.rpm | cpio -imdV
tar xf libssh-0.9.0.tar.xz
mkdir build
cd build
cmake ../libssh-0.9.0 -DOPENSSL_ROOT_DIR=/opt/vagrant/embedded/
make
cp lib/libssh* /opt/vagrant/embedded/lib64

that got it working for me. When the libvirt provider is run it picks up libssh.so.4 from the embedded libraries which are now linked to openssl from the same place.

@guits
Copy link

guits commented Feb 25, 2020

@JamesReynolds thanks, that works like charm in fedora 31!

@rmb938
Copy link

rmb938 commented May 4, 2020

The fix mentioned above doesn't work on fedora 32, it can't find the source for libssh

dnf download --source libssh 

Last metadata expiration check: 0:06:01 ago on Mon 04 May 2020 06:17:03 PM CDT.
No package libssh-0.9.4-2.fc32.src available.
Exiting due to strict setting.
Error: No package libssh-0.9.4-2.fc32.src available.

@mbevc1
Copy link
Author

mbevc1 commented May 8, 2020

Worked perfectly on F32 here 🎉 . Thanks @JamesReynolds !
I needed to use same approach for missing /usr/lib64/libk5crypto.so.3 as well:

dnf download --source krb5-libs
rpm2cpio krb5-1.18-1.fc32.src.rpm | cpio -imdV
tar xf krb5-1.18.tar.gz
cd krb5-1.18/src
./configure
make
cp -a lib/crypto/libk5crypto.* /opt/vagrant/embedded/lib64/

That will ensure to pick those pre-compiled libs in 👍

@buckaroogeek
Copy link

@mbevc1 - For some reason I still get the libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL error when running vagrant with libvirt. Is there a step in your instructions to link the kerberos lib compile to the openssl used by vagrant?

@mbevc1
Copy link
Author

mbevc1 commented May 8, 2020

Pre-compilation is dome before building native extensions for the Vagrnat plugin. Then during that it should pick up libs from embedded folder 🤔

@buckaroogeek
Copy link

buckaroogeek commented May 8, 2020

@mbevc1 - thanks Marko. For some reason I was still getting get this error:" /usr/lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b" which indicates to me that the libk5crypto files in /opt/vagrant/embedded/lib64 are being ignored. But I did a fresh install of vagrant 2.2.9, copied both the ssh and libk5crypto libraries as noted earlier in this thread and 'vagrant plugin install vagrant-libvirt' worked cleanly and the plugin also is running correctly. I do appreciate your help!

@electrofelix
Copy link
Contributor

dnf download --source krb5-libs
rpm2cpio krb5-1.18-1.fc32.src.rpm | cpio -imdV
tar xf krb5-1.18.tar.gz
cd krb5-1.18/src
./configure
make
cp -a lib/crypto/libk5crypto.* /opt/vagrant/embedded/lib64/

Is there something extra getting built or copied over with this approach compared to the system library? Just wondering if you could simply install the krb5-libs package and copy over the library files instead of recompiling them?

@mbevc1
Copy link
Author

mbevc1 commented May 10, 2020

I think you could also pick those libs from the system and copy it to embedded folder. I have that package installed and I reckon it's not picking up default lib path on Fedora 🤔

@xyloplax
Copy link

I can confirm this works on Fedora 32 with the official RPM (not the one in the Fedora repos) once the steps tlisted above to manually build and copy libssh and the libk5 into /opt/vagrant are performed. Thank you!

@rfm83
Copy link

rfm83 commented May 17, 2020

Is anybody having issues running "make"? I get this "yacc: Command not found" error.
btw, I'm on Fedora 32 with the latest vagrant 2.2.9.

yacc getdate.y
make[2]: yacc: Command not found
make[2]: *** [: getdate.c] Error 127
make[2]: Leaving directory '/home/fabio/sandbox/krb5/krb5-1.18/src/kadmin/cli'
make[1]: *** [Makefile:865: all-recurse] Error 1
make[1]: Leaving directory '/home/fabio/sandbox/krb5/krb5-1.18/src/kadmin'
make: *** [Makefile:1546: all-recurse] Error 1

@snagoor
Copy link

snagoor commented May 17, 2020

Install byacc RPM

 # sudo dnf install byacc

@rfm83
Copy link

rfm83 commented May 17, 2020

@snagoor. Thanks, I was able to get past the yacc error after installing byacc.
However, I still have the issue with running vagrant 2.2.9 with the following error:

_opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': /usr/lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b - /home/fabio/.vagrant.d/gems/2.6.6/gems/ruby-libvirt-0.7.1/lib/libvirt.so (LoadError)

I have tried re-installing vagrant and the libvirt plugin, but nothing seems to help.

@buckaroogeek
Copy link

@rochafab - did you also try the steps @mbevc1 outlined in #11020 (comment) above?

dnf download --source krb5-libs
rpm2cpio krb5-1.18-1.fc32.src.rpm | cpio -imdV
tar xf krb5-1.18.tar.gz
cd krb5-1.18/src
./configure
make
cp -a lib/crypto/libk5crypto.* /opt/vagrant/embedded/lib64/

And then install vagrant-libvirt plugin?

@rfm83
Copy link

rfm83 commented May 17, 2020

@buckaroogeek - Yes, I did install the krb5-libs and then the vagrant-libvirt plugin as mentioned above, but I still get that error.
I was successfully running vagrant and libvirt with the distro version 2.2.6 and libvirt 0.0.45, but I had issues trying to assign static IPs to the vms. Somebody mentioned downgrading to libvirt 0.0.40 fixes that issue, but I was unsuccessful. Then I decided to pick up the latest upstream vagrant version 2.2.9 and libvirt 0.1.2, but now I get this libk5crypto.so.3: error and can't even run vagrant.

@snagoor
Copy link

snagoor commented May 18, 2020

@rochafab, I was having the same issue you mentioned on Fedora Workstation 32. Tried all the steps mentioned here. But none worked.

So, finally I tried this vagrant-libvirt/vagrant-libvirt#1031 (comment) and it worked.

It throws few deprecated warnings, which is still fine for me until I'm able to bring up some hosts using vagrant.

@rfm83
Copy link

rfm83 commented May 18, 2020

@snagoor - which part specifically did you try that worked?

@snagoor
Copy link

snagoor commented May 18, 2020

Remove HashiCorp package:

# sudo dnf remove vagrant

Install Fedora vagrant package:

# sudo dnf install vagrant

Remove vagrant-libvirt package from Fedora:

# sudo dnf remove vagrant-libvirt
# sudo dnf remove rubygem-fog-core (conflict with plugin building)

Build the upstream vagrant-libvirt:

# vagrant plugin install vagrant-libvirt

and then

# vagrant up 

@jirihnidek
Copy link

I have same problem with vagrant-libvirt plugin

[jhnidek@localhost candlepin]$ vagrant status
/opt/vagrant/embedded/lib/ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:in `require': /usr/lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b - /home/jhnidek/.vagrant.d/gems/2.4.9/gems/ruby-libvirt-0.7.1/lib/_libvirt.so (LoadError)

I suspect that problem is with ruby-libvirt gem, because it is not compiled, but it is simply (including _libvirt.so) downloaded as you can see here:

[jhnidek@localhost candlepin]$ vagrant plugin install vagrant-libvirt
Installing the 'vagrant-libvirt' plugin. This can take a few minutes...
Fetching: excon-0.73.0.gem (100%)
Fetching: formatador-0.2.5.gem (100%)
Fetching: fog-core-2.2.0.gem (100%)
Fetching: fog-json-1.2.0.gem (100%)
Fetching: mini_portile2-2.4.0.gem (100%)
Fetching: nokogiri-1.10.9.gem (100%)
Building native extensions.  This could take a while...
Fetching: fog-xml-0.1.3.gem (100%)
Fetching: ruby-libvirt-0.7.1.gem (100%)
Building native extensions.  This could take a while...
Fetching: fog-libvirt-0.7.0.gem (100%)
Fetching: vagrant-libvirt-0.1.2.gem (100%)
Installed the plugin 'vagrant-libvirt (0.1.2)'!

BTW: setting environment variable LD_LIBRARY_PATH to /opt/vagrant/embedded/lib64 didn't solve this issue either.

@electrofelix
Copy link
Contributor

@jirihnidek can you try the instructions mentioned in #11020 (comment).

ruby-libvirt is compiling, the file /home/jhnidek/.vagrant.d/gems/2.4.9/gems/ruby-libvirt-0.7.1/lib/_libvirt.so is what it compiles against the system libvirt and the ruby embedded with vagrant.

/home/jhnidek/.vagrant.d/gems/2.4.9/gems/ruby-libvirt-0.7.1/lib/_libvirt.so is loading likely loading openssl, but it's the version from under /opt/vagrant/embedded/lib64, and it is also loading to load /usr/lib64/libk5crypto.so.3 which requires some symbols that are not present in the version bundled with vagrant. If you build the libssh package and add the resulting files to the vagrant lib directory, it should allow everything to load correctly.

I will try to get to merging the correct instructions for vagrant-libvirt on Fedora 30+ along with alternative suggestion to consider using the distro bundled version to avoid the issues with libraries mismatching.

@buckaroogeek
Copy link

If you are switching back and forth between upstream vagrant and the fedora package, it might be helpful to remove or rename ~/.vagrant.d after a 'dnf erase vagrant' is executed.

@rfm83
Copy link

rfm83 commented May 26, 2020

@jirihnidek - I finally was able to work around this problem and I believe what fixed the issue in the end was to install both the libssh and krb5-libs from source. Just make sure you match the libssh version accordingly so you don't get errors.

dnf download --source libssh
rpm2cpio libssh-0.9.0-5.fc30.src.rpm | cpio -imdV
tar xf libssh-0.9.0.tar.xz
mkdir build
cd build
cmake ../libssh-0.9.0 -DOPENSSL_ROOT_DIR=/opt/vagrant/embedded/
make
cp lib/libssh* /opt/vagrant/embedded/lib64


dnf download --source krb5-libs
rpm2cpio krb5-1.18-1.fc32.src.rpm | cpio -imdV
tar xf krb5-1.18.tar.gz
cd krb5-1.18/src
./configure
make
cp -a lib/crypto/libk5crypto.* /opt/vagrant/embedded/lib64/

@EmmanuelKasper
Copy link

EmmanuelKasper commented Jun 21, 2020

On a RHEL8 systems building libssh using the steps above caused me the following error:

[ 60%] Linking CXX executable libsshpp_noexcept
/usr/lib64/libk5crypto.so: undefined reference to `EVP_KDF_derive@OPENSSL_1_1_1b'
/usr/lib64/libk5crypto.so: undefined reference to `EVP_KDF_ctrl@OPENSSL_1_1_1b'

to build libssh successfully, I disabled the GSSAPI / Kerberos support in libssh since I don't plan to use it.

Before executing cmake from the commands above:

sed -i 's/WITH_GSSAPI "Build with GSSAPI support" ON/WITH_GSSAPI "Build with GSSAPI support" OFF/' ../libssh-0.9.0/DefineOptions.cmake

After copying the libraries, I had to build the plugin with the following command, to make sure the embedded libraries where picked:

CONFIGURE_ARGS='with-ldflags=-L/opt/vagrant/embedded/lib with-libvirt-include=/usr/include/libvirt with-libvirt-lib=/usr/lib'  vagrant plugin install vagrant-libvirt

And no more run time error :

vagrant status 
Current machine states:
default                   running (libvirt)

I suppose everybody in this thread now understand the value of Linux distributions doing that tedious packaging work :)

@voxik
Copy link
Contributor

voxik commented Aug 18, 2020

So would it be the solution to include libssh with disabled GSSAPI support in Vagrant? Anybody care to submit PR?

@squarebracket
Copy link

I'm unable to get it working on F32 as of 2020-08-27. Can someone confirm the following steps are what's needed?

dnf download --source libssh
rpm2cpio libssh-*.src.rpm | cpio -imdV
tar xf libssh-*.tar.xz
mkdir build
cd build
cmake ../libssh-* -DOPENSSL_ROOT_DIR=/opt/vagrant/embedded/
make
sudo cp lib/libssh* /opt/vagrant/embedded/lib64
  • Build krb5-libs without fedora patches and copy to vagrant folder
dnf download --source krb5-libs
rpm2cpio krb5-*.src.rpm | cpio -imdV
tar xf krb5-*.tar.gz
cd krb5-*/src
./configure
make
sudo cp -a lib/crypto/libk5crypto.* /opt/vagrant/embedded/lib64/
  • Build plugin with configure arguments telling vagrant where to look for libvirt headers/libs:
CONFIGURE_ARGS="with-libvirt-include=/usr/include/libvirt with-libvirt-lib=/usr/lib64" \
  vagrant plugin install vagrant-libvirt

rhjhunt added a commit to rhjhunt/vagrant-container that referenced this issue Sep 22, 2020
The Vagrant package provided by Fedora doesn't support the Ruby library for WinRM that is need by Vagrant for talking to Windows.

Using the upstream version of Vagrant had a few issues that are reported by these bugs:
vagrant-libvirt/vagrant-libvirt#1127
hashicorp/vagrant#11020

I've added a workaround in the Dockerfile to address them.
ssbarnea added a commit to ansible-community/molecule-vagrant that referenced this issue Oct 22, 2020
Due to lack of support from Vagrant for CentOS 8 we disable testing
on that platform and document it. Some power users may be able to
get it going but the level of hacking involved makes not suitable
for any production environments.

hashicorp/vagrant#11020
vagrant-libvirt/vagrant-libvirt#1127
ssbarnea added a commit to ansible-community/molecule-vagrant that referenced this issue Oct 22, 2020
Due to lack of support from Vagrant for CentOS 8 we disable testing
on that platform and document it. Some power users may be able to
get it going but the level of hacking involved makes not suitable
for any production environments.

hashicorp/vagrant#11020
vagrant-libvirt/vagrant-libvirt#1127
ssbarnea added a commit to ansible-community/molecule-vagrant that referenced this issue Oct 22, 2020
Due to lack of support from Vagrant for CentOS 8 we disable testing
on that platform and document it. Some power users may be able to
get it going but the level of hacking involved makes not suitable
for any production environments.

hashicorp/vagrant#11020
vagrant-libvirt/vagrant-libvirt#1127
@ericzolf
Copy link

ericzolf commented May 6, 2021

For what it's worth, under F34, I got it working "only" with CONFIGURE_ARGS="with-libvirt-include=/usr/include/libvirt with-libvirt-lib=/usr/lib64" vagrant plugin install vagrant-libvirt, I didn't need to do any of the source compiling and copying steps.

@ericzolf
Copy link

ericzolf commented May 6, 2021

OK, I wrote too fast, it fails at runtime, not at build time:

$ vagrant status
/opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require': /usr/lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b - /home/myuser/.vagrant.d/gems/2.6.7/gems/ruby-libvirt-0.7.1/lib/_libvirt.so (LoadError)
	from /opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	from /home/myuser/.vagrant.d/gems/2.6.7/gems/ruby-libvirt-0.7.1/lib/libvirt.rb:11:in `<top (required)>'
	from /opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	from /opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	from /home/myuser/.vagrant.d/gems/2.6.7/gems/fog-libvirt-0.8.0/lib/fog/libvirt.rb:4:in `<top (required)>'
	from /opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	from /opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	from /home/myuser/.vagrant.d/gems/2.6.7/gems/vagrant-libvirt-0.4.1/lib/vagrant-libvirt/driver.rb:1:in `<top (required)>'
	from /opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	from /opt/vagrant/embedded/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
	from /home/myuser/.vagrant.d/gems/2.6.7/gems/vagrant-libvirt-0.4.1/lib/vagrant-libvirt/provider.rb:29:in `driver'
	from /home/myuser/.vagrant.d/gems/2.6.7/gems/vagrant-libvirt-0.4.1/lib/vagrant-libvirt/provider.rb:94:in `state'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/machine.rb:541:in `state'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/machine.rb:154:in `initialize'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/vagrantfile.rb:81:in `new'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/vagrantfile.rb:81:in `machine'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/environment.rb:716:in `machine'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/plugin/v2/command.rb:180:in `block in with_target_vms'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/plugin/v2/command.rb:213:in `block in with_target_vms'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/plugin/v2/command.rb:212:in `map'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/plugin/v2/command.rb:212:in `with_target_vms'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/plugins/commands/status/command.rb:20:in `execute'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/cli.rb:67:in `execute'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/lib/vagrant/environment.rb:290:in `cli'
	from /opt/vagrant/embedded/gems/2.2.16/gems/vagrant-2.2.16/bin/vagrant:231:in `<main>'

@husky-parul
Copy link

I'm unable to get it working on F32 as of 2020-08-27. Can someone confirm the following steps are what's needed?

dnf download --source libssh
rpm2cpio libssh-*.src.rpm | cpio -imdV
tar xf libssh-*.tar.xz
mkdir build
cd build
cmake ../libssh-* -DOPENSSL_ROOT_DIR=/opt/vagrant/embedded/
make
sudo cp lib/libssh* /opt/vagrant/embedded/lib64
  • Build krb5-libs without fedora patches and copy to vagrant folder
dnf download --source krb5-libs
rpm2cpio krb5-*.src.rpm | cpio -imdV
tar xf krb5-*.tar.gz
cd krb5-*/src
./configure
make
sudo cp -a lib/crypto/libk5crypto.* /opt/vagrant/embedded/lib64/
  • Build plugin with configure arguments telling vagrant where to look for libvirt headers/libs:
CONFIGURE_ARGS="with-libvirt-include=/usr/include/libvirt with-libvirt-lib=/usr/lib64" \
  vagrant plugin install vagrant-libvirt

Were you able to get this up? I am on Fedora 33 and still facing this issue even after doing all the steps

/opt/vagrant/embedded/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:83:in `require': /usr/lib64/libk5crypto.so.3: undefined symbol: EVP_KDF_ctrl, version OPENSSL_1_1_1b - /home/parsingh/.vagrant.d/gems/2.7.4/gems/ruby-libvirt-0.7.1/lib/_libvirt.so (LoadError)

@mikerosile
Copy link

I can confirm the following worked for me /w Fedora 34.
I installed vagrant v2.2.18 from Hasicorp's repo.

dnf download --source libssh
rpm2cpio libssh-0.9.6-1.fc34.src.rpm | cpio -imdV
tar xf libssh-0.9.6.tar.xz
mkdir build
cd build
cmake ../libssh-0.9.6 -DOPENSSL_ROOT_DIR=/opt/vagrant/embedded/
make
sudo cp lib/libssh* /opt/vagrant/embedded/lib64

dnf download --source krb5-libs
rpm2cpio krb5-1.19.2-2.fc34.src.rpm | cpio -imdV
tar xf krb5-1.19.2.tar.gz
cd krb5-1.19.2/src
./configure
make
sudo cp -P lib/crypto/libk5crypto.* /opt/vagrant/embedded/lib64/

vagrant plugin install vagrant-libvirt

@electrofelix
Copy link
Contributor

For anyone else bumping into this issue, there are a small number of paths that have been worked out for using vagrant-libvirt to reduce the risk of bumping into the conflict between the distro libraries and vagrant embedded libraries on Fedora.

  1. Using the distro packaged vagrant and distro packaged dependencies of vagrant-libvirt along with the latest vagrant-libvirt gem
    # following command should only be needed if you've already installed vagrant via the upstream repo
    sudo dnf config-manager --set-disabled hashicorp
    # the following command might not be needed if dnf automatically replaces it with the distro one below
    # but since it might be a newer version I'm not sure what it'll actually do
    sudo dnf remove vagrant
    
    sudo dnf update
    sudo dnf install @virtualization vagrant-libvirt
    sudo systemctl enable --now libvirtd
    sudo dnf remove --noautoremove vagrant-libvirt
    vagrant plugin install vagrant-libvirt
    
    After this if you want to use the plugin exactly as this Fedora has configured the packaged version of vagrant-libvirt, make sure to set the following:
    export LIBVIRT_DEFAULT_URI=qemu:///session
    
    Otherwise if you want to use the system context to launch VMs, you can enable that by doing the following
    sudo usermod -a -G libvirt vagrant
    exec sg libvirt newgrp `id -gn`
    
    The exec is a hack to make your current terminal session switch groups, otherwise just log out and back in and your groups should be updated.
  2. Use docker/podman container image following the instructions at https://github.com/vagrant-libvirt/vagrant-libvirt#using-the-container-image
  3. Using upstream vagrant (this repo) and patching the libraries bundled with the embedded ruby to allow correct compiling
    For this https://github.com/vagrant-libvirt/vagrant-libvirt-qa/blob/master/scripts/install.bash which codifies the extra steps needed as described under https://github.com/vagrant-libvirt/vagrant-libvirt#additional-notes-for-fedora-and-similar-linux-distributions in a manner that should work across multiple versions of the libraries.
    But keep in mind, this is still replacing libraries that have been built and bundled with vagrant, so replacing them may cause other issues that will break your vagrant install.

My recommendation is to follow option 1 for Fedora. It's not an option for RHEL/CentOS, so for those you'll have decide which of the remaining two. I do hope to enable some automatic checking on the vagrant-libvirt-qa so that it'll regularly test that the install of the latest upstream vagrant and tip of the mainline branch of this project remain installable, but I'd consider it a fall back solution.

I'll try to ensure all of this is added to the README in vagrant-libvirt soon so it reduces the chance of people bumping into this.

@buckaroogeek
Copy link

@electrofelix - thank you for the outstanding vagrant-libvirt support and work.

@electrofelix
Copy link
Contributor

Small correction to my comment above about installing on Fedora

If you have already installed vagrant-libvirt from the package you'll want to run the following to prevent future removal of dependencies:

dependencies=$(
    sudo dnf repoquery --qf "%{name}" $(
        for dep in $(sudo dnf repoquery --depends vagrant-libvirt 2>/dev/null | cut -d' ' -f1)
        do
            echo "--whatprovides ${dep} "
        done
     ) 2>/dev/null
)

sudo dnf mark install ${dependencies}

Or if installing from pristine:

sudo dnf update
dependencies=$(
    sudo dnf repoquery --qf "%{name}" $(
        for dep in $(sudo dnf repoquery --depends vagrant-libvirt 2>/dev/null | cut -d' ' -f1)
        do
            echo "--whatprovides ${dep} "
        done
     ) 2>/dev/null
)
sudo dnf install @virtualization ${dependencies}
sudo systemctl enable --now libvirtd
vagrant plugin install vagrant-libvirt

That will prevent any of the distro packages that vagrant installed plugin vagrant-libvirt depends on and are being used to satisfy the linking to the libvirt library from being accidentally removed in the future.

@ghost
Copy link

ghost commented Feb 22, 2022

@electrofelix in your comment here, are these commands typo's?

sudo dnf install @virtualization vagrant-libvirt
sudo dnf remove --noautoremove vagrant-libvirt

I assume what you meant was this?

sudo dnf install @virtualization vagrant
sudo dnf remove --noautoremove vagrant

If not, then aren't you missing the step to re-install vagrant?

@electrofelix
Copy link
Contributor

@troyhamilton-cloud the intent of those commands is to have all the dependencies of vagrant-libvirt installed but not the distro packaged vagrant-libvirt. Vagrant is a dependency so it will have been installed by the command

sudo dnf install @virtualization vagrant-libvirt 

The following command will only uninstall vagrant-libvirt while keeping everything else that was installed due to it on the machine

sudo dnf remove --noautoremove vagrant-libvirt

So that will keep the likes of vagrant, fog-libvirt and ruby-libvirt, the latter being the one that causes issues about being run with the upstream vagrant embedded while also needing to be compiled against the system libraries to provide the bindings to libvirt.

However this causes an issue in that just because the dependencies were not removed, doesn't mean they will be kept after future dnf commands, so it becomes necessary to run the following scriptlet as well

dependencies=$(
    sudo dnf repoquery --qf "%{name}" $(
        for dep in $(sudo dnf repoquery --depends vagrant-libvirt 2>/dev/null | cut -d' ' -f1)
        do
            echo "--whatprovides ${dep} "
        done
     ) 2>/dev/null
)

sudo dnf mark install ${dependencies}

Hence the subsequent recommendation that anyone wanting this should follow this:

sudo dnf update
dependencies=$(
    sudo dnf repoquery --qf "%{name}" $(
        for dep in $(sudo dnf repoquery --depends vagrant-libvirt 2>/dev/null | cut -d' ' -f1)
        do
            echo "--whatprovides ${dep} "
        done
     ) 2>/dev/null
)
sudo dnf install @virtualization ${dependencies}
sudo systemctl enable --now libvirtd
vagrant plugin install vagrant-libvirt

josefbacik added a commit to josefbacik/kdevops that referenced this issue Mar 17, 2022
The pre-built vagrant binaries don't work properly with building the
latest vagrant-libvirt plugin, you get weird symbol errors as described
here

hashicorp/vagrant#11020 (comment)

However the distro packages work just fine, so install the distro
packages and carry on.

Signed-off-by: Josef Bacik <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests