Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only add katello as a dependency if not already declared #202

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ekohl
Copy link
Member

@ekohl ekohl commented Sep 13, 2024

This inspects the known dependencies and only adds katello if it's not already defined. This is needed because in development it's common to have local git checkouts, but then still load all gemfile.d/*.rb files from the various plugins.

This is an alternative to #201. It's currently untested since I don't have an env set up. I suspect it depends on the order they're defined and that makes it unreliable.

This inspects the known dependencies and only adds katello if it's not
already defined. This is needed because in development it's common to
have local git checkouts, but then still load all gemfile.d/*.rb files
from the various plugins.
@chris1984
Copy link
Member

This inspects the known dependencies and only adds katello if it's not already defined. This is needed because in development it's common to have local git checkouts, but then still load all gemfile.d/*.rb files from the various plugins.

This is an alternative to #201. It's currently untested since I don't have an env set up. I suspect it depends on the order they're defined and that makes it unreliable.

@ekohl is the best way to test this, to merge it and then spin up a new devel box with forklift and see if we get past the error, or is there a better way to test this?

@ekohl
Copy link
Member Author

ekohl commented Sep 16, 2024

I'd think you can check out the PR (the GH cli can make this very easy) and use bundle update to see if it still breaks.

@chris1984
Copy link
Member

I'd think you can check out the PR (the GH cli can make this very easy) and use bundle update to see if it still breaks.

We got further, bundle_update gives this now:

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /tmp/bundler20240917-59316-e9worhpsych-5.1.2/gems/psych-5.1.2/ext/psych
/usr/bin/ruby -I /usr/share/rubygems -r ./siteconf20240917-59316-60jxpz.rb extconf.rb
Ignoring io-console-0.7.2 because its extensions are not built. Try: gem pristine io-console --version 0.7.2
Ignoring json-2.7.2 because its extensions are not built. Try: gem pristine json --version 2.7.2
Ignoring nio4r-2.7.3 because its extensions are not built. Try: gem pristine nio4r --version 2.7.3
Ignoring racc-1.8.1 because its extensions are not built. Try: gem pristine racc --version 1.8.1
Ignoring stringio-3.1.1 because its extensions are not built. Try: gem pristine stringio --version 3.1.1
Ignoring websocket-driver-0.7.6 because its extensions are not built. Try: gem pristine websocket-driver --version 0.7.6
checking for yaml.h... no
yaml.h not found
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib64
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/usr/bin/$(RUBY_BASE_NAME)
        --with-libyaml-source-dir
        --without-libyaml-source-dir
        --with-yaml-0.1-config
        --without-yaml-0.1-config
        --with-pkg-config
        --without-pkg-config
        --with-libyaml-dir
        --without-libyaml-dir
        --with-libyaml-include
        --without-libyaml-include=${libyaml-dir}/include
        --with-libyaml-lib
        --without-libyaml-lib=${libyaml-dir}/lib64

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /tmp/bundler20240917-59316-e9worhpsych-5.1.2/extensions/x86_64-linux/3.0.0/psych-5.1.2/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /tmp/bundler20240917-59316-e9worhpsych-5.1.2/gems/psych-5.1.2 for inspection.
Results logged to /tmp/bundler20240917-59316-e9worhpsych-5.1.2/extensions/x86_64-linux/3.0.0/psych-5.1.2/gem_make.out

  /usr/share/rubygems/rubygems/ext/builder.rb:93:in `run'
  /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:47:in `block in build'
  /usr/share/ruby/tempfile.rb:317:in `open'
  /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:26:in `build'
  /usr/share/rubygems/rubygems/ext/builder.rb:159:in `build_extension'
  /usr/share/rubygems/rubygems/ext/builder.rb:193:in `block in build_extensions'
  /usr/share/rubygems/rubygems/ext/builder.rb:190:in `each'
  /usr/share/rubygems/rubygems/ext/builder.rb:190:in `build_extensions'
  /usr/share/rubygems/rubygems/installer.rb:837:in `build_extensions'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/rubygems_gem_installer.rb:71:in `build_extensions'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/rubygems_gem_installer.rb:28:in `install'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/source/rubygems.rb:204:in `install'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/installer/gem_installer.rb:54:in `install'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/installer/gem_installer.rb:16:in `install_from_spec'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/installer/parallel_installer.rb:186:in `do_install'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/installer/parallel_installer.rb:177:in `block in worker_pool'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/worker.rb:62:in `apply_func'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/worker.rb:57:in `block in process_queue'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/worker.rb:54:in `loop'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/worker.rb:54:in `process_queue'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/worker.rb:91:in `block (2 levels) in create_threads'

An error occurred while installing psych (5.1.2), and Bundler cannot continue.

In Gemfile:
  foreman_virt_who_configure was resolved to 0.5.23, which depends on
    katello was resolved to 3.4.5, which depends on
      bastion was resolved to 5.0.8, which depends on
        angular-rails-templates was resolved to 0.1.2, which depends on
          railties was resolved to 7.1.4, which depends on
            irb was resolved to 1.14.0, which depends on
              rdoc was resolved to 6.7.0, which depends on
                psych


Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    current directory: /tmp/bundler20240917-59316-l8hk6xqpid_messaging-0.34.1/gems/qpid_messaging-0.34.1/ext/cqpid
/usr/bin/ruby -I /usr/share/rubygems -r ./siteconf20240917-59316-7db6q.rb extconf.rb
Ignoring io-console-0.7.2 because its extensions are not built. Try: gem pristine io-console --version 0.7.2
Ignoring nio4r-2.7.3 because its extensions are not built. Try: gem pristine nio4r --version 2.7.3
Ignoring racc-1.8.1 because its extensions are not built. Try: gem pristine racc --version 1.8.1
Ignoring stringio-3.1.1 because its extensions are not built. Try: gem pristine stringio --version 3.1.1
checking for -lstdc++... yes
checking for -lstdc++... yes
checking for -lqpidclient... no
Missing required library: qpidclient
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib64
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/usr/bin/$(RUBY_BASE_NAME)
        --with-qpid-dir
        --without-qpid-dir
        --with-qpid-include
        --without-qpid-include=${qpid-dir}/include
        --with-qpid-lib
        --without-qpid-lib=${qpid-dir}/lib64
        --with-stdc++-dir
        --without-stdc++-dir
        --with-stdc++-include
        --without-stdc++-include=${stdc++-dir}/include
        --with-stdc++-lib
        --without-stdc++-lib=${stdc++-dir}/lib64
        --with-stdc++lib
        --without-stdc++lib
        --with-stdc++lib
        --without-stdc++lib
        --with-qpidclient-dir
        --without-qpidclient-dir
        --with-qpidclient-include
        --without-qpidclient-include=${qpidclient-dir}/include
        --with-qpidclient-lib
        --without-qpidclient-lib=${qpidclient-dir}/lib64
        --with-qpidclientlib
        --without-qpidclientlib

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /tmp/bundler20240917-59316-l8hk6xqpid_messaging-0.34.1/extensions/x86_64-linux/3.0.0/qpid_messaging-0.34.1/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /tmp/bundler20240917-59316-l8hk6xqpid_messaging-0.34.1/gems/qpid_messaging-0.34.1 for inspection.
Results logged to /tmp/bundler20240917-59316-l8hk6xqpid_messaging-0.34.1/extensions/x86_64-linux/3.0.0/qpid_messaging-0.34.1/gem_make.out

  /usr/share/rubygems/rubygems/ext/builder.rb:93:in `run'
  /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:47:in `block in build'
  /usr/share/ruby/tempfile.rb:317:in `open'
  /usr/share/rubygems/rubygems/ext/ext_conf_builder.rb:26:in `build'
  /usr/share/rubygems/rubygems/ext/builder.rb:159:in `build_extension'
  /usr/share/rubygems/rubygems/ext/builder.rb:193:in `block in build_extensions'
  /usr/share/rubygems/rubygems/ext/builder.rb:190:in `each'
  /usr/share/rubygems/rubygems/ext/builder.rb:190:in `build_extensions'
  /usr/share/rubygems/rubygems/installer.rb:837:in `build_extensions'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/rubygems_gem_installer.rb:71:in `build_extensions'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/rubygems_gem_installer.rb:28:in `install'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/source/rubygems.rb:204:in `install'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/installer/gem_installer.rb:54:in `install'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/installer/gem_installer.rb:16:in `install_from_spec'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/installer/parallel_installer.rb:186:in `do_install'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/installer/parallel_installer.rb:177:in `block in worker_pool'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/worker.rb:62:in `apply_func'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/worker.rb:57:in `block in process_queue'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/worker.rb:54:in `loop'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/worker.rb:54:in `process_queue'
  /usr/share/gems/gems/bundler-2.2.33/lib/bundler/worker.rb:91:in `block (2 levels) in create_threads'

An error occurred while installing qpid_messaging (0.34.1), and Bundler cannot continue.

In Gemfile:
  foreman_virt_who_configure was resolved to 0.5.23, which depends on
    katello was resolved to 3.4.5, which depends on
      qpid_messaging

@chris1984
Copy link
Member

Do you want me to make a pr to update the pins we have, or is this related to something else?

@ekohl
Copy link
Member Author

ekohl commented Sep 17, 2024

checking for yaml.h... no
yaml.h not found

This indicates it's missing libyaml-devel

  /tmp/bundler20240917-59316-l8hk6xqpid_messaging-0.34.1/extensions/x86_64-linux/3.0.0/qpid_messaging-0.34.1/mkmf.log

Why is it trying to install qpid related libraries? Are you trying to install for an older version?

@chris1984
Copy link
Member

chris1984 commented Sep 17, 2024

checking for yaml.h... no
yaml.h not found

This indicates it's missing libyaml-devel

  /tmp/bundler20240917-59316-l8hk6xqpid_messaging-0.34.1/extensions/x86_64-linux/3.0.0/qpid_messaging-0.34.1/mkmf.log

Why is it trying to install qpid related libraries? Are you trying to install for an older version?

I installed a fresh centos9-stream box and then ran the ansible playbook to install it:

ansible-playbook -kbv playbooks/katello_devel.yml -e '{ "katello_repositories_version":"nightly", "foreman_repositories_version":"nightly", "foreman_installer_options":["--foreman-proxy-content-enable-ostree=true", "--katello-devel-modulestream-nodejs=18"] }'

I installed libyaml-devel and reran the bundle update and getting the original error again:

[vagrant@toledodevel2 foreman]$ bundle update
Ignoring bigdecimal-3.1.8 because its extensions are not built. Try: gem pristine bigdecimal --version 3.1.8
Ignoring date-3.3.4 because its extensions are not built. Try: gem pristine date --version 3.3.4
Ignoring io-console-0.7.2 because its extensions are not built. Try: gem pristine io-console --version 0.7.2
Ignoring json-2.7.2 because its extensions are not built. Try: gem pristine json --version 2.7.2
Ignoring nio4r-2.7.3 because its extensions are not built. Try: gem pristine nio4r --version 2.7.3
Ignoring racc-1.8.1 because its extensions are not built. Try: gem pristine racc --version 1.8.1
Ignoring stringio-3.1.1 because its extensions are not built. Try: gem pristine stringio --version 3.1.1
Ignoring websocket-driver-0.7.6 because its extensions are not built. Try: gem pristine websocket-driver --version 0.7.6
WARNING: File locale/bn/LC_MESSAGES/foreman_virt_who_configure.mo outdated, regenerate with 'make all-mo'
WARNING: File locale/fr/LC_MESSAGES/foreman_virt_who_configure.mo outdated, regenerate with 'make all-mo'
WARNING: File locale/he_IL/LC_MESSAGES/foreman_virt_who_configure.mo outdated, regenerate with 'make all-mo'
WARNING: File locale/ta/LC_MESSAGES/foreman_virt_who_configure.mo outdated, regenerate with 'make all-mo'
Your Gemfile lists the gem katello (>= 0) more than once.
You should probably keep only one of them.
Remove any duplicate entries and specify the gem only once.
While it's not a problem now, it could cause errors if you change the version of one of them later.

[!] There was an error parsing `Gemfile`: You cannot specify the same gem twice coming from different sources.
You specified that katello (>= 0) should come from https://github.com/Katello/katello.git and source at `../katello`
. Bundler cannot continue.

 #  from /home/vagrant/foreman/Gemfile:58
 #  -------------------------------------------
 #  Dir["#{File.dirname(FOREMAN_GEMFILE)}/bundler.d/*.rb"].each do |bundle|
 >    instance_eval(Bundler.read_file(bundle))
 #  end
 #  -------------------------------------------

Everything is on the latest:

[vagrant@toledodevel2 katello]$ git branch
* master
[vagrant@toledodevel2 foreman]$ git branch
* develop
[vagrant@toledodevel2 foreman_virt_who_configure]$ git branch
  main
* make-katello-gem-optional
[vagrant@toledodevel2 foreman_virt_who_configure]$

@chris1984
Copy link
Member

The comment I initially posted, I must have ran bundle update from the foreman-virt-who-configure directory. The latest comment shows from Foreman, sorry about that.

@ekohl
Copy link
Member Author

ekohl commented Sep 17, 2024

I suspect it depends on the order they're defined and that makes it unreliable.

Please check which bundler files are included and the order. You can sort them by filename and name foreman_virt_who_configure something like zz_foreman_virt_who_configure to make sure it's last.

@chris1984
Copy link
Member

I suspect it depends on the order they're defined and that makes it unreliable.

Please check which bundler files are included and the order. You can sort them by filename and name foreman_virt_who_configure something like zz_foreman_virt_who_configure to make sure it's last.

bundle update worked this time :) I did the rename like you suggested. Here is my layout:

-rw-r--r--. 1 vagrant vagrant 251 Sep 17 14:57 assets.rb
-rw-r--r--. 1 vagrant vagrant 104 Sep 17 14:57 console.rb
-rw-r--r--. 1 vagrant vagrant 640 Sep 17 14:57 development.rb
-rw-r--r--. 1 vagrant vagrant 103 Sep 17 14:57 dynflow_sidekiq.rb
-rw-r--r--. 1 vagrant vagrant  53 Sep 17 14:57 ec2.rb
-rw-r--r--. 1 vagrant vagrant  36 Sep 17 14:57 facter.rb
-rw-r--r--. 1 vagrant vagrant 228 Sep 17 14:57 foreman_remote_execution.local.rb
-rw-r--r--. 1 vagrant vagrant 137 Sep 17 14:57 journald.rb
-rw-r--r--. 1 vagrant vagrant  65 Sep 17 14:57 jsonp.rb
-rw-r--r--. 1 vagrant vagrant 160 Sep 17 14:57 katello.local.rb
-rw-r--r--. 1 vagrant vagrant 109 Sep 17 14:57 libvirt.rb
-rw-r--r--. 1 vagrant vagrant  34 Sep 17 14:57 nulldb.rb
-rw-r--r--. 1 vagrant vagrant  51 Sep 17 14:57 openid.rb
-rw-r--r--. 1 vagrant vagrant  69 Sep 17 14:57 openstack.rb
-rw-r--r--. 1 vagrant vagrant  57 Sep 17 14:57 ovirt.rb
-rw-r--r--. 1 vagrant vagrant 129 Sep 17 14:57 postgresql.rb
-rw-r--r--. 1 vagrant vagrant  46 Sep 17 14:57 redis.rb
-rw-r--r--. 1 vagrant vagrant 162 Sep 17 14:57 service.rb
-rw-r--r--. 1 vagrant vagrant 100 Sep 17 14:57 telemetry.rb
-rw-r--r--. 1 vagrant vagrant 895 Sep 17 14:57 test.rb
-rw-r--r--. 1 vagrant vagrant  62 Sep 17 14:57 vmware.rb
-rw-r--r--. 1 vagrant vagrant 236 Sep 17 14:57 zz_foreman_virt_who_configure.local.rb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants