Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide JLLs for GAP packages #614

Open
fingolfin opened this issue Jan 27, 2021 · 2 comments
Open

Provide JLLs for GAP packages #614

fingolfin opened this issue Jan 27, 2021 · 2 comments

Comments

@fingolfin
Copy link
Member

fingolfin commented Jan 27, 2021

I thought we had an issue for this already, but couldn't find one...

The idea here being that we provide JLLs for all GAP packages (resp. at least those which compile code). This would solve a bunch of issues

  • user's don't need a C/C++ compiler and -dev versions of libraries (e.g. on Debian, they don't need to install libjulia-dev anymore)
    • (well, at least assuming we also treat JuliaInterface to this, which is slightly more annoying than for other GAP packages; so we might want to postpone that, and focus on packaging things like browse, io and ferret
  • all kinds of things can go wrong when compiled C/C++ code -- this source of bugs would be eliminate in one go
  • no more C++ library version mismatch (see issue libstdc++.so.6: version `GLIBCXX_3.4.26' not found #553 )
  • we can control how those kernel extensions are compiled and linked (so we could ensure that e.g. Browse is built against libncurses from ncurses_jll, and perhaps also ensure that when its initialization runs, then TERMINFO_DIRS is set right (see issue problem to load GAP's Browse package #456 )
  • we could even prebuild interface like NormalizInterface which would probably be very convenient for some users
  • installing GAP packages would be super easy for Julia users, and also declaring dependencies on them: they'd be regular Julia packages, after all.

Some thoughts on implementation:

  • I'd call the JLLs GAP_pkgnameinlowercase_jll
  • I imagine writing a GAP/Julia script which parses PackageInfo.g files and uses those to produce build_tarballs.jl files suitable for use in Yggdrasil; that way, updating to new versions becomes easy (of course for the initial prototyping, we can write the packages "manually")
  • those JLLs would have GAP.jl as a dependency
  • to get GAP to load these GAP packages:
    • we could put them into a suitable gaproot (but there are all kinds of pitfalls, which I won't go into right now), or
    • we could let them call GAP.Globals.SetPackagePath in their __init__ section? For this to work properly, though, we must have the dependencies between the packages declared correctly (and this will be a problem because some GAP packages have cyclic dependencies, e.g. polymake and alnuth; so actually we can't use the SetPackagePath idea with them, because they need to be visible to GAP at the same time... ) Actually, I think this is the way to go.
  • in this model, each JLL would contain the full package; an alternative would be to only put the binaries into the JLL, and have the "common" files (GAP code, documentation, ...) in a separate artifact. Then one would need both GAP_pkgnameinlowercase_jll.jl and GAP_pkgnameinlowercase.jl. This is more work to maintain and get working; but it might help resolve issues with circular dependencies, and reduce download churn (if we have to regenerate a JLL for a given version of a GAP package, a complete new artifact needs to be downloaded, and of course that's a lot smaller if it only contains executables, not GAP data files)
  • some packages may want to write into their package directory, we'd need to deal with that
@fingolfin
Copy link
Member Author

@fingolfin
Copy link
Member Author

We now use JLLs for most GAP packages directly in GAP.jl. The JLLs only contain the binaries, the rest of the packages are pulled in as artifacts via one big Artifacts.toml in GAP.jl.

There is still an idea of having Julia packages GAP_pkg_NAME_OF_PACKAGE.jl which do this on a per-package basis, and the work I've begun on this ages ago still is in https://github.com/oscar-system/GAP_pkg but is currently stalled because of open questions like

  • dealing with circular dependencies between GAP packages ...
  • conflicts between GAP's "regular" loading mechanism versus loading a GAP package via the GAP_pkg_*.jl Julia wrappers
  • mapping GAP package versions to Julia packages (that bit actually mostly works fine) and dealing with the fact that GAP packages don't follow semver, but Julia does (and thus if a GAP package AAA decide to upgrade from 1.x to 2.0, on the GAP side nobody would notice, but the Julia wrapper packages for package BBB that depends on AAA >= 1.0 now would have to be adjusted to depend on "AAA >= 1.0, AAA >= 2.0" (i.e. as a Julia compat string: `AAA = 1, 2```).
  • insufficient support by Julia tooling for having > 150 packages in one repository: while this is in principle possible, AFAIK if one has just updated say 40 of those packages then one has to separately tell JuliaRegistrator to release each of them (there is no "release all packages in this repository that had their version bumped to a non-DEV version" or so)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant