Skip to content

Nix portability improvements #13005

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: master
Choose a base branch
from

Conversation

hacker1024
Copy link

This MR updates the Nix package in the interest of improving portability (in regards to cross-compilation and other libcs).

Changes:

  • Use a multi-output derivation to reduce runtime closure sizes
  • Move dependencies into the native/propagated buildInput sets where appropriate
  • Do not link stdc++fs on platforms that do not support it
  • Allow setting LLAMA_BUILD_* options through Nix
  • Enable tests in the Nix build
  • Add myself as a Nix package maintainer

Tested platforms:

  • GCC/glibc on x86_64 for x86_64
  • LLVM/glibc on x86_64 for x86_64
  • GCC/glibc on x86_64 for aarch64
  • GCC/musl on x86_64 for static aarch64

@github-actions github-actions bot added nix Issues specific to consuming flake.nix, or generally concerned with ❄ Nix-based llama.cpp deployment devops improvements to build systems and github actions ggml changes relating to the ggml tensor library for machine learning labels Apr 18, 2025
Copy link
Collaborator

@SomeoneSerge SomeoneSerge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .nix diff looks good; @hacker1024 would you be up to also run the tests? Speaking of, I still think it would make sense to add at least impure builds (i.e. build in a devShell) if not ccache to the CI, just that I'm occupied with other projects

@@ -213,7 +213,7 @@ add_library(ggml

target_link_libraries(ggml PUBLIC ggml-base)

if (CMAKE_SYSTEM_NAME MATCHES "Linux")
Copy link
Collaborator

@SomeoneSerge SomeoneSerge Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a special branch for https://releases.llvm.org/10.0.0/projects/libcxx/docs/UsingLibcxx.html#id2? Note that clang does not necessarily mean libc++, and gcc does not necessarily mean libstdc++ (e.g. in Nixpkgs all stdenvs use the same stdlib determined by the platform)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also idk about libdl handling

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potentially. I guess it depends on how many systems we actually want to support - the original change was required for GCC on RHEL 8, so there's a reason for this branch for that use case.

I'm not sure if it's worth preemptively handling ancient LLVM versions as well unless either someone explicitly requires it or we're actively testing it...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GCC on RHEL 8,

Let's then add a comment explaining the "why"s

@philiptaron
Copy link
Collaborator

philiptaron commented Apr 23, 2025

Tested. Nice. Let's do a nix flake update too. I have on my list to set up an automated PR system to push nix flake updates since #10526 landed.

@SomeoneSerge
Copy link
Collaborator

SomeoneSerge commented Apr 25, 2025

Tested. Nice. Let's do a nix flake update too. I have on my list to set up an automated PR system to push nix flake updates since #10526 landed.

We could also consider replacing the flake with normal Nix and e.g. following the release channel. EDIT: well we'd have to keep around a flake for compatibility, but we could e.g. leave out the lock file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devops improvements to build systems and github actions ggml changes relating to the ggml tensor library for machine learning nix Issues specific to consuming flake.nix, or generally concerned with ❄ Nix-based llama.cpp deployment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants