Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enabling building llama.cpp using system libggml #12321

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ckastner
Copy link
Contributor

A llama.cpp build could make use of an already installed libggml using the latter's cmake package. This would be the typical case for Linux distributions.

Note: This requires a ggml build where GGML_BIN_DIR is commented out, see 82d9ca4.

# Build and install ggml to some custom path
$ git clone https://github.com/ggml-org/ggml.git && cd ggml
$ cmake -B build -DCMAKE_INSTALL_PREFIX=/tmp/system-ggml -DCMAKE_INSTALL_RPATH='$ORIGIN'
$ cmake --build build && cmake --install build

# Build and install llama against this ggml, using ggml's cmake package
$ cmake -B build \
	-DLLAMA_USE_SYSTEM_GGML=ON \
	-DCMAKE_PREFIX_PATH=/tmp/system-ggml/lib/cmake/ggml \
	-DLLAMA_BUILD_EXAMPLES=ON \
	-DLLAMA_BUILD_TESTS=ON
$ cmake --build build
$ build/bin/llama-bench
...
$ bash ./ci/run.sh ./tmp/results ./tmp/mnt
...

# Negative check, to make sure we didn't break the regular build
$ cmake -B build \
	-DLLAMA_USE_SYSTEM_GGML=OFF \
	-DLLAMA_BUILD_EXAMPLES=ON \
	-DLLAMA_BUILD_TESTS=ON
$ cmake --build build
...
$ bash ./ci/run.sh ./tmp/results ./tmp/mnt
...

The tests passed but I get a ModuleNotFoundError (torch) from convert_hf_to_gguf.py, which is unexpected. I haven't investigated yet.

llama.cpps's build requires it, too, and we may want to make use of it
without add_subdirectory(ggml).
@github-actions github-actions bot added build Compilation issues examples ggml changes relating to the ggml tensor library for machine learning labels Mar 10, 2025
This facilitates package maintenance for Linux distributions, where the
libggml library most likely will be shipped as an individual package
upon which a llama.cpp package depends.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build Compilation issues examples ggml changes relating to the ggml tensor library for machine learning
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants