Deployed 99c2448 to dev with MkDocs 1.6.1 and mike 2.1.3

dev/components-gallery/embeddings/index.html

@@ -4997,6 +4997,12 @@ <h1 id="embeddings-gallery">Embeddings Gallery<a class="headerlink" href="#embed
 <div class="grid cards">
 <ul>
 <li>
+<p><span class="twemoji lg middle"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M15 3v4.59L7.59 15H3v6h6v-4.58L16.42 9H21V3m-4 2h2v2h-2M5 17h2v2H5"/></svg></span> <strong>LlamaCppEmbeddings</strong></p>
+<hr />
+<p><code>LlamaCpp</code> library implementation for embedding generation.</p>
+<p><a class="bottom" href="llamacppembeddings/"><span class="twemoji"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M13.22 19.03a.75.75 0 0 1 0-1.06L18.19 13H3.75a.75.75 0 0 1 0-1.5h14.44l-4.97-4.97a.749.749 0 0 1 .326-1.275.75.75 0 0 1 .734.215l6.25 6.25a.75.75 0 0 1 0 1.06l-6.25 6.25a.75.75 0 0 1-1.06 0"/></svg></span> LlamaCppEmbeddings</a></p>
+</li>
+<li>
 <p><span class="twemoji lg middle"><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M15 3v4.59L7.59 15H3v6h6v-4.58L16.42 9H21V3m-4 2h2v2h-2M5 17h2v2H5"/></svg></span> <strong>SentenceTransformerEmbeddings</strong></p>
 <hr />
 <p><code>sentence-transformers</code> library implementation for embedding generation.</p>

dev/components-gallery/embeddings/index.md

@@ -10,6 +10,14 @@ hide:
 <div class="grid cards" markdown>
 
 
+-   :material-vector-line:{ .lg .middle } __LlamaCppEmbeddings__
+
+    ---
+
+    `LlamaCpp` library implementation for embedding generation.
+
+    [:octicons-arrow-right-24: LlamaCppEmbeddings](llamacppembeddings.md){ .bottom }
+
 -   :material-vector-line:{ .lg .middle } __SentenceTransformerEmbeddings__
 
     ---

dev/components-gallery/embeddings/llamacppembeddings.md

@@ -0,0 +1,138 @@
+---
+hide:
+  - navigation
+---
+# LlamaCppEmbeddings
+
+
+`LlamaCpp` library implementation for embedding generation.
+
+
+
+
+
+
+
+### Attributes
+
+- **model_name**: contains the name of the GGUF quantized model, compatible with the  installed version of the `llama.cpp` Python bindings.
+
+- **model_path**: contains the path to the GGUF quantized model, compatible with the  installed version of the `llama.cpp` Python bindings.
+
+- **repo_id**: the Hugging Face Hub repository id.
+
+- **verbose**: whether to print verbose output. Defaults to `False`.
+
+- **n_gpu_layers**: number of layers to run on the GPU. Defaults to `-1` (use the GPU if available).
+
+- **disable_cuda_device_placement**: whether to disable CUDA device placement. Defaults to `True`.
+
+- **normalize_embeddings**: whether to normalize the embeddings. Defaults to `False`.
+
+- **seed**: RNG seed, -1 for random
+
+- **n_ctx**: Text context, 0 = from model
+
+- **n_batch**: Prompt processing maximum batch size
+
+- **extra_kwargs**: additional dictionary of keyword arguments that will be passed to the  `Llama` class of `llama_cpp` library. Defaults to `{}`.
+
+
+
+
+
+### Runtime Parameters
+
+- **n_gpu_layers**: the number of layers to use for the GPU. Defaults to `-1`.
+
+- **verbose**: whether to print verbose output. Defaults to `False`.
+
+- **normalize_embeddings**: whether to normalize the embeddings. Defaults to `False`.
+
+- **extra_kwargs**: additional dictionary of keyword arguments that will be passed to the  `Llama` class of `llama_cpp` library. Defaults to `{}`.
+
+
+
+
+### Examples
+
+
+#### Generate sentence embeddings using a local model
+```python
+from pathlib import Path
+from distilabel.models.embeddings import LlamaCppEmbeddings
+
+# You can follow along this example downloading the following model running the following
+# command in the terminal, that will download the model to the `Downloads` folder:
+# curl -L -o ~/Downloads/all-MiniLM-L6-v2-Q2_K.gguf https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF/resolve/main/all-MiniLM-L6-v2-Q2_K.gguf
+
+model_path = "Downloads/"
+model = "all-MiniLM-L6-v2-Q2_K.gguf"
+embeddings = LlamaCppEmbeddings(
+    model=model,
+    model_path=str(Path.home() / model_path),
+)
+
+embeddings.load()
+
+results = embeddings.encode(inputs=["distilabel is awesome!", "and Argilla!"])
+print(results)
+embeddings.unload()
+```
+
+#### Generate sentence embeddings using a HuggingFace Hub model
+```python
+from distilabel.models.embeddings import LlamaCppEmbeddings
+# You need to set environment variable to download private model to the local machine
+
+repo_id = "second-state/All-MiniLM-L6-v2-Embedding-GGUF"
+model = "all-MiniLM-L6-v2-Q2_K.gguf"
+embeddings = LlamaCppEmbeddings(model=model,repo_id=repo_id)
+
+embeddings.load()
+
+results = embeddings.encode(inputs=["distilabel is awesome!", "and Argilla!"])
+print(results)
+embeddings.unload()
+# [
+#   [-0.05447685346007347, -0.01623094454407692, ...],
+#   [4.4889533455716446e-05, 0.044016145169734955, ...],
+# ]
+```
+
+#### Generate sentence embeddings with cpu
+```python
+from pathlib import Path
+from distilabel.models.embeddings import LlamaCppEmbeddings
+
+# You can follow along this example downloading the following model running the following
+# command in the terminal, that will download the model to the `Downloads` folder:
+# curl -L -o ~/Downloads/all-MiniLM-L6-v2-Q2_K.gguf https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF/resolve/main/all-MiniLM-L6-v2-Q2_K.gguf
+
+model_path = "Downloads/"
+model = "all-MiniLM-L6-v2-Q2_K.gguf"
+embeddings = LlamaCppEmbeddings(
+    model=model,
+    model_path=str(Path.home() / model_path),
+    n_gpu_layers=0,
+    disable_cuda_device_placement=True,
+)
+
+embeddings.load()
+
+results = embeddings.encode(inputs=["distilabel is awesome!", "and Argilla!"])
+print(results)
+embeddings.unload()
+# [
+#   [-0.05447685346007347, -0.01623094454407692, ...],
+#   [4.4889533455716446e-05, 0.044016145169734955, ...],
+# ]
+```
+
+
+
+
+### References
+
+- [Offline inference embeddings](https://llama-cpp-python.readthedocs.io/en/stable/#embeddings)
+