Skip to content

Commit

Permalink
add rough draft
Browse files Browse the repository at this point in the history
  • Loading branch information
guimachiavelli committed Nov 28, 2024
1 parent 310647c commit bd4a8a7
Showing 1 changed file with 32 additions and 0 deletions.
32 changes: 32 additions & 0 deletions learn/ai_powered_search/choose_an_embedder.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
title: How to choose an embedder — Meilisearch documentation
description: This article contains general guidance on how to choose the embedder best suited for projects using AI-powered search.
---

# How to choose an embedder

Meilisearch officially supports many different embedders, such as OpenAI, Hugging Face, and Ollama, as well as the majority of embedding generators with a RESTful API. It can be difficult to understand their differences and how to pick one.

This article contains general guidance on how to choose the embedder best suited for your project.

## When in doubt, choose OpenAI

OpenAI returns relevant search results across different subjects and datasets. It is suited for the majority of applications and Meilisearch actively supports and improves OpenAI functionality with every new release.

In the majority of cases, and especially if this is your first time working with LLMS, choose OpenAI.

## If you are already using a specific AI service, choose the REST embedder

If you are already using a specific model from a compatible embedder, choose Meilisearch's REST embedder. This ensures you continue building upon tooling and workflows already in place with minimal configuration necessary.

## If dealing with non-textual content, choose the user-provided embedder

Meilisearch does not support searching images, audio, or any other content not presented as text. This limitation applies to both queries and documents—for example, you cannot search using an image instead of text, and you cannot use text to search for images without attached textual metadata.

In these cases, you will have to supply your own embedder.

## If working with small static datasets, consider choosing Hugging Face

Although it returns very relevant search results, the Hugging Face embedder must run directly in your server. This may lead to lower performance and extra costs when you are hosting Meilisearch in a service like DigitalOcean or AWS.

That said, Hugging Face can be a good embedder for datasets under 10k documents that you don't intend to update often.

0 comments on commit bd4a8a7

Please sign in to comment.