diff --git a/docs/docs.trychroma.com/markdoc/content/docs/overview/architecture.md b/docs/docs.trychroma.com/markdoc/content/docs/overview/architecture.md index 86891ca7351..bb5b4207822 100644 --- a/docs/docs.trychroma.com/markdoc/content/docs/overview/architecture.md +++ b/docs/docs.trychroma.com/markdoc/content/docs/overview/architecture.md @@ -71,7 +71,7 @@ These components operate differently depending on the deployment mode, particula - In Local and Single Node mode, all components share a process and use the local filesystem for durability. - In **Distributed** mode, components are deployed as independent services. - The log and built indexes are stored in cloud object storage. - - The system catalog is backed by a SQL database. + - The system catalog is backed by an SQL database. - All services use local SSDs as caches to reduce object storage latency and cost. ## Request Sequences diff --git a/docs/docs.trychroma.com/markdoc/content/docs/overview/troubleshooting.md b/docs/docs.trychroma.com/markdoc/content/docs/overview/troubleshooting.md index 9981cc12d63..8269afc434b 100644 --- a/docs/docs.trychroma.com/markdoc/content/docs/overview/troubleshooting.md +++ b/docs/docs.trychroma.com/markdoc/content/docs/overview/troubleshooting.md @@ -26,7 +26,7 @@ export default withChroma(nextConfig); ## Cannot return the results in a contiguous 2D array. Probably ef or M is too small -This error happens when the HNSW index fails to retrieve the requested number of results for a query, given its structure and your data. he way to resolve this is to either decrease the number of results you request from a query (n_result), or increase the HNSW parameters `M`, `ef_construction`, and `ef_search`. You can read more about HNSW configurations [here](/docs/collections/configure). +This error happens when the HNSW index fails to retrieve the requested number of results for a query, given its structure and your data. The way to resolve this is to either decrease the number of results you request from a query (n_result), or increase the HNSW parameters `M`, `ef_construction`, and `ef_search`. You can read more about HNSW configurations [here](/docs/collections/configure). ## Using .get or .query, embeddings say `None` diff --git a/docs/docs.trychroma.com/public/llms-docs-overview-architecture.txt b/docs/docs.trychroma.com/public/llms-docs-overview-architecture.txt index aab0b6699ec..768cabe371d 100644 --- a/docs/docs.trychroma.com/public/llms-docs-overview-architecture.txt +++ b/docs/docs.trychroma.com/public/llms-docs-overview-architecture.txt @@ -64,7 +64,7 @@ These components operate differently depending on the deployment mode, particula - In Local and Single Node mode, all components share a process and use the local filesystem for durability. - In **Distributed** mode, components are deployed as independent services. - The log and built indexes are stored in cloud object storage. - - The system catalog is backed by a SQL database. + - The system catalog is backed by an SQL database. - All services use local SSDs as caches to reduce object storage latency and cost. ## Request Sequences diff --git a/docs/docs.trychroma.com/public/llms-docs-overview-troubleshooting.txt b/docs/docs.trychroma.com/public/llms-docs-overview-troubleshooting.txt index 182e463264f..14ee9e1834b 100644 --- a/docs/docs.trychroma.com/public/llms-docs-overview-troubleshooting.txt +++ b/docs/docs.trychroma.com/public/llms-docs-overview-troubleshooting.txt @@ -21,7 +21,7 @@ export default withChroma(nextConfig); ## Cannot return the results in a contiguous 2D array. Probably ef or M is too small -This error happens when the HNSW index fails to retrieve the requested number of results for a query, given its structure and your data. he way to resolve this is to either decrease the number of results you request from a query (n_result), or increase the HNSW parameters `M`, `ef_construction`, and `ef_search`. You can read more about HNSW configurations [here](/docs/collections/configure). +This error happens when the HNSW index fails to retrieve the requested number of results for a query, given its structure and your data. The way to resolve this is to either decrease the number of results you request from a query (n_result), or increase the HNSW parameters `M`, `ef_construction`, and `ef_search`. You can read more about HNSW configurations [here](/docs/collections/configure). ## Using .get or .query, embeddings say `None` diff --git a/docs/docs.trychroma.com/public/llms-full.text b/docs/docs.trychroma.com/public/llms-full.text index c4e509869c9..8169618d4d7 100644 --- a/docs/docs.trychroma.com/public/llms-full.text +++ b/docs/docs.trychroma.com/public/llms-full.text @@ -1937,7 +1937,7 @@ These components operate differently depending on the deployment mode, particula - In Local and Single Node mode, all components share a process and use the local filesystem for durability. - In **Distributed** mode, components are deployed as independent services. - The log and built indexes are stored in cloud object storage. - - The system catalog is backed by a SQL database. + - The system catalog is backed by an SQL database. - All services use local SSDs as caches to reduce object storage latency and cost. ## Request Sequences @@ -3041,7 +3041,7 @@ export default withChroma(nextConfig); ## Cannot return the results in a contiguous 2D array. Probably ef or M is too small -This error happens when the HNSW index fails to retrieve the requested number of results for a query, given its structure and your data. he way to resolve this is to either decrease the number of results you request from a query (n_result), or increase the HNSW parameters `M`, `ef_construction`, and `ef_search`. You can read more about HNSW configurations [here](/docs/collections/configure). +This error happens when the HNSW index fails to retrieve the requested number of results for a query, given its structure and your data. The way to resolve this is to either decrease the number of results you request from a query (n_result), or increase the HNSW parameters `M`, `ef_construction`, and `ef_search`. You can read more about HNSW configurations [here](/docs/collections/configure). ## Using .get or .query, embeddings say `None` diff --git a/sample_apps/generative_benchmarking/data/chroma_docs.json b/sample_apps/generative_benchmarking/data/chroma_docs.json index 7a0dde70005..9c7e637bf67 100644 --- a/sample_apps/generative_benchmarking/data/chroma_docs.json +++ b/sample_apps/generative_benchmarking/data/chroma_docs.json @@ -131,7 +131,7 @@ "05317095-b882-4191-ae71-f3a3eb72b545": "__call__` now takes a single argument, `input`, to support data of any type `D`. The `texts` argument has been removed.\n\n### Migration from >0.4.0 to 0.4.0 - July 17, 2023\n\nWhat's new in this version?\n\n- New easy way to create clients\n- Changed storage method\n- `.persist()` removed, `.reset()` no longer on by default\n\n**New Clients**\n\n```python\n### in-memory ephemeral client\n\n# before\nimport chromadb\nclient = chromadb.Client()\n\n# after\nimport chromadb\nclient = chromadb.EphemeralClient()\n\n\n### persistent client\n\n# before\nimport chromadb\nfrom chromadb.config import Settings\nclient = chromadb.Client(Settings(\n chroma_db_impl=\"duckdb+parquet\",\n persist_directory=\"/path/to/persist/directory\" # Optional, defaults to .chromadb/ in the current directory\n))\n\n# after\nimport chromadb\nclient = chromadb.PersistentClient(path=\"/path/to/persist/directory\")\n\n\n### http client (to talk to server backend)\n\n# before\nimport chromadb\nfrom chromadb.config import Settings\nclient = chromadb.Client(Settings(ch", "09f8b89c-27ce-4fd7-a8bd-97f7c2dc3e0e": "roma_api_impl=\"rest\",\n chroma_server_host=\"localhost\",\n chroma_server_http_port=\"8000\"\n ))\n\n# after\nimport chromadb\nclient = chromadb.HttpClient(host=\"localhost\", port=\"8000\")\n\n```\n\nYou can still also access the underlying `.Client()` method. If you want to turn off telemetry, all clients support custom settings:\n\n```python\nimport chromadb\nfrom chromadb.config import Settings\nclient = chromadb.PersistentClient(\n path=\"/path/to/persist/directory\",\n settings=Settings(anonymized_telemetry=False))\n```\n\n**New data layout**\n\nThis version of Chroma drops `duckdb` and `clickhouse` in favor of `sqlite` for metadata storage. This means migrating data over. We have created a migration CLI utility to do this.\n\nIf you upgrade to `0.4.0` and try to access data stored in the old way, you will see this error message\n\n> You are using a deprecated configuration of Chroma. Please pip install chroma-migrate and run `chroma-migrate` to upgrade your configuration. See https://docs.trychroma.com/deployment/migration for more information or join our discord at https://", "e3a27357-f6e6-4b35-9cef-656e4ed7ab9d": "discord.gg/MMeYNTmh3x for help!\n\nHere is how to install and use the CLI:\n\n```terminal\npip install chroma-migrate\nchroma-migrate\n```\n\n![](/chroma-migrate.png)\n\nIf you need any help with this migration, please reach out! We are on [Discord](https://discord.com/channels/1073293645303795742/1129286514845691975) ready to help.\n\n**Persist & Reset**\n\n`.persist()` was in the old version of Chroma because writes were only flushed when forced to. Chroma `0.4.0` saves all writes to disk instantly and so `persist` is no longer needed.\n\n`.reset()`, which resets the entire database, used to by enabled-by-default which felt wrong. `0.4.0` has it disabled-by-default. You can enable it again by passing `allow_reset=True` to a Settings object. For example:\n\n```python\nimport chromadb\nfrom chromadb.config import Settings\nclient = chromadb.PersistentClient(path=\"./path/to/chroma\", settings=Settings(allow_reset=True))\n```\n\n# Troubleshooting\n\nThis page is a list of common gotchas or", - "cfefb49a-3328-4f1b-a74c-2385afc499d7": " issues and how to fix them.\n\nIf you don't see your problem listed here, please also search the [Github Issues](https://github.com/chroma-core/chroma/issues).\n\n## Chroma JS-Client failures on NextJS projects\n\nWhen using Chroma with Next.js, be sure to do any embedding in the server - client-side embedding is not supported.\n\nNext.js v15.2.1 includes a fix for embedding functions used by Chroma. If you're using an earlier version of Next.js, you may need to add this configuration to your `next.config.{js|ts}` file:\n\n```typescript\nconst nextConfig = {\n serverExternalPackages: ['chromadb', 'chromadb-default-embed'],\n};\nmodule.exports = nextConfig\n```\n\nIn addition, make sure you're using the latest of the `chromadb` package. Version v2.0.0 includes some important fixes for Next.js environments.\n\n\n## Cannot return the results in a contiguous 2D array. Probably ef or M is too small\n\nThis error happens when the HNSW index fails to retrieve the requested number of results for a query, given its structure and your data. he way to resolve this is to either decrease the number of", + "cfefb49a-3328-4f1b-a74c-2385afc499d7": " issues and how to fix them.\n\nIf you don't see your problem listed here, please also search the [Github Issues](https://github.com/chroma-core/chroma/issues).\n\n## Chroma JS-Client failures on NextJS projects\n\nWhen using Chroma with Next.js, be sure to do any embedding in the server - client-side embedding is not supported.\n\nNext.js v15.2.1 includes a fix for embedding functions used by Chroma. If you're using an earlier version of Next.js, you may need to add this configuration to your `next.config.{js|ts}` file:\n\n```typescript\nconst nextConfig = {\n serverExternalPackages: ['chromadb', 'chromadb-default-embed'],\n};\nmodule.exports = nextConfig\n```\n\nIn addition, make sure you're using the latest of the `chromadb` package. Version v2.0.0 includes some important fixes for Next.js environments.\n\n\n## Cannot return the results in a contiguous 2D array. Probably ef or M is too small\n\nThis error happens when the HNSW index fails to retrieve the requested number of results for a query, given its structure and your data. The way to resolve this is to either decrease the number of", "fc0216bb-1be7-4073-9f97-7b000e4dc7de": " results you request from a query (n_result), or increase the HNSW parameters `M`, `ef_construction`, and `ef_search`. You can read more about HNSW configurations [here](/docs/collections/configure).\n\n## Using .get or .query, embeddings say `None`\n\nThis is actually not an error. Embeddings are quite large and heavy to send back. Most application don't use the underlying embeddings and so, by default, chroma does not send them back.\n\nTo send them back: add `include=[\"embeddings\", \"documents\", \"metadatas\", \"distances\"]` to your query to return all information.\n\nFor example:\n\n```python\nresults = collection.query(\n query_texts=\"hello\",\n n_results=1,\n include=[\"embeddings\", \"documents\", \"metadatas\", \"distances\"],\n)\n```\n\n{% note type=\"tip\" %}\nWe may change `None` to something else to more clearly communicate why they were not returned.\n{% /note %}\n\n\n## Build error when running `pip install chromadb`\n\nIf you encounter an error like this during setup\n\n```\nFailed to build hnswlib\nERROR: Could not build wheels for hnsw", "0b978b27-b452-41ec-b0ec-8524f573dd5a": "lib, which is required to install pyproject.toml-based projects\n```\n\nTry these few tips from the [community](https://github.com/chroma-core/chroma/issues/221):\n\n1. If you get the error: `clang: error: the clang compiler does not support '-march=native'`, set this ENV variable, `export HNSWLIB_NO_NATIVE=1`\n2. If on Mac, install/update xcode dev tools, `xcode-select --install`\n3. If on Windows, try [these steps](https://github.com/chroma-core/chroma/issues/250#issuecomment-1540934224)\n\n## SQLite\n\nChroma requires SQLite > 3.35, if you encounter issues with having too low of a SQLite version please try the following.\n\n1. Install the latest version of Python 3.10, sometimes lower versions of python are bundled with older versions of SQLite.\n2. If you are on a Linux system, you can install pysqlite3-binary, `pip install pysqlite3-binary` and then override the default\n sqlite3 library before running Chroma with the steps [here](https://gist.github.com/defulmere/8b9695", "1d0932ca-5fa7-4e2f-84e3-b018d49f27c6": "e415a44271061cc8e272f3c300).\n Alternatively you can compile SQLite from scratch and replace the library in your python installation with the latest version as documented [here](https://github.com/coleifer/pysqlite3#building-a-statically-linked-library).\n3. If you are on Windows, you can manually download the latest version of SQLite from https://www.sqlite.org/download.html and\n replace the DLL in your python installation's DLLs folder with the latest version. You can find your python installation path by running `os.path.dirname(sys.executable)` in python.\n4. If you are using a Debian based Docker container, older Debian versions do not have an up to date SQLite, please use `bookworm` or higher.\n\n## Illegal instruction (core dumped)\n\nIf you encounter an error like this during setup and are using Docker - you may have built the library on a machine with a different CPU architecture than the one you are running it on. Try rebuilding the Docker image on the machine you are running it on.\n\n## My data directory is too large\n\nIf you were using Chroma prior to v0.5.6, you may be able to significantly shrink your database by [vac",