Skip to content

Commit

Permalink
Use memmap_threshold instead of memmap_threshold_kb (#1254)
Browse files Browse the repository at this point in the history
  • Loading branch information
bashofmann authored Oct 25, 2024
1 parent a561499 commit 518fc72
Show file tree
Hide file tree
Showing 5 changed files with 8 additions and 8 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ client.create_collection(
)
```

For other configurations like `hnsw_config.on_disk` or `memmap_threshold_kb`, see the Qdrant documentation for [Storage.](https://qdrant.tech/documentation/concepts/storage/)
For other configurations like `hnsw_config.on_disk` or `memmap_threshold`, see the Qdrant documentation for [Storage.](https://qdrant.tech/documentation/concepts/storage/)

### SDKs

Expand Down
2 changes: 1 addition & 1 deletion qdrant-landing/content/documentation/concepts/optimizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ storage:
# Memmap storage is disabled by default, to enable it, set this threshold to a reasonable value.
# To disable memmap storage, set this to `0`.
# Note: 1Kb = 1 vector of size 256
memmap_threshold_kb: 200000
memmap_threshold: 200000

# Maximum size (in kilobytes) of vectors allowed for plain index, exceeding this threshold will enable vector indexing
# Default value is 20,000, based on <https://github.com/google-research/google-research/blob/master/scann/docs/algorithms.md>.
Expand Down
4 changes: 2 additions & 2 deletions qdrant-landing/content/documentation/concepts/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,11 +157,11 @@ This will create a collection with all vectors immediately stored in memmap stor
This is the recommended way, in case your Qdrant instance operates with fast disks and you are working with large collections.


- Set up `memmap_threshold_kb` option (deprecated). This option will set the threshold after which the segment will be converted to memmap storage.
- Set up `memmap_threshold` option. This option will set the threshold after which the segment will be converted to memmap storage.

There are two ways to do this:

1. You can set the threshold globally in the [configuration file](/documentation/guides/configuration/). The parameter is called `memmap_threshold_kb`.
1. You can set the threshold globally in the [configuration file](/documentation/guides/configuration/). The parameter is called `memmap_threshold` (previously `memmap_threshold_kb`).
2. You can set the threshold for each collection separately during [creation](/documentation/concepts/collections/#create-collection) or [update](/documentation/concepts/collections/#update-collection-parameters).

```http
Expand Down
6 changes: 3 additions & 3 deletions qdrant-landing/content/documentation/guides/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,8 +251,8 @@ storage:
# To enable memmap storage, lower the threshold
# Note: 1Kb = 1 vector of size 256
# To explicitly disable mmap optimization, set to `0`.
# If not set, will be disabled by default.
memmap_threshold_kb: null
# If not set, will be disabled by default. Previously this was calles memmap_threshold_kb.
memmap_threshold: null

# Maximum size (in KiloBytes) of vectors allowed for plain index.
# Default value based on https://github.com/google-research/google-research/blob/master/scann/docs/algorithms.md
Expand All @@ -277,7 +277,7 @@ storage:
# vacuum_min_vector_number: 1000
# default_segment_number: 0
# max_segment_size_kb: null
# memmap_threshold_kb: null
# memmap_threshold: null
# indexing_threshold_kb: 20000
# flush_interval_sec: 5
# max_optimization_threads: null
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ memmaps may be enabled on a per-vector basis using the `on_disk` parameter. This
will store vector data directly on disk at all times. It is suitable for
ingesting a large amount of data, essential for the billion scale benchmark.

Using `memmap_threshold_kb` is not recommended in this case. It would require
Using `memmap_threshold` is not recommended in this case. It would require
the [optimizer](/documentation/concepts/optimizer/) to constantly
transform in-memory segments into memmap segments on disk. This process is
slower, and the optimizer can be a bottleneck when ingesting a large amount of
Expand Down

0 comments on commit 518fc72

Please sign in to comment.