From 9a9e44e7b6adbb87449df2f52c2642244cd2989b Mon Sep 17 00:00:00 2001 From: gui machiavelli Date: Tue, 31 Oct 2023 16:22:43 +0100 Subject: [PATCH 1/7] version bump --- package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/package.json b/package.json index c32d0a5f52..c6acc858d4 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "documentation", - "version": "1.4.0", + "version": "1.5.0", "description": "", "main": "index.js", "dependencies": {}, From a1aadce6e9d28d9a19d4c8167c728856dbf7e8df Mon Sep 17 00:00:00 2001 From: gui machiavelli Date: Thu, 9 Nov 2023 15:20:50 +0100 Subject: [PATCH 2/7] Update `/metrics` with HTTP API activation info (#2614) --- learn/experimental/overview.mdx | 2 +- reference/api/experimental_features.mdx | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/learn/experimental/overview.mdx b/learn/experimental/overview.mdx index be24d0f086..41842276ed 100644 --- a/learn/experimental/overview.mdx +++ b/learn/experimental/overview.mdx @@ -44,7 +44,7 @@ Activating or deactivating experimental features this way does not require you t | Name | Description | How to configure | |----------------------------------------------------------------------------------|------------------------------------------------------------|---------------------------------------------------| -| [Metrics API](/learn/experimental/metrics) | Exposes Prometheus-compatible analytics data | At launch with a CLI flag or environment variable | +| [Metrics API](/learn/experimental/metrics) | Exposes Prometheus-compatible analytics data | At launch with a CLI flag or environment variable, during runtime with the API route | | [Ranking score details](/learn/experimental/ranking_rule_score_details) | Exposes advanced result ranking score data | During runtime with the API route | | [Reduce indexing memory usage](/learn/experimental/reduce-indexing-memory-usage) | Optimizes indexing performance | At launch with a CLI flag or environment variable | | [Vector search](/learn/experimental/vector_search) | Allows Meilisearch to function as a vector embedding store | During runtime with the API route | diff --git a/reference/api/experimental_features.mdx b/reference/api/experimental_features.mdx index b32a0f8494..2c6b81995e 100644 --- a/reference/api/experimental_features.mdx +++ b/reference/api/experimental_features.mdx @@ -17,6 +17,7 @@ The experimental API route is not compatible with all experimental features. Con ```json { + "metrics": false, "scoreDetails": false, "vectorSearch": false } @@ -24,10 +25,10 @@ The experimental API route is not compatible with all experimental features. Con | Name | Type | Description | | :----------------- | :------ | :--------------------------------------------- | +| **`metrics`** | Boolean | `true` if feature is active, `false` otherwise | | **`scoreDetails`** | Boolean | `true` if feature is active, `false` otherwise | | **`vectorSearch`** | Boolean | `true` if feature is active, `false` otherwise | - ## Get all experimental features @@ -42,6 +43,7 @@ Get a list of all experimental features that can be activated via the `/experime ```json { + "metrics": false, "scoreDetails": false, "vectorSearch": false } @@ -67,6 +69,7 @@ Setting a field to `null` leaves its value unchanged. ```json { + "metrics": false, "scoreDetails": true, "vectorSearch": false } From cdfdb8f16039287e7fdbd1f7c1e7ca27f05f1a5f Mon Sep 17 00:00:00 2001 From: gui machiavelli Date: Thu, 9 Nov 2023 15:21:16 +0100 Subject: [PATCH 3/7] v1.5: Add Khmer to list of officially supported languages (#2617) * Add Khmer to the list of supported languages * shorten and future-proof tokenization page --- learn/inner_workings/tokenization.mdx | 10 ++-------- learn/what_is_meilisearch/language.mdx | 1 + 2 files changed, 3 insertions(+), 8 deletions(-) diff --git a/learn/inner_workings/tokenization.mdx b/learn/inner_workings/tokenization.mdx index 9f3ffc077b..09b4e83a7b 100644 --- a/learn/inner_workings/tokenization.mdx +++ b/learn/inner_workings/tokenization.mdx @@ -20,12 +20,6 @@ We can break down the tokenization process like so: 1. Crawl the document(s), splitting each field by script 2. Go back over the documents part-by-part, running the corresponding tokenization pipeline, if it exists -Pipelines include many language-specific operations. Currently, we have four pipelines: +Pipelines include many language-specific operations. Currently, we have a number of pipelines, including a default pipeline for languages that use whitespace to separate words, and dedicated pipelines for Chinese, Japanese, Hebrew, Thai, and Khmer. -1. A default Meilisearch pipeline for languages that use whitespace to separate words. Uses [Unicode segmenter](https://github.com/unicode-rs/unicode-segmentation) -2. A specialized Chinese pipeline using [Jieba](https://github.com/messense/jieba-rs) -3. A specialized Japanese pipeline using [Lindera](https://github.com/lindera-morphology/lindera) -4. A specialized Hebrew pipeline based off the default Meilisearch pipeline. Uses [Niqqud](https://docs.rs/niqqud/latest/niqqud/) for normalization -5. A specialized Thai pipeline using [dictionary-based](https://github.com/PyThaiNLP/nlpo3) segmentation - -For more details, check out the [tokenizer contribution guide](https://github.com/meilisearch/charabia/blob/main/CONTRIBUTING.md). +For more details, check out the [tokenizer contribution guide](https://github.com/meilisearch/charabia). diff --git a/learn/what_is_meilisearch/language.mdx b/learn/what_is_meilisearch/language.mdx index 4ea871f2bf..50b74badae 100644 --- a/learn/what_is_meilisearch/language.mdx +++ b/learn/what_is_meilisearch/language.mdx @@ -11,6 +11,7 @@ Meilisearch is multilingual, featuring optimized support for: - Chinese - Hebrew - Japanese +- Khmer - Korean - Thai From 372ec968b4c76f287e54d9ce4b2acbf7f53d19ba Mon Sep 17 00:00:00 2001 From: gui machiavelli Date: Thu, 9 Nov 2023 15:21:40 +0100 Subject: [PATCH 4/7] Bump docker version (#2618) --- learn/cookbooks/docker.mdx | 16 ++++++++-------- learn/getting_started/installation.mdx | 4 ++-- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/learn/cookbooks/docker.mdx b/learn/cookbooks/docker.mdx index 0f01659494..9a00f8dfbf 100644 --- a/learn/cookbooks/docker.mdx +++ b/learn/cookbooks/docker.mdx @@ -14,7 +14,7 @@ Docker is a tool that bundles applications into containers. Docker containers en Docker containers are distributed in images. To use Meilisearch, use the `docker pull` command to download a Meilisearch image: ```sh -docker pull getmeili/meilisearch:v1.4 +docker pull getmeili/meilisearch:v1.5 ``` Meilisearch deploys a new Docker image with every release of the engine. Each image is tagged with the corresponding Meilisearch version, indicated in the above example by the text following the `:` symbol. You can see [the full list of available Meilisearch Docker images](https://hub.docker.com/r/getmeili/meilisearch/tags#!) on Docker Hub. @@ -31,7 +31,7 @@ After completing the previous step, use `docker run` to launch the Meilisearch i docker run -it --rm \ -p 7700:7700 \ -v $(pwd)/meili_data:/meili_data \ - getmeili/meilisearch:v1.4 + getmeili/meilisearch:v1.5 ``` ### Configure Meilisearch @@ -47,7 +47,7 @@ docker run -it --rm \ -p 7700:7700 \ -e MEILI_MASTER_KEY='MASTER_KEY'\ -v $(pwd)/meili_data:/meili_data \ - getmeili/meilisearch:v1.4 + getmeili/meilisearch:v1.5 ``` #### Passing instance options with CLI arguments @@ -58,7 +58,7 @@ If you want to pass command-line arguments to Meilisearch with Docker, you must docker run -it --rm \ -p 7700:7700 \ -v $(pwd)/meili_data:/meili_data \ - getmeili/meilisearch:v1.4 + getmeili/meilisearch:v1.5 meilisearch --master-key="MASTER_KEY" ``` @@ -76,7 +76,7 @@ To keep your data intact between reboots, specify a dedicated volume by running docker run -it --rm \ -p 7700:7700 \ -v $(pwd)/meili_data:/meili_data \ - getmeili/meilisearch:v1.4 + getmeili/meilisearch:v1.5 ``` The example above uses `$(pwd)/meili_data`, which is a directory in the host machine. Depending on your OS, mounting volumes from the host to the container might result in performance loss and is only recommended when developing your application. @@ -91,7 +91,7 @@ To import a dump, use Meilisearch's `--import-dump` command-line option and spec docker run -it --rm \ -p 7700:7700 \ -v $(pwd)/meili_data:/meili_data \ - getmeili/meilisearch:v1.4 + getmeili/meilisearch:v1.5 meilisearch --import-dump /meili_data/dumps/20200813-042312213.dump ``` @@ -111,7 +111,7 @@ To generate a Meilisearch snapshot with Docker, launch Meilisearch with `--sched docker run -it --rm \ -p 7700:7700 \ -v $(pwd)/meili_data:/meili_data \ - getmeili/meilisearch:v1.4 + getmeili/meilisearch:v1.5 meilisearch --schedule-snapshot --snapshot-dir /meili_data/snapshots ``` @@ -123,7 +123,7 @@ To import a snapshot, launch Meilisearch with the `--import-snapshot` option: docker run -it --rm \ -p 7700:7700 \ -v $(pwd)/meili_data:/meili_data \ - getmeili/meilisearch:v1.4 + getmeili/meilisearch:v1.5 meilisearch --import-snapshot /meili_data/snapshots/data.ms.snapshot ``` diff --git a/learn/getting_started/installation.mdx b/learn/getting_started/installation.mdx index 36a7f5b806..66147b7da3 100644 --- a/learn/getting_started/installation.mdx +++ b/learn/getting_started/installation.mdx @@ -51,14 +51,14 @@ These commands launch the **latest stable release** of Meilisearch. ```bash # Fetch the latest version of Meilisearch image from DockerHub -docker pull getmeili/meilisearch:v1.4 +docker pull getmeili/meilisearch:v1.5 # Launch Meilisearch in development mode with a master key docker run -it --rm \ -p 7700:7700 \ -e MEILI_ENV='development' \ -v $(pwd)/meili_data:/meili_data \ - getmeili/meilisearch:v1.4 + getmeili/meilisearch:v1.5 # Use ${pwd} instead of $(pwd) in PowerShell ``` From 943ed243501e2903d5cd916b8881b6316e81b08c Mon Sep 17 00:00:00 2001 From: gui machiavelli Date: Thu, 9 Nov 2023 15:22:23 +0100 Subject: [PATCH 5/7] v1.5: Postman collection (#2619) * Update postman collection for v1.5 * trying to fix postman export weirdness --- .../misc/meilisearch-collection-postman.json | 49 ++++++++++++++++++- 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a/assets/misc/meilisearch-collection-postman.json b/assets/misc/meilisearch-collection-postman.json index 0ffcc5be90..c5d23007ee 100644 --- a/assets/misc/meilisearch-collection-postman.json +++ b/assets/misc/meilisearch-collection-postman.json @@ -1,7 +1,7 @@ { "info": { - "_postman_id": "ca046707-e482-4e2c-a524-0574bd10dd5f", - "name": "Meilisearch v1.4", + "_postman_id": "5ac31b23-8cd6-41d5-b1ed-a47123edc334", + "name": "Meilisearch v1.5", "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json" }, "item": [ @@ -2437,6 +2437,51 @@ } ] }, + { + "name": "Snapshots", + "item": [ + { + "name": "Create a snapshot", + "protocolProfileBehavior": { + "disabledSystemHeaders": {} + }, + "request": { + "method": "POST", + "header": [], + "url": { + "raw": "{{url}}/snapshots", + "host": [ + "{{url}}" + ], + "path": [ + "snapshots" + ] + } + }, + "response": [] + } + ], + "event": [ + { + "listen": "prerequest", + "script": { + "type": "text/javascript", + "exec": [ + "" + ] + } + }, + { + "listen": "test", + "script": { + "type": "text/javascript", + "exec": [ + "" + ] + } + } + ] + }, { "name": "Facet search", "item": [ From 121696bbeb40cb00a9b8fa228324da107e68db23 Mon Sep 17 00:00:00 2001 From: gui machiavelli Date: Thu, 16 Nov 2023 12:02:47 +0100 Subject: [PATCH 6/7] v1.5: Puffin reports (#2613) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Clément Renault --- config/sidebar-learn.json | 5 +++++ learn/experimental/overview.mdx | 1 + learn/experimental/puffin_reports.mdx | 18 ++++++++++++++++++ reference/api/experimental_features.mdx | 20 ++++++++++++-------- 4 files changed, 36 insertions(+), 8 deletions(-) create mode 100644 learn/experimental/puffin_reports.mdx diff --git a/config/sidebar-learn.json b/config/sidebar-learn.json index b9cc15bb2d..38f1c2df10 100644 --- a/config/sidebar-learn.json +++ b/config/sidebar-learn.json @@ -397,6 +397,11 @@ "label": "Metrics", "slug": "metrics" }, + { + "source": "learn/experimental/puffin_reports.mdx", + "label": "Puffin reports", + "slug": "puffin_reports" + }, { "source": "learn/experimental/reduce-indexing-memory-usage.mdx", "label": "Reduce indexing memory usage", diff --git a/learn/experimental/overview.mdx b/learn/experimental/overview.mdx index 41842276ed..48c703200d 100644 --- a/learn/experimental/overview.mdx +++ b/learn/experimental/overview.mdx @@ -48,3 +48,4 @@ Activating or deactivating experimental features this way does not require you t | [Ranking score details](/learn/experimental/ranking_rule_score_details) | Exposes advanced result ranking score data | During runtime with the API route | | [Reduce indexing memory usage](/learn/experimental/reduce-indexing-memory-usage) | Optimizes indexing performance | At launch with a CLI flag or environment variable | | [Vector search](/learn/experimental/vector_search) | Allows Meilisearch to function as a vector embedding store | During runtime with the API route | +| [Puffin reports](/learn/experimental/puffin_reports) | Export profiler data to diagnose performance issues | During runtime with the API route | diff --git a/learn/experimental/puffin_reports.mdx b/learn/experimental/puffin_reports.mdx new file mode 100644 index 0000000000..fe2e0e7989 --- /dev/null +++ b/learn/experimental/puffin_reports.mdx @@ -0,0 +1,18 @@ +--- +title: Puffin reports — Meilisearch documentation +description: Puffin is an instrumentation profiler for Rust applications. Use this experimental feature when trying to diagnose performance issues during indexing. +--- + +# Puffin reports + +Puffin is an instrumentation profiler for Rust applications. Use this feature when trying to diagnose performance issues during indexing. Avoid enabling Puffin reports in production, as it might affect the performance and memory usage of other Meilisearch operations. + +[Visit the GitHub discussion to learn how to use this feature](https://github.com/meilisearch/product/discussions/693). + +Are you using this feature? Meilisearch wants to hear from you! Tell us about your experience in the [GitHub discussion](https://github.com/meilisearch/product/discussions/693). All feedback is useful and helps make Meilisearch better and easier to use. + + +Exporting Puffin reports is an experimental feature. Experimental features are unstable: their API might significantly change and become incompatible between releases. Meilisearch does not recommend using experimental features in a production environment. + +Meilisearch makes experimental features available expecting they will become stable in a future release. However, it is not possible to guarantee when and if this will happen. + diff --git a/reference/api/experimental_features.mdx b/reference/api/experimental_features.mdx index 2c6b81995e..d3e4565be8 100644 --- a/reference/api/experimental_features.mdx +++ b/reference/api/experimental_features.mdx @@ -19,15 +19,17 @@ The experimental API route is not compatible with all experimental features. Con { "metrics": false, "scoreDetails": false, - "vectorSearch": false + "vectorSearch": false, + "exportPuffinReports": false } ``` -| Name | Type | Description | -| :----------------- | :------ | :--------------------------------------------- | -| **`metrics`** | Boolean | `true` if feature is active, `false` otherwise | -| **`scoreDetails`** | Boolean | `true` if feature is active, `false` otherwise | -| **`vectorSearch`** | Boolean | `true` if feature is active, `false` otherwise | +| Name | Type | Description | +| :------------------------ | :------ | :--------------------------------------------- | +| **`metrics`** | Boolean | `true` if feature is active, `false` otherwise | +| **`scoreDetails`** | Boolean | `true` if feature is active, `false` otherwise | +| **`vectorSearch`** | Boolean | `true` if feature is active, `false` otherwise | +| **`exportPuffinReports`** | Boolean | `true` if feature is active, `false` otherwise | ## Get all experimental features @@ -45,7 +47,8 @@ Get a list of all experimental features that can be activated via the `/experime { "metrics": false, "scoreDetails": false, - "vectorSearch": false + "vectorSearch": false, + "exportPuffinReports": false } ``` @@ -71,6 +74,7 @@ Setting a field to `null` leaves its value unchanged. { "metrics": false, "scoreDetails": true, - "vectorSearch": false + "vectorSearch": false, + "exportPuffinReports": false } ``` \ No newline at end of file From 12b4227d568f4ab50b8cbcb8d5df9f478a43b2e8 Mon Sep 17 00:00:00 2001 From: gui machiavelli Date: Thu, 16 Nov 2023 12:03:12 +0100 Subject: [PATCH 7/7] v1.5: New `/snapshots` API route (#2616) --- .code-samples.meilisearch.yaml | 3 +++ config/sidebar-reference.json | 5 +++++ learn/data_backup/snapshots.mdx | 34 +++++++++++++++++----------- reference/api/keys.mdx | 1 + reference/api/snapshots.mdx | 39 +++++++++++++++++++++++++++++++++ 5 files changed, 69 insertions(+), 13 deletions(-) create mode 100644 reference/api/snapshots.mdx diff --git a/.code-samples.meilisearch.yaml b/.code-samples.meilisearch.yaml index 787dcc6169..3430cf3762 100644 --- a/.code-samples.meilisearch.yaml +++ b/.code-samples.meilisearch.yaml @@ -1144,3 +1144,6 @@ update_dictionary_1: |- reset_dictionary_1: |- curl \ -X DELETE 'http://localhost:7700/indexes/books/settings/dictionary' +create_snapshot_1: |- + curl \ + -X POST 'http://localhost:7700/snapshots' diff --git a/config/sidebar-reference.json b/config/sidebar-reference.json index 1e99dd19e2..9f162aa6ab 100644 --- a/config/sidebar-reference.json +++ b/config/sidebar-reference.json @@ -48,6 +48,11 @@ "label": "Settings", "slug": "settings" }, + { + "source": "reference/api/snapshots.mdx", + "label": "Snapshots", + "slug": "snapshots" + }, { "source": "reference/api/stats.mdx", "label": "Stats", diff --git a/learn/data_backup/snapshots.mdx b/learn/data_backup/snapshots.mdx index 79b8ac8e13..27c3a2a4b0 100644 --- a/learn/data_backup/snapshots.mdx +++ b/learn/data_backup/snapshots.mdx @@ -11,35 +11,43 @@ Since databases are bound to the Meilisearch version that created them and snaps ## Creating snapshots -To create snapshots, use the [`--schedule-snapshot` configuration option](/learn/configuration/instance_options#schedule-snapshot-creation): +### Creating a single snapshot + +Use [the `/snapshots` route](/reference/api/snapshots) to generate a single snapshot: + + + +This will return a summarized task object. Use [the `/tasks` route](/reference/api/tasks) to monitor the status of this operation. + +### Scheduling periodic snapshots + +Use the [`--schedule-snapshot` configuration option](/learn/configuration/instance_options#schedule-snapshot-creation) to create snapshots at regular time intervals: ```bash meilisearch --schedule-snapshot ``` -By default, Meilisearch creates snapshots in a directory called `snapshots/` at the root of your Meilisearch. +The first snapshot is created on launching Meilisearch. After that, snapshots are created on a set interval until you deactivate snapshots by ending the Meilisearch instance. By default, one snapshot is taken every 24 hours. -The destination can be modified with [`--snapshot-dir`](/learn/configuration/instance_options#snapshot-destination): +Modify the interval between each new snapshot by providing an integer to [`--schedule-snapshot`](/learn/configuration/instance_options#schedule-snapshot-creation): ```bash -meilisearch --schedule-snapshot --snapshot-dir mySnapShots/ +meilisearch --schedule-snapshot=3600 ``` -Now snapshots are created in `mySnapShots/` directory. +[Learn more about snapshots flags and environment variables.](/learn/configuration/instance_options#schedule-snapshot-creation) + +### Snapshot directory -The first snapshot is created on launching Meilisearch. After that, snapshots are created routinely on a set interval until you deactivate snapshots by ending the Meilisearch instance. By default, one snapshot is taken every 24 hours. +By default, Meilisearch creates snapshots in a directory called `snapshots/` at the root of your project. -The interval between each new snapshot can be modified by providing an integer to [`--schedule-snapshot`](/learn/configuration/instance_options#schedule-snapshot-creation): +The destination can be modified with [`--snapshot-dir`](/learn/configuration/instance_options#snapshot-destination): ```bash -meilisearch --schedule-snapshot=3600 +meilisearch --schedule-snapshot --snapshot-dir mySnapShots/ ``` -After running the above code, a snapshot is created every hour (3600 seconds). - -During snapshot creation, old snapshots are **automatically overwritten**. This means only the most recent snapshot should be present in the folder at any given time. - -[More about snapshots flags and environment variables](/learn/configuration/instance_options#schedule-snapshot-creation) +Old snapshots are **automatically overwritten** during snapshot creation. Only the most recent snapshot will be present in the folder at any given time. ## Starting from a snapshot diff --git a/reference/api/keys.mdx b/reference/api/keys.mdx index e1ff4ea718..a35a714d78 100644 --- a/reference/api/keys.mdx +++ b/reference/api/keys.mdx @@ -95,6 +95,7 @@ For security reasons, we do not recommend creating keys that can perform all act | **`settings.update`** | Provides access to the [update settings](/reference/api/settings#update-settings) and [reset settings](/reference/api/settings#reset-settings) endpoints and equivalents for all subroutes | | **`stats.get`** | Provides access to the [get stats of an index](/reference/api/stats#get-stats-of-an-index) endpoint and the [get stats of all indexes](/reference/api/stats#get-stats-of-all-indexes) endpoint. For the latter, **non-authorized `indexes` are omitted from the response** | | **`dumps.create`** | Provides access to the [create dump](/reference/api/dump#create-a-dump) endpoint. **Not restricted by `indexes`** | +| **`snapshots.create`** | Provides access to the [create snapshot](/reference/api/snapshots#create-a-snapshot) endpoint. **Not restricted by `indexes`** | | **`version`** | Provides access to the [get Meilisearch version](/reference/api/version#get-version-of-meilisearch) endpoint | | **`keys.get`** | Provides access to the [get all keys](#get-all-keys) endpoint | | **`keys.create`** | Provides access to the [create key](#create-a-key) endpoint | diff --git a/reference/api/snapshots.mdx b/reference/api/snapshots.mdx new file mode 100644 index 0000000000..ef2896365e --- /dev/null +++ b/reference/api/snapshots.mdx @@ -0,0 +1,39 @@ +--- +title: Snapshots — Meilisearch API reference +description: The /snapshots route creates database snapshots. Use snapshots to backup your Meilisearch data. +--- + +# Snapshots + +The `/snapshot` route allows you to create database snapshots. Snapshots are `.snapshot` files that can be used to make quick backups of Meilisearch data. + +[Learn more about snapshots.](/learn/advanced/snapshots) + +## Create a snapshot + + + +Triggers a snapshot creation task. Once the process is complete, Meilisearch creates a snapshot in the [snapshot directory](/learn/configuration/instance_options#snapshot-destination). If the snapshot directory does not exist yet, it will be created. + +Snapshot tasks take priority over other tasks in the queue. + +[Learn more about asynchronous operations](/learn/async/asynchronous_operations). + +### Example + + + +#### Response: `202 Accepted` + +```json +{ + "taskUid": 1, + "indexUid": null, + "status": "enqueued", + "type": "snapshotCreation", + "enqueuedAt": "2023-06-21T11:09:36.417758Z" +} +``` + +You can use this `taskUid` to get more details on [the status of the task](/reference/api/tasks#get-one-task) +