From 0e99f1917ee49ce11c96a28bbddfa9e95b5e7519 Mon Sep 17 00:00:00 2001 From: Krystle Salazar Date: Wed, 12 Jul 2023 13:00:35 -0400 Subject: [PATCH] Add note of optionality of new aliases --- ...ntation_plan_staging_elasticsearch_reindex_dags.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/documentation/projects/proposals/search_relevancy_sandbox/20230530-implementation_plan_staging_elasticsearch_reindex_dags.md b/documentation/projects/proposals/search_relevancy_sandbox/20230530-implementation_plan_staging_elasticsearch_reindex_dags.md index cc49854aab2..17a74ed3e09 100644 --- a/documentation/projects/proposals/search_relevancy_sandbox/20230530-implementation_plan_staging_elasticsearch_reindex_dags.md +++ b/documentation/projects/proposals/search_relevancy_sandbox/20230530-implementation_plan_staging_elasticsearch_reindex_dags.md @@ -19,9 +19,10 @@ Elasticsearch (ES) index creation ––full and proportional-by-provider–– will allow us to decouple the process from the long Ingestion server's data refresh process and experiment with smaller indices. The DAG factories will create a DAG per media type supported, which currently consists `image` and -`audio`. Also includes the adoption of two new index aliases for ease of -handling and querying the new index types from the API with the -[`internal__index`][api_ii_param] param. +`audio`, limited to staging environment at first. Also includes the adoption of +two new index aliases for ease of handling and querying the new index types from +the API with the [`internal__index`][api_ii_param] param. The use of these +aliases is optional as the resulting indices can continue to be used directly. [api_ii_param]: https://github.com/WordPress/openverse/pull/2073 @@ -223,9 +224,9 @@ We can discard the DAGs if the results are not as expected. -Elasticsearch does not impose any limit on the amount of indices one can create +Elasticsearch does not impose any limit on the amount of indexes one can create but naturally they come with a cost. We don't have policies for creating or -deleting indices for the time being so we should monitor if we reach a point +deleting indexes for the time being so we should monitor if we reach a point where having many indexes impact the cluster performance. ## Prior art