diff --git a/Documentation/5.2/Raven.Documentation.Pages/studio/database/Indexes/indexes-overview.markdown b/Documentation/5.2/Raven.Documentation.Pages/studio/database/Indexes/indexes-overview.markdown index 456788e063..bfe1b45e42 100644 --- a/Documentation/5.2/Raven.Documentation.Pages/studio/database/Indexes/indexes-overview.markdown +++ b/Documentation/5.2/Raven.Documentation.Pages/studio/database/Indexes/indexes-overview.markdown @@ -59,72 +59,78 @@ {NOTE: } #### 1. Index Definition +--- * **The index definition** tells RavenDB how to index the data. -* It specifies the fields to be indexed and how those fields should be indexed (i.e. allowing a full-text-search option). - These fields can be specified explicitly or defined dynamically supporting any document structure. +* It specifies which fields to index and defines how they should be indexed, + e.g., configuring a field for full-text search, selecting the analyzer to use, etc. + These fields can be specified explicitly or defined dynamically supporting any document structure. -* The index definition is created by the client (**Static-Index**), or by the server Query Optimizer(**Auto-Index**). +* The index definition is created by the client (**Static-Index**), or by the server (**Auto-Index**). * Note: Data from related documents can also be indexed using 'LoadDocument'. Learn more in [Indexing Related Documents](../../../indexes/indexing-related-documents). -{NOTE/} +{NOTE/} {NOTE: } #### 2. Indexing Process +--- -* **Indexing** is the process of indexing the data, iterating over the documents, and creating a map - between the terms indexed and the actual documents that contain them. +* **Indexing** is the process of iterating over the raw documents, indexing their data as defined by the index definition, + and building a map between the indexed terms and the raw documents that contain them. -* Indexing is a background operation, it is scheduled to occur in an async manner upon any document change. - e.g. A document write operation doesn't wait for the index to complete processing - - The write operation is completed as soon as the transaction is written to disk. +* Indexing is a background operation, it is scheduled to occur in an async manner upon any document change. + Once defined and deployed, an index will initially process the entire dataset. + After that, the index will only process documents that were modified, added or deleted. -* An index is considered [Stale](../../../indexes/stale-indexes) if it had not yet processed all of the data. +* A document write operation doesn't wait for the index to complete processing - + the write operation is completed as soon as the transaction is written to disk. + However, a write operation can wait for the indexing process to finish before acknowledging the write by using method `WaitForIndexesAfterSaveChanges`. -* A query can request that results are returned only when the index is up-to-date - by using method `WaitForNonStaleResults`. - A write operation can wait for the indexing process to finish before acknowledging the write - by using method `WaitForIndexesAfterSaveChanges`. - See: [Understanding Eventual Consistency](../../../users-issues/understanding-eventual-consistency). +* An index is considered [Stale](../../../indexes/stale-indexes) if it has not yet processed all the data. + A query can request that results are returned only when the index is up-to-date by using method `WaitForNonStaleResults`. + Learn more in: [Understanding Eventual Consistency](../../../users-issues/understanding-eventual-consistency). * The async indexing process works with hard resets, shutdowns, and the like. If the database was restarted _after_ a document was modified but _before_ it was indexed, the indexing process will just pick up from where it left off and complete the work. * Each index is assigned a dedicated thread, thus no indexing process can interfere with any other. - By default, indexing threads start with a lower priority than request-processing threads. + By default, indexing threads start with a lower priority than request-processing threads. The indexing-thread priority can be set higher and RavenDB will update this at the operating system level. * Indexing can be **throttled** to delay indexing tasks by a pre-set time period. - Throttling is helpful when sufficient server resources need to remain available - for users while heavy-duty indexing tasks are due. + Throttling is helpful when sufficient server resources need to remain available for users while heavy-duty indexing tasks are due. See: [Index Throttling](../../../indexes/index-throttling) {NOTE/} - {NOTE: } #### 3. Indexed Data +--- -* The resulting output of 'step 2' (the indexing process) is also referred to as an Index. - Queries operate on **indexed data** to get documents result. - -* Note: The full document is _not_ stored in the index - only the document ID. - Upon a query match, we load the document itself from the document storage. +* The resulting output of the indexing process from 'step 2' above is also referred to as an 'Index'. -* **Index Entry** - _Index-Entries_ are all of the document fields that are requested to be indexed, as defined in the index definition. +* **Index-Entries** + During the indexing process, an _index-entry_ is created for each raw document that is processed. + Usually a single _index-entry_ is created per raw document, unless working with a [fanout index](../../../indexes/indexing-nested-data#fanout-index---multiple-index-entries-per-document). -* **Term** - The index-entries values are broken into _Terms_ according to the specified analyzer used in the index definition. - _Term_ is the actual indexed value that is stored in the index. +* **Index-Fields and Terms** + Each _index-entry_ contains the _index-fields_ that were defined in the index definition. + Each _index-field_ contains _terms_ that are generated from the data in the raw documents. + + The _terms_ generated depend on the [analyzer](../../../indexes/using-analyzers) used, and they are the actual indexed values that are stored in the index. + When querying the index, you can retrieve the original documents and filter the results based on these _terms_. * **Stored Data** - In addition to the _Terms_, some document [fields can be stored directly](../../../indexes/storing-data-in-index) in the index data. + In addition to the _terms_, some document [fields can be stored directly](../../../indexes/storing-data-in-index) in the index data. This allows for query results to be fetched from the index itself instead of loading the original document. + + Note: The full document is _not_ stored in the index - only the document ID. + Upon a query match, we load the document itself from the document storage. + {NOTE/} {PANEL/} @@ -134,7 +140,8 @@ Indexes in RavenDB are split across the following multiple axes: {NOTE: } -### Auto Indexes -vs- Static Indexes +#### Auto Indexes -vs- Static Indexes +--- * **Auto Indexes**: * Auto-indexes are created by the server. @@ -152,10 +159,10 @@ Indexes in RavenDB are split across the following multiple axes: as the indexed data can be a computed value. {NOTE/} - {NOTE: } -### Map Indexes -vs- Map-Reduce Indexes +#### Map Indexes -vs- Map-Reduce Indexes +--- * **Map Indexes**: [Map indexes](../../../studio/database/indexes/create-map-index) are simple indexes. @@ -166,11 +173,12 @@ Indexes in RavenDB are split across the following multiple axes: [Map-Reduce indexes](../../../studio/database/indexes/create-map-reduce-index) allow performing complex data aggregation. The _Map_ stage is similar to a regular Map-Index, defining what data should be indexed. The _Reduce_ stage operates on the Map results, specifying how the data should be grouped and aggregated. -{NOTE/} +{NOTE/} {NOTE: } -### Single-Collection Indexes -vs- Multi-Collection Indexes +#### Single-Collection Indexes -vs- Multi-Collection Indexes +--- * **Single-Collection Indexes**: Index definition contains only one Map function defined on a specific collection. @@ -178,6 +186,7 @@ Indexes in RavenDB are split across the following multiple axes: * **Multi-Collection Indexes**: Data from several collections can be indexed (each in a different Map) and the results are united in a single index. The only requirement is that all the Map definitions have the same output shape. + {NOTE/} {PANEL/} diff --git a/Documentation/5.3/Raven.Documentation.Pages/studio/database/Queries/query-view.markdown b/Documentation/5.3/Raven.Documentation.Pages/studio/database/Queries/query-view.markdown index 299c67ba34..d1e0fb9cb1 100644 --- a/Documentation/5.3/Raven.Documentation.Pages/studio/database/Queries/query-view.markdown +++ b/Documentation/5.3/Raven.Documentation.Pages/studio/database/Queries/query-view.markdown @@ -45,7 +45,7 @@ ![Load Query](images/query-view-load-query.png "Load Query") Hover over a query name to display its preview. Click the query name or the preview **Load** button to load the query. -7. **Query Settings** +7. **Query Settings** Click to set query settings. ![Query Settings](images/query-view-settings.png "Query Settings") * a. **Cache enabled** diff --git a/Documentation/5.3/Raven.Documentation.Pages/studio/database/documents/patch-view.markdown b/Documentation/5.3/Raven.Documentation.Pages/studio/database/documents/patch-view.markdown index 95cfbd80da..e7d2ce00a4 100644 --- a/Documentation/5.3/Raven.Documentation.Pages/studio/database/documents/patch-view.markdown +++ b/Documentation/5.3/Raven.Documentation.Pages/studio/database/documents/patch-view.markdown @@ -49,7 +49,7 @@ ![Load Patch](images/patch-view-load-patch.png "Load Patch") Hover over a patch name to display its preview. Click the patch name or the preview **Load** button to load the patch. -3. **Patch Settings** +3. **Patch Settings** ![Patch Settings](images/patch-view-settings.png "Patch Settings") * a. **Patch immediately** diff --git a/Documentation/5.3/Raven.Documentation.Pages/studio/database/settings/studio-configuration.markdown b/Documentation/5.3/Raven.Documentation.Pages/studio/database/settings/studio-configuration.markdown index eff1374a68..6baba9fefb 100644 --- a/Documentation/5.3/Raven.Documentation.Pages/studio/database/settings/studio-configuration.markdown +++ b/Documentation/5.3/Raven.Documentation.Pages/studio/database/settings/studio-configuration.markdown @@ -63,21 +63,30 @@ In this page: {PANEL: About Auto-Indexes and the Studio} **What are Auto-Indexes** -When a query or patch cannot be answered by an existing index, RavenDB creates an [Auto-Index (Dynamic Index)](../../../indexes/creating-and-deploying#auto-indexes) -by default. -These indexes are dynamically maintained to change automatically in response to changing query demands. +* Auto-indexes are created when all of the following conditions are met: + * A query is issued without specifying an index (a [dynamic query](../../../client-api/session/querying/how-to-query#dynamicQuery)). + * The query includes a filtering condition. + * No suitable auto-index exists that can satisfy the query. + * Creation of auto-indexes has not been disabled. -After a certain amount of time that an index is not used [(30 minutes by default)](../../../server/configuration/indexing-configuration#indexing.timetowaitbeforemarkingautoindexasidleinmin), the index goes into an [idle state](../../../studio/database/indexes/indexes-list-view#index-state) -and deleted after a set time-period [(72 hours by default)](../../../server/configuration/indexing-configuration#indexing.timetowaitbeforedeletingautoindexmarkedasidleinhrs). +* For such queries, RavenDB's Query Optimizer searches for an existing auto-index that can satisfy the query. + If no suitable auto-index is found, RavenDB will either [create a new auto-index](../../../indexes/creating-and-deploying#auto-indexes) or optimize an existing auto-index. -To provide for fast queries, indexes process information in the background. -If they are processing large datasets, each index can be demanding on I/O resources. +* Note: dynamic queries can be issued either when [querying](../../../studio/database/queries/query-view#query-view) or when [patching](../../../studio/database/documents/patch-view#patch-configuration). + +* Auto-indexes are dynamically maintained to change automatically in response to changing query demands. + After a certain amount of time that an auto-index is not used [(30 minutes by default)](../../../server/configuration/indexing-configuration#indexing.timetowaitbeforemarkingautoindexasidleinmin), + the index goes into an [idle state](../../../studio/database/indexes/indexes-list-view#index-state) + and deleted after a set time-period [(72 hours by default)](../../../server/configuration/indexing-configuration#indexing.timetowaitbeforedeletingautoindexmarkedasidleinhrs). + +* To provide for fast queries, indexes process information in the background. + If they are processing large datasets, each index can be demanding on I/O resources. **Why disable Auto-Index in Studio queries or patches** Some people use the Studio for one-time, ad-hoc queries and don't want a new index to start using resources. In a playground database, it may be worth keeping auto-indexing active, even for random queries, because you want to be able to experiment. -On the other hand, disabling it in production can prevent expensive indexes from being created and running in the background. +On the other hand, disabling it in production can prevent expensive indexes from being created and running in the background. {PANEL/} @@ -88,8 +97,8 @@ If you disabled Auto-Indexing in the Studio Database Configuration page, and you want a one-time Auto-Index set up to satisfy a Query or Patch, temporarily allow Auto-Index in the Query or Patch settings interface. -* [Query Settings](../../../studio/database/queries/query-view#query-view) -* [Patch Settings](../../../studio/database/documents/patch-view#patch-configuration) +* [Query Settings](../../../studio/database/queries/query-view#query-settings) +* [Patch Settings](../../../studio/database/documents/patch-view#patch-settings) Note: These settings only affect Auto-Indexing as a result of Queries or Patches done in the Studio. They do not affect API-based Queries or Patches. diff --git a/Documentation/5.4/Raven.Documentation.Pages/client-api/operations/maintenance/indexes/put-indexes.dotnet.markdown b/Documentation/5.4/Raven.Documentation.Pages/client-api/operations/maintenance/indexes/put-indexes.dotnet.markdown index fe3366d076..46bab5c76b 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/client-api/operations/maintenance/indexes/put-indexes.dotnet.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/client-api/operations/maintenance/indexes/put-indexes.dotnet.markdown @@ -1,5 +1,4 @@ # Put Indexes Operation - --- {NOTE: } @@ -21,28 +20,37 @@ {PANEL: Ways to deploy indexes - short summary } -#### Static index: +{NOTE: } + +##### Static-indexes: There are a few ways to deploy a static-index from the Client API: - * Call `Execute()` on a specific index instance - * Call `IndexCreation.CreateIndexes()` to deploy multiple indexes - * Execute `PutIndexesOperation` maintenance operation on the Document Store - see below - * Learn more in [static indexes](../../../../indexes/creating-and-deploying#static-indexes) +* The following methods are explained in section [Deploy a static-index](../../../../indexes/creating-and-deploying#deploy-a-static-index): + * Call `Execute()` on a specific index instance. + * Call `ExecuteIndex()` or `ExecuteIndexes()` on your _DocumentStore_ object. + * Call `IndexCreation.CreateIndexes()`. + +* Alternatively, you can execute the `PutIndexesOperation` maintenance operation on the _DocumentStore_, **as explained below**. -#### Auto index: +{NOTE/} +{NOTE: } - * An auto-index is created by the server when making a filtering query that doesn't specify which index to use - * Learn more in [auto indexes](../../../../indexes/creating-and-deploying#auto-indexes) +##### Auto-indexes: + * An auto-index is created by the server when making a filtering query that doesn't specify which index to use. + Learn more in [Creating auto indexes](../../../../indexes/creating-and-deploying#auto-indexes). + +{NOTE/} {PANEL/} {PANEL: Put indexes operation with IndexDefinition } -Using `PutIndexesOperation` with **IndexDefinition** allows the following: +Using `PutIndexesOperation` with **IndexDefinition** allows you to: - * Choosing any name for the index. - * Setting low-level properties available in _IndexDefinition_. + * Choose any name for the index. + This string-based name is specified when querying the index. + * Set low-level properties available in _IndexDefinition_. {CODE-TABS} {CODE-TAB:csharp:LINQ-index-Sync put_1@ClientApi\Operations\Maintenance\Indexes\Put.cs /} @@ -73,38 +81,38 @@ Using `PutIndexesOperation` with an IndexDefinition created from an **IndexDefin {CODE syntax@ClientApi\Operations\Maintenance\Indexes\Put.cs /} -| Parameters | Type | Description | -| - |- | - | +| Parameter | Type | Description | +|------------------|----------------------------|----------------------------------| | **indexesToAdd** | `params IndexDefinition[]` | Definitions of indexes to deploy | -| `IndexDefinition` parameter | Type | Description | -| - | - | - | -| Name | `string` | Name of the index, a unique identifier | -| Maps | `HashSet` | All the map functions for the index | -| Reduce | `string` | The index reduce function | -| DeploymentMode | `IndexDeploymentMode?` | Deployment mode
(Parallel, Rolling) | -| State | `IndexState?` | State of index
(Normal, Disabled, Idle, Error) | -| Priority | `IndexPriority?` | Priority of index
(Low, Normal, High) | -| LockMode | `IndexLockMode?` | Lock mode of index
(Unlock, LockedIgnore, LockedError) | -| Fields | `Dictionary` | _IndexFieldOptions_ per index field | -| AdditionalSources | `Dictionary` | Additional code files to be compiled with this index | -| AdditionalAssemblies | `HashSet` | Additional assemblies that are referenced | -| Configuration | `IndexConfiguration` | Can override [indexing configuration](../../../../server/configuration/indexing-configuration) by setting this dictionary | -| OutputReduceToCollection | `string` | A collection name for saving the reduce results as documents | -| ReduceOutputIndex | `long?` | This number will be part of the reduce results documents IDs | -| PatternForOutputReduceToCollectionReferences | `string` | Pattern for documents IDs which reference IDs of reduce results documents | -| PatternReferencesCollectionName | `string` | A collection name for the reference documents created based on provided pattern | - -| `store.Maintenance.Send(putIndexesOp)` return value | Description | -| - | - | -| `PutIndexResult[]` | List of _PutIndexResult_ per index | - -| `PutIndexResult` parameter | Type | Description | -| - | - | - | -| Index | `string` | Name of the index that was added | -| RaftCommandIndex | `long` | Index of raft command that was executed | +| `IndexDefinition` parameter | Type | Description | +|----------------------------------------------|-----------------------------------------|---------------------------------------------------------------------------------------------------------------------------| +| Name | `string` | Name of the index, a unique identifier | +| Maps | `HashSet` | All the map functions for the index | +| Reduce | `string` | The index reduce function | +| DeploymentMode | `IndexDeploymentMode?` | Deployment mode
(Parallel, Rolling) | +| State | `IndexState?` | State of index
(Normal, Disabled, Idle, Error) | +| Priority | `IndexPriority?` | Priority of index
(Low, Normal, High) | +| LockMode | `IndexLockMode?` | Lock mode of index
(Unlock, LockedIgnore, LockedError) | +| Fields | `Dictionary` | _IndexFieldOptions_ per index field | +| AdditionalSources | `Dictionary` | Additional code files to be compiled with this index | +| AdditionalAssemblies | `HashSet` | Additional assemblies that are referenced | +| Configuration | `IndexConfiguration` | Can override [indexing configuration](../../../../server/configuration/indexing-configuration) by setting this dictionary | +| OutputReduceToCollection | `string` | A collection name for saving the reduce results as documents | +| ReduceOutputIndex | `long?` | This number will be part of the reduce results documents IDs | +| PatternForOutputReduceToCollectionReferences | `string` | Pattern for documents IDs which reference IDs of reduce results documents | +| PatternReferencesCollectionName | `string` | A collection name for the reference documents created based on provided pattern | + +| `store.Maintenance.Send(putIndexesOp)` return value | Description | +|-------------------------------------------------------|------------------------------------| +| `PutIndexResult[]` | List of _PutIndexResult_ per index | + +| `PutIndexResult` parameter | Type | Description | +|-----------------------------|----------|-----------------------------------------| +| Index | `string` | Name of the index that was added | +| RaftCommandIndex | `long` | Index of raft command that was executed | {PANEL/} diff --git a/Documentation/5.4/Raven.Documentation.Pages/client-api/operations/maintenance/indexes/put-indexes.js.markdown b/Documentation/5.4/Raven.Documentation.Pages/client-api/operations/maintenance/indexes/put-indexes.js.markdown index 9ad3af116d..0ed7a6b37e 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/client-api/operations/maintenance/indexes/put-indexes.js.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/client-api/operations/maintenance/indexes/put-indexes.js.markdown @@ -20,28 +20,37 @@ {PANEL: Ways to deploy indexes - short summary } -#### Static index: +{NOTE: } + +##### Static-indexes: -There are a few ways to deploy a static-index from the Client API: +There are a few ways to deploy a static-index from the Client API: - * Call `execute()` on a specific index instance - * Call `IndexCreation.createIndexes()` to deploy multiple indexes - * Execute `PutIndexesOperation` maintenance operation on the Document Store - see below - * Learn more in [static indexes](../../../../indexes/creating-and-deploying#static-indexes) +* The following methods are explained in section [Deploy a static-index](../../../../indexes/creating-and-deploying#deploy-a-static-index): + * Call `execute()` on a specific index instance. + * Call `executeIndex()` or `executeIndexes()` on your _DocumentStore_ object. + * Call `IndexCreation.createIndexes()`. -#### Auto index: +* Alternatively, you can execute the `PutIndexesOperation` maintenance operation on the _DocumentStore_, **as explained below**. - * An auto-index is created by the server when making a filtering query that doesn't specify which index to use - * Learn more in [auto indexes](../../../../indexes/creating-and-deploying#auto-indexes) +{NOTE/} +{NOTE: } +##### Auto-indexes: + +* An auto-index is created by the server when making a filtering query that doesn't specify which index to use. + Learn more in [Creating auto indexes](../../../../indexes/creating-and-deploying#auto-indexes). + +{NOTE/} {PANEL/} {PANEL: Put indexes operation with IndexDefinition } -Using `PutIndexesOperation` with **IndexDefinition** allows the following: +Using `PutIndexesOperation` with **IndexDefinition** allows you to: - * Choosing any name for the index - * Setting low-level properties available in _IndexDefinition_ + * Choose any name for the index. + This string-based name is specified when querying the index. + * Set low-level properties available in _IndexDefinition_. {CODE-TABS} {CODE-TAB:nodejs:LINQ-index put_1@client-api\operations\maintenance\indexes\put.js /} @@ -54,38 +63,38 @@ Using `PutIndexesOperation` with **IndexDefinition** allows the following: {CODE:nodejs syntax@client-api\operations\maintenance\indexes\put.js /} -| Parameters | Type | Description | -| - |- | - | +| Parameter | Type | Description | +|------------------|------------------------|----------------------------------| | **indexesToAdd** | `...IndexDefinition[]` | Definitions of indexes to deploy | -| `IndexDefinition` parameter | Type | Description | -| - | - | - | -| name | `string` | Name of the index, a unique identifier | -| maps | `Set` | All the map functions for the index | -| reduce | `string` | The index reduce function | -| deploymentMode | `object` | Deployment mode
(Parallel, Rolling) | -| state | `object` | State of index
(Normal, Disabled, Idle, Error) | -| priority | `object` | Priority of index
(Low, Normal, High) | -| lockMode | `object` | Lock mode of index
(Unlock, LockedIgnore, LockedError) | -| fields | `Record` | _IndexFieldOptions_ per index field | -| additionalSources | `Record` | Additional code files to be compiled with this index | -| additionalAssemblies | `object[]` | Additional assemblies that are referenced | -| configuration | `object` | Can override [indexing configuration](../../../../server/configuration/indexing-configuration) by setting this Record<string, string> | -| outputReduceToCollection | `string` | A collection name for saving the reduce results as documents | -| reduceOutputIndex | `number` | This number will be part of the reduce results documents IDs | -| patternForOutputReduceToCollectionReferences | `string` | Pattern for documents IDs which reference IDs of reduce results documents | -| patternReferencesCollectionName | `string` | A collection name for the reference documents created based on provided pattern | - -| `store.maintenance.send(putIndexesOp)` return value| Description | -| - | - | -| `object[]` | operation result per index | - -| Operation result per index | Type | Description | -| - | - | - | -| index | `string` | Name of the index that was added | -| raftCommandIndex | `long` | Index of raft command that was executed | +| `IndexDefinition` parameter | Type | Description | +|----------------------------------------------|--------------------------|---------------------------------------------------------------------------------------------------------------------------------------------| +| name | `string` | Name of the index, a unique identifier | +| maps | `Set` | All the map functions for the index | +| reduce | `string` | The index reduce function | +| deploymentMode | `object` | Deployment mode
(Parallel, Rolling) | +| state | `object` | State of index
(Normal, Disabled, Idle, Error) | +| priority | `object` | Priority of index
(Low, Normal, High) | +| lockMode | `object` | Lock mode of index
(Unlock, LockedIgnore, LockedError) | +| fields | `Record` | _IndexFieldOptions_ per index field | +| additionalSources | `Record` | Additional code files to be compiled with this index | +| additionalAssemblies | `object[]` | Additional assemblies that are referenced | +| configuration | `object` | Can override [indexing configuration](../../../../server/configuration/indexing-configuration) by setting this Record<string, string> | +| outputReduceToCollection | `string` | A collection name for saving the reduce results as documents | +| reduceOutputIndex | `number` | This number will be part of the reduce results documents IDs | +| patternForOutputReduceToCollectionReferences | `string` | Pattern for documents IDs which reference IDs of reduce results documents | +| patternReferencesCollectionName | `string` | A collection name for the reference documents created based on provided pattern | + +| `store.maintenance.send(putIndexesOp)` return value | Description | +|------------------------------------------------------|----------------------------| +| `object[]` | operation result per index | + +| Operation result per index | Type | Description | +|-----------------------------|----------|-----------------------------------------| +| index | `string` | Name of the index that was added | +| raftCommandIndex | `long` | Index of raft command that was executed | {PANEL/} diff --git a/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.dotnet.markdown b/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.dotnet.markdown index 07b3998e5b..1d39286492 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.dotnet.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.dotnet.markdown @@ -15,7 +15,7 @@ {INFO: } * Including explanations is available only when using **Lucene** as the underlying search engine. - * You can configure which search engine will be used. Learn how in [Selecting the search engine](../../indexes/search-engine/corax#selecting-the-search-engine). + * You can configure which search engine will be used. Learn how in [Selecting the search engine](../../../../indexes/search-engine/corax#selecting-the-search-engine). {INFO/} --- diff --git a/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.java.markdown b/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.java.markdown index 0499227c66..90823ce98f 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.java.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.java.markdown @@ -5,7 +5,7 @@ {INFO: } * Including explanations is available only when using **Lucene** as the underlying search engine. -* You can configure which search engine will be used. Learn how in [Selecting the search engine](../../indexes/search-engine/corax#selecting-the-search-engine). +* You can configure which search engine will be used. Learn how in [Selecting the search engine](../../../../indexes/search-engine/corax#selecting-the-search-engine). {INFO/} diff --git a/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.js.markdown b/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.js.markdown index 8f6d1a427f..34166602b8 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.js.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.js.markdown @@ -14,7 +14,7 @@ {INFO: } * Including explanations is available only when using **Lucene** as the underlying search engine. - * You can configure which search engine will be used. Learn how in [Selecting the search engine](../../indexes/search-engine/corax#selecting-the-search-engine). + * You can configure which search engine will be used. Learn how in [Selecting the search engine](../../../../indexes/search-engine/corax#selecting-the-search-engine). {INFO/} --- diff --git a/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.php.markdown b/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.php.markdown index 5b4b455ffd..51562d2b7d 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.php.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.php.markdown @@ -14,7 +14,7 @@ {INFO: } * Including explanations is available only when using **Lucene** as the underlying search engine. - * You can configure which search engine will be used. Learn how in [Selecting the search engine](../../indexes/search-engine/corax#selecting-the-search-engine). + * You can configure which search engine will be used. Learn how in [Selecting the search engine](../../../../indexes/search-engine/corax#selecting-the-search-engine). {INFO/} --- diff --git a/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.python.markdown b/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.python.markdown index b5f7905c02..88a1da8d28 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.python.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/client-api/session/querying/debugging/include-explanations.python.markdown @@ -14,7 +14,7 @@ {INFO: } * Including explanations is available only when using **Lucene** as the underlying search engine. - * You can configure which search engine will be used. Learn how in [Selecting the search engine](../../indexes/search-engine/corax#selecting-the-search-engine). + * You can configure which search engine will be used. Learn how in [Selecting the search engine](../../../../indexes/search-engine/corax#selecting-the-search-engine). {INFO/} --- diff --git a/Documentation/5.4/Raven.Documentation.Pages/document-extensions/timeseries/indexing.dotnet.markdown b/Documentation/5.4/Raven.Documentation.Pages/document-extensions/timeseries/indexing.dotnet.markdown index 3225ce2a31..3fd6305f1f 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/document-extensions/timeseries/indexing.dotnet.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/document-extensions/timeseries/indexing.dotnet.markdown @@ -239,7 +239,7 @@ Learn more about JavaScript indexes in [JavaScript Indexes](../../indexes/javasc public class TimeSeriesIndexDefinition : IndexDefinition {CODE-BLOCK/} -While `TimeSeriesIndexDefinition` is currently functionally equivalent to the regular [`IndexDefinition`](../../indexes/creating-and-deploying#using-maintenance-operations) class from which it inherits, +While `TimeSeriesIndexDefinition` is currently functionally equivalent to the regular [`IndexDefinition`](../../client-api/operations/maintenance/indexes/put-indexes#put-indexes-operation-with-indexdefinition) class from which it inherits, it is recommended to use `TimeSeriesIndexDefinition` when creating a time series index definition in case additional functionality is added in future versions of RavenDB. --- diff --git a/Documentation/5.4/Raven.Documentation.Pages/document-extensions/timeseries/indexing.js.markdown b/Documentation/5.4/Raven.Documentation.Pages/document-extensions/timeseries/indexing.js.markdown index 663257d580..1bd6c54932 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/document-extensions/timeseries/indexing.js.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/document-extensions/timeseries/indexing.js.markdown @@ -166,7 +166,7 @@ abstract class AbstractRawJavaScriptTimeSeriesIndexCreationTask class TimeSeriesIndexDefinition extends IndexDefinition {CODE-BLOCK/} -While `TimeSeriesIndexDefinition` is currently functionally equivalent to the regular [`IndexDefinition`](../../indexes/creating-and-deploying#using-maintenance-operations) class from which it inherits, +While `TimeSeriesIndexDefinition` is currently functionally equivalent to the regular [`IndexDefinition`](../../client-api/operations/maintenance/indexes/put-indexes#put-indexes-operation-with-indexdefinition) class from which it inherits, it is recommended to use `TimeSeriesIndexDefinition` when creating a time series index definition in case additional functionality is added in future versions of RavenDB. --- diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/creating-and-deploying.dotnet.markdown b/Documentation/5.4/Raven.Documentation.Pages/indexes/creating-and-deploying.dotnet.markdown new file mode 100644 index 0000000000..ce92ac43b4 --- /dev/null +++ b/Documentation/5.4/Raven.Documentation.Pages/indexes/creating-and-deploying.dotnet.markdown @@ -0,0 +1,344 @@ +# Creating and Deploying Indexes +--- + +{NOTE: } + +* This article explains how to create indexes in RavenDB. + For a general overview of indexes, see [What are indexes](../indexes/what-are-indexes). + +* You can either: + * [create a Static-index](../indexes/creating-and-deploying#define-a-static-index) yourself, which involves **Defining** the index and **Deploying** it to the RavenDB server, or + * let the RavenDB server [create an Auto-index](../indexes/creating-and-deploying#creating-auto-indexes) for you based on query patterns. + +* Static-indexes can be created: + * using the Client API, as outlined in this article, or + * from the [Indexes list view](../studio/database/indexes/indexes-list-view) in the Studio. + +--- + +* In this page: + * [Static-indexes](../indexes/creating-and-deploying#static-indexes) + * [Define a static-index](../indexes/creating-and-deploying#define-a-static-index) + * [Deploy a static-index](../indexes/creating-and-deploying#deploy-a-static-index) + * [Deploy single index](../indexes/creating-and-deploying#deploy-single-index) + * [Deploy multiple indexes](../indexes/creating-and-deploying#deploy-multiple-indexes) + * [Deploy syntax](../indexes/creating-and-deploying#deploy-syntax) + * [Deployment behavior](../indexes/creating-and-deploying#deployment-behavior) + * [Creating a static-index - Example](../indexes/creating-and-deploying#create-a-static-index---example) + * [Creating a static-index - using an Operation](../indexes/creating-and-deploying#create-a-static-index---using-an-operation) + * [Auto-indexes](../indexes/creating-and-deploying#auto-indexes) + * [Creating auto-indexes](../indexes/creating-and-deploying#creating-auto-indexes) + * [Disabling auto-indexes](../indexes/creating-and-deploying#disabling-auto-indexes) + +{NOTE/} + + +{PANEL: Define a static-index} + +{CONTENT-FRAME: } + +##### Static-indexes +--- + +* Indexes that are explicitly **created by the user** are called `static` indexes. +* Static-indexes can perform calculations, data conversions, and other processes behind the scenes. + This reduces the workload at query time by offloading these costly operations to the indexing phase. +* To query with a static-index, you must explicitly specify the index in the query definition. + For more details, see [Querying an index](../indexes/querying/query-index). + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Define a static-index using a custom class +--- + +* To define a static-index using a custom class inherit from `AbstractIndexCreationTask`. +* This method is recommended over the [Creating an index using an operation](../indexes/creating-and-deploying#create-a-static-index---using-an-operation) method + for its simplicity and the following advantages: + * **Strongly-typed syntax**: + Provides strong typing when defining the index, making it easier to work with. + * **Ease of querying**: + Lets you use the index class name in a query, instead of hard-coding the index name. +{CODE indexes_1@Indexes/Creating.cs /} +* A complete example of creating a static-index is provided [below](../indexes/creating-and-deploying#create-a-static-index---example). + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Naming convention +--- + +* Static-index class names follow a single naming convention: + Each `_` in the class name is translated to `/` in the index name on the server. +* In the above example, the index class name is `Orders_ByTotal`. + The name of the index that will be generated on the server will be: `Orders/ByTotal`. + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Customizing configuration +--- + +* You can set various [indexing configuration](../server/configuration/indexing-configuration) values within the index definition. +* Setting a configuration value within the index will override the matching indexing configuration values set at the server or database level. +{CODE indexes_2@Indexes/Creating.cs /} + +{CONTENT-FRAME/} +{PANEL/} + +{PANEL: Deploy a static-index} + +* To begin indexing data, the index must be deployed to the server. +* This section provides options for deploying indexes that inherit from `AbstractIndexCreationTask`. +* To create and deploy an index using the `IndexDefinition` class via `PutIndexesOperation`, + see [Creating a static-index - using an Operation](../indexes/creating-and-deploying#create-a-static-index---using-an-operation). + +--- + +{CONTENT-FRAME: } + +##### Deploy single index +--- + +* Use `Execute()` or `ExecuteIndex()` to deploy a single index. +* The following examples deploy index `Ordes/ByTotal` to the default database defined in your _DocumentStore_ object. + See the [syntax](../indexes/creating-and-deploying#deploy-syntax) section below for all available overloads. + +{CODE-TABS} +{CODE-TAB:csharp:Execute indexes_3@Indexes/Creating.cs /} +{CODE-TAB:csharp:Execute_async indexes_4@Indexes/Creating.cs /} +{CODE-TABS/} + +{CODE-TABS} +{CODE-TAB:csharp:ExecuteIndex indexes_5@Indexes/Creating.cs /} +{CODE-TAB:csharp:ExecuteIndex_async indexes_6@Indexes/Creating.cs /} +{CODE-TABS/} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Deploy multiple indexes +--- + +* Use `ExecuteIndexes()` or `IndexCreation.CreateIndexes()` to deploy multiple indexes. +* The `IndexCreation.CreateIndexes` method attempts to create all indexes in a single request. + If it fails, it will repeat the execution by calling the `Execute` method for each index, one by one, + in separate requests. +* The following examples deploy indexes `Ordes/ByTotal` and `Employees/ByLastName` to the default database defined in your _DocumentStore_ object. + See the [syntax](../indexes/creating-and-deploying#deploy-syntax) section below for all available overloads. + +{CODE-TABS} +{CODE-TAB:csharp:ExecuteIndexes indexes_7@Indexes/Creating.cs /} +{CODE-TAB:csharp:ExecuteIndexes_async indexes_8@Indexes/Creating.cs /} +{CODE-TABS/} + +{CODE-TABS} +{CODE-TAB:csharp:CreateIndexes indexes_9@Indexes/Creating.cs /} +{CODE-TAB:csharp:CreateIndexes_async indexes_10@Indexes/Creating.cs /} +{CODE-TABS/} + +--- + +###### Deploy ALL indexes from an assembly + +* The following overload allows you to deploy ALL indexes from a specified assembly: + +{CODE-TABS} +{CODE-TAB:csharp:CreateIndexes indexes_11@Indexes/Creating.cs /} +{CODE-TAB:csharp:CreateIndexes_async indexes_12@Indexes/Creating.cs /} +{CODE-TABS/} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Deploy syntax +--- + +{CODE-TABS} +{CODE-TAB:csharp:Deploy_methods syntax_1@Indexes/Creating.cs /} +{CODE-TAB:csharp:Deploy_methods_async syntax_2@Indexes/Creating.cs /} +{CODE-TABS/} + +| Parameter | Type | Description | +|--------------------|-------------------------------------------|-------------------------------------------------------------------------------------------------------------------| +| **store** | `IDocumentStore` | Your document store object. | +| **conventions** | `DocumentConventions` | The [Conventions](../client-api/configuration/conventions) used by the document store. | +| **database** | `string` | The target database to deploy the index to. If not specified, the default database set on the store will be used. | +| **index** | `IAbstractIndexCreationTask` | The index object to deploy. | +| **indexes** | `IEnumerable` | A list of index objects to deploy. | +| **assemblyToScan** | `Assembly ` | Deploy all indexes that are contained in this assembly. | + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Deployment behavior +--- + +{INFO: } + +###### Deployment mode: +--- + +* When your database spans multiple nodes, + you can choose between **Rolling** index deployment or **Parallel** index deployment. +* Rolling deployment applies the index to one node at a time, + while Parallel deployment deploys the index on all nodes simultaneously. +* Learn more in [Rolling index deployment](../indexes/rolling-index-deployment). + +{INFO/} + +{SAFE: } + +###### When the index you are deploying already exists on the server: +--- + +* **If the index definition is updated**: + * RavenDB uses a side-by-side strategy for all index updates. + * When an existing index definition is modified, RavenDB creates a new index with the updated definition. + The new index will replace the existing index once it becomes non-stale. + * If you want to swap the indexes immediately, you can do so through the Studio. + For more details, see [Side by side indexing](../studio/database/indexes/indexes-list-view#indexes-list-view---side-by-side-indexing). +* **If the index definition is unchanged**: + * If the definition of the index being deployed is identical to the one on the server, + the existing index will not be overwritten. + * The indexed data will remain intact, and the indexing process will not restart. + +{SAFE/} +{CONTENT-FRAME/} +{PANEL/} + +{PANEL: Create a static-index - Example} + +{CODE indexes_13@Indexes/Creating.cs /} + +{PANEL/} + +{PANEL: Create a static-index - using an Operation} + +* An index can also be defined and deployed using the [PutIndexesOperation](../client-api/operations/maintenance/indexes/put-indexes) maintenance operation. + +* When using this operation: + + * Unlike the [naming convention](../indexes/creating-and-deploying#naming-convention) used with indexes inheriting from `AbstractIndexCreationTask`, + you can choose any string-based name for the index. + However, when querying, you must use that string-based name rather than the index class type. + + * You can also modify various low-level settings available in the [IndexDefinition](../client-api/operations/maintenance/indexes/put-indexes#put-indexes-operation-with-indexdefinition) + and [IndexDefinitionBuilder](../client-api/operations/maintenance/indexes/put-indexes#put-indexes-operation-with-indexdefinitionbuilder) classes. + +* Consider using this operation only if inheriting from `AbstractIndexCreationTask` is not an option. + +* For a detailed explanation and examples, refer to the dedicated article: [Put Indexes Operation](../client-api/operations/maintenance/indexes/put-indexes). + +{PANEL/} + + +{PANEL: Creating auto-indexes} + +{CONTENT-FRAME: } + +##### Auto-indexes creation +--- + +* Indexes **created by the server** are called `dynamic` or `auto` indexes. +* Auto-indexes are created when all of the following conditions are met: + * A query is issued without specifying an index (a dynamic query). + * The query includes a filtering condition. + * No suitable auto-index exists that can satisfy the query. + * Creation of auto-indexes has not been disabled. +* For such queries, RavenDB's Query Optimizer searches for an existing auto-index that can satisfy the query. + If no suitable auto-index is found, RavenDB will either create a new auto-index or optimize an existing auto-index. + (Static-indexes are not taken into account when determining which auto-index should handle the query). +* Note: dynamic queries can be issued either when [querying](../studio/database/queries/query-view#query-view) or when [patching](../studio/database/documents/patch-view#patch-configuration). +* Over time, RavenDB automatically adjusts and merges auto-indexes to efficiently serve your queries. + For more details, see [Query a collection - with filtering (dynamic query)](../client-api/session/querying/how-to-query#dynamicQuery). + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Naming convention +--- + +* Auto-indexes are easily identified by their names, which start with the `Auto/` prefix. +* Their name also includes the name of the queried collection and a list of fields used in the query predicate to filter matching results. +* For example, issuing the following query: + {CODE-TABS} + {CODE-TAB:csharp:Query indexes_14@Indexes/Creating.cs /} + {CODE-TAB-BLOCK:sql:RQL} +from Employees +where FirstName = "Robert" and LastName = "King" + {CODE-TAB-BLOCK/} + {CODE-TABS/} + will result in the creation of an auto-index named `Auto/Employees/ByFirstNameAndLastName`. + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Auto-index idle state +--- + +* To reduce server load, an auto-index is marked as `idle` when it hasn't been used for a while. + Specifically, if the time difference between the last time the auto-index was queried + and the last time a query was made on the database (using any index) exceeds the configured threshold (30 minutes by default), + the auto-index will be marked as `idle`. +* This is done in order to avoid marking indexes as idle for databases that were offline for a long period of time, + as well as for databases that were just restored from a snapshot or a backup. +* To set the time before marking an index as idle, use the + [Indexing.TimeToWaitBeforeDeletingAutoIndexMarkedAsIdleInHrs](../server/configuration/indexing-configuration#indexing.timetowaitbeforedeletingautoindexmarkedasidleinhrs) configuration key. + Setting this value too high is not recommended, as it may lead to performance degradation by causing unnecessary and redundant work for the indexes. +* An `idle` auto-index will resume its work and return to `normal` state upon its next query, + or when resetting the index. +* If not resumed, the idle auto-index will be deleted by the server after the time period defined in the + [Indexing.TimeToWaitBeforeDeletingAutoIndexMarkedAsIdleInHrs](../server/configuration/indexing-configuration#indexing.timetowaitbeforedeletingautoindexmarkedasidleinhrs) configuration key + (72 hours by default). + +{CONTENT-FRAME/} +{PANEL/} + +{PANEL: Disabling auto-indexes} + +**Why disable**: + +* Disabling auto-index creation prevents the accidental deployment of resource-consuming auto-indexes that may result from one-time, ad-hoc queries issued from the Studio. +* In production environments, disabling this feature helps avoid the creation and background execution of expensive indexes. + +**How to disable**: + +* You can disable auto-indexes by setting the [Indexing.DisableQueryOptimizerGeneratedIndexes](../server/configuration/indexing-configuration#indexing.disablequeryoptimizergeneratedindexes) configuration key. + This will affect all queries made both from the **Client API** and the **Studio**. + +* Alternatively, you can disable auto-indexes from the Studio. + However, this will affect queries made only from the **Studio**. + + * To disable auto-index creation for a specific query made from the query view, see these [Query settings](../studio/database/queries/query-view#query-settings). + * To disable auto-index creation for a specific query made from the patch view, see these [Patch settings](../studio/database/documents/patch-view#patch-settings). + * Disabling auto-index creation for ALL queries made on a database can be configured in the [Studio configuration view](../studio/database/settings/studio-configuration#disabling-auto-index-creation-on-studio-queries-or-patches). + +{PANEL/} + +## Related Articles + +### Indexes +- [What are Indexes](../indexes/what-are-indexes) +- [Indexing Basics](../indexes/indexing-basics) + +### Operations +- [Put Indexes Operation](../client-api/operations/maintenance/indexes/put-indexes) + +### Querying +- [Query Overview](../client-api/session/querying/how-to-query) +- [Querying an Index](../indexes/querying/query-index) + +### Studio +- [Indexes: Overview](../studio/database/indexes/indexes-overview#indexes-overview) +- [Studio Index List View](../studio/database/indexes/indexes-list-view) + +--- + +### Inside RavenDB +- [Working with Indexes](https://ravendb.net/learn/inside-ravendb-book/reader/4.0/12-working-with-indexes) + +--- + +### Code Walkthrough +- [See various indexing guides in the online DEMO](https://demo.ravendb.net/) diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/creating-and-deploying.java.markdown b/Documentation/5.4/Raven.Documentation.Pages/indexes/creating-and-deploying.java.markdown new file mode 100644 index 0000000000..8904c8c689 --- /dev/null +++ b/Documentation/5.4/Raven.Documentation.Pages/indexes/creating-and-deploying.java.markdown @@ -0,0 +1,196 @@ +# Creating and Deploying Indexes +--- + +{NOTE: } + +* Indexes are used by the server to satisfy queries. + They are at the heart of RavenDB's efficiency and should be understood before indexes and queries are defined in production. + +* Static indexes can do a number of operations on the data behind the scenes so that queries that use this already processed data are as fast as possible. + Indexes keep the processed data in a separate storage so that the raw data isn't affected. + +* Whenever a user issues a query _that doesn't specify an index_, RavenDB's Query Optimizer will try to find an + existing auto-index that fulfills the query. + * If one doesn't yet exist, RavenDB will either create an auto-index or optimize an existing one if it almost satisfies the query. + +* Indexes process data assigned to them as the data changes. For example, if changes are made to documents in the collection "Orders", + the indexes that are defined to handle queries on "Orders" will be triggered to update the index with the new data. + * These behind-the-scenes processes remove a lot of burden from queries. Also, indexes need to process entire datasets just once, + after which, they only process new data. + Still, they utilize machine resources and this should be considered when defining indexes and queries. + + +{INFO: Auto and Static Indexes} + +* Indexes created by issuing a query are called `dynamic` or `Auto` indexes. + * They can be easily identified. Their name starts with the `Auto/` prefix. + * If no [Auto-Index](../indexes/creating-and-deploying#auto-indexes) exists to satisfy a query, + a new Auto-Index will be created and maintained automatically. +* Indexes created explicitly by the user are called `static`. + * [To use a Static Index in a query](../indexes/querying/query-index), + **you must specify the index in the query definition**. If you don't specify the index, + RavenDB will look for an auto-index and potentially create a new one. + * Static Indexes can be defined to do calculations, conversions, and various other processes behind the scenes, to prevent + doing these costly processes at query time (see [Using AbstractIndexCreationTask](../indexes/creating-and-deploying#using-abstractindexcreationtask), + our [map-indexes](../indexes/map-indexes) article, [indexing multiple collections](../indexes/multi-map-indexes), + and [map-reduce indexing](../indexes/map-reduce-indexes)). + +{INFO/} + +* In this page: + * [Static indexes](../indexes/creating-and-deploying#static-indexes) + * [Using AbstractIndexCreationTask](../indexes/creating-and-deploying#using-abstractindexcreationtask) + * [Naming convention](../indexes/creating-and-deploying#naming-convention) + * [Sending to server](../indexes/creating-and-deploying#sending-to-server) + * [Creating an index with custom configuration](../indexes/creating-and-deploying#creating-an-index-with-custom-configuration) + * [Using assembly scanner](../indexes/creating-and-deploying#using-assembly-scanner) + * [Example](../indexes/creating-and-deploying#example) + * [Using maintenance operations](../indexes/creating-and-deploying#using-maintenance-operations) + * [IndexDefinitionBuilder](../indexes/creating-and-deploying#indexdefinitionbuilder) + * [Auto-indexes](../indexes/creating-and-deploying#auto-indexes) + * [Naming convention](../indexes/creating-and-deploying#naming-convention-1) + * [Auto indexes and indexing state](../indexes/creating-and-deploying#auto-indexes-and-indexing-state) + * [If indexes exhaust system resources](../indexes/creating-and-deploying#if-indexes-exhaust-system-resources) + + +{NOTE/} + +{PANEL: Static indexes} + +There are a couple of ways to create a `static index` and send it to the server. We can use [maintenance operations](../indexes/creating-and-deploying#using-maintenance-operations) or create a [custom class](../indexes/creating-and-deploying#using-abstractindexcreationtask). + +
+ +### Using AbstractIndexCreationTask + +AbstractIndexCreationTask let you avoid hard-coding index names in every query. + +{NOTE We recommend creating and using indexes in this form due to its simplicity. There are many benefits and few disadvantages. /} + +#### Naming convention + +There is only one naming convention: each `_` in the class name will be translated to `/` in the index name. + +e.g. + +In the `Northwind` samples, there is a index called `Orders/Totals`. To get such a index name, we need to create a class called `Orders_Totals`. + +{CODE:java indexes_1@Indexes/Creating.java /} + +#### Sending to server + +There is not much use from an index if it is not deployed to the server. To do so, we need to create an instance of our class that inherits from `AbstractIndexCreationTask` and use `execute` method. + +{CODE:java indexes_2@Indexes/Creating.java /} + +{CODE:java indexes_3@Indexes/Creating.java /} + +{SAFE If an index exists on the server and the stored definition is the same as the one that was sent, it will not be overwritten. The indexed data will not be deleted and indexation will not start from scratch. /} + +#### Creating an index with custom configuration + +If you need to create an index with a custom [`index configuration`](../server/configuration/indexing-configuration) you can set them in the index class constructor like so: +{CODE:java indexes_9@Indexes/Creating.java /} + +#### Example + +{CODE:java indexes_8@Indexes/Creating.java /} + +
+ +### Using maintenance operations + +The `PutIndexesOperation` maintenance operation (which API references can be found [here](../client-api/operations/maintenance/indexes/put-indexes)) can be used also to send index(es) to the server. + +The benefit of this approach is that you can choose the name as you feel fit, and change various settings available in `IndexDefinition`. You will have to use string-based names of indexes when querying. + +{CODE:java indexes_5@Indexes/Creating.java /} + +#### IndexDefinitionBuilder + +`IndexDefinitionBuilder` is a very useful class that enables you to create `IndexDefinitions` using strongly-typed syntax with access to low-level settings not available when the `AbstractIndexCreationTask` approach is used. + +{CODE:java indexes_6@Indexes/Creating.java /} + +#### Remarks + +{INFO Maintenance Operations or `IndexDefinitionBuilder` approaches are not recommended and should be used only if you can't do it by inheriting from `AbstractIndexCreationTask`. /} + +{INFO:Side-by-Side} + +Since RavenDB 4.0, **all** index updates are side-by-side by default. The new index will replace the existing one once it becomes non-stale. If you want to force an index to swap immediately, you can use the Studio for that. + +{INFO/} + +{PANEL/} + +{PANEL:**Auto indexes**} + +Auto-indexes are **created** when queries that do **not specify an index name** are executed and, after in-depth query analysis, **no matching AUTO index is found** on the server-side. + +{NOTE The query optimizer doesn't take into account the static indexes when it determines what index should be used to handle a query. /} + +### Naming convention + +Auto-indexes can be recognized by the `Auto/` prefix in their name. Their name also contains the name of a collection that was queried, and list of fields that were required to find valid query results. + +For instance, issuing a query like this + +{CODE-TABS} +{CODE-TAB:java:Java indexes_7@Indexes/Creating.java /} +{CODE-TAB-BLOCK:sql:RQL} +from Employees +where FirstName = 'Robert' and LastName = 'King' +{CODE-TAB-BLOCK/} +{CODE-TABS/} + +will result in a creation of a index named `Auto/Employees/ByFirstNameAndLastName`. + +### Auto indexes and indexing state + +To reduce the server load, if auto-indexes are not queried for a certain amount of time defined in `Indexing.TimeToWaitBeforeMarkingAutoIndexAsIdleInMin` setting (30 minutes by default), then they will be marked as `Idle`. You can read more about the implications of marking index as `Idle` [here](../studio/database/indexes/indexes-list-view#index-state). + +Setting this configuration option to a high value may result in performance degradation due to the possibility of having a high amount of unnecessary work that is all redundant and not needed by indexes to perform. This is _not_ a recommended configuration. + +{PANEL/} + +{PANEL: If indexes exhaust system resources} + +* The indexing process utilizes machine resources to keep the data up-to-date for queries. + +* If indexing drains system resources, it may indicate one or more of the following: + * Indexes may have been defined in a way that causes inefficient processing. + * The [license](https://ravendb.net/buy) may need to be upgraded, + * Your [cloud instance](../cloud/cloud-instances#a-production-cloud-cluster) (if used) may require optimization. + * Hardware upgrades may be necessary to better support your workload. + +* Refer to the [Indexing Performance View](../studio/database/indexes/indexing-performance) in the Studio to analyze the indexing process and optimize indexes. + This view provides graphical representations and detailed statistics of all index activities at each stage. + +* Additionally, refer to the [Common indexing issues](../studio/database/indexes/indexing-performance#common-indexing-issues) section + for troubleshooting and resolving indexing challenges. + +{PANEL/} + +## Related Articles + +### Indexes +- [What are Indexes](../indexes/what-are-indexes) +- [Indexing Basics](../indexes/indexing-basics) + +### Querying +- [Basics](../indexes/querying/query-index) + +### Studio +- [Indexes: Overview](../studio/database/indexes/indexes-overview#indexes-overview) +- [Studio Index List View](../studio/database/indexes/indexes-list-view) + +--- + +### Inside RavenDB +- [Working with Indexes](https://ravendb.net/learn/inside-ravendb-book/reader/4.0/12-working-with-indexes) + +--- + +### Code Walkthrough +- [See various indexing guides in the online DEMO](https://demo.ravendb.net/) diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/creating-and-deploying.js.markdown b/Documentation/5.4/Raven.Documentation.Pages/indexes/creating-and-deploying.js.markdown new file mode 100644 index 0000000000..f8ecf2dc09 --- /dev/null +++ b/Documentation/5.4/Raven.Documentation.Pages/indexes/creating-and-deploying.js.markdown @@ -0,0 +1,310 @@ +# Creating and Deploying Indexes +--- + +{NOTE: } + +* This article explains how to create indexes in RavenDB. + For a general overview of indexes, see [What are indexes](../indexes/what-are-indexes). + +* You can either: + * [create a Static-index](../indexes/creating-and-deploying#define-a-static-index) yourself, which involves **Defining** the index and **Deploying** it to the RavenDB server, or + * let the RavenDB server [create an Auto-index](../indexes/creating-and-deploying#creating-auto-indexes) for you based on query patterns. + +* Static-indexes can be created: + * using the Client API, as outlined in this article, or + * from the [Indexes list view](../studio/database/indexes/indexes-list-view) in the Studio. + +--- + +* In this page: + * [Static-indexes](../indexes/creating-and-deploying#static-indexes) + * [Define a static-index](../indexes/creating-and-deploying#define-a-static-index) + * [Deploy a static-index](../indexes/creating-and-deploying#deploy-a-static-index) + * [Deploy single index](../indexes/creating-and-deploying#deploy-single-index) + * [Deploy multiple indexes](../indexes/creating-and-deploying#deploy-multiple-indexes) + * [Deploy syntax](../indexes/creating-and-deploying#deploy-syntax) + * [Deployment behavior](../indexes/creating-and-deploying#deployment-behavior) + * [Creating a static-index - Example](../indexes/creating-and-deploying#create-a-static-index---example) + * [Creating a static-index - using an Operation](../indexes/creating-and-deploying#create-a-static-index---using-an-operation) + * [Auto-indexes](../indexes/creating-and-deploying#auto-indexes) + * [Creating auto-indexes](../indexes/creating-and-deploying#creating-auto-indexes) + * [Disabling auto-indexes](../indexes/creating-and-deploying#disabling-auto-indexes) + +{NOTE/} + + +{PANEL: Define a static-index} + +{CONTENT-FRAME: } + +##### Static-indexes +--- + +* Indexes that are explicitly **created by the user** are called `static` indexes. +* Static-indexes can perform calculations, data conversions, and other processes behind the scenes. + This reduces the workload at query time by offloading these costly operations to the indexing phase. +* To query with a static-index, you must explicitly specify the index in the query definition. + For more details, see [Querying an index](../indexes/querying/query-index). + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Define a static-index using a custom class +--- + +* To define a static-index using a custom class, extend the `AbstractJavaScriptIndexCreationTask` class. +* This method is recommended over the [Creating an index using an operation](../indexes/creating-and-deploying#create-a-static-index---using-an-operation) method + for its simplified index definition, offering a straightforward way to define the index. +{CODE:nodejs indexes_1@indexes/creating.js /} +* A complete example of creating a static-index is provided [below](../indexes/creating-and-deploying#create-a-static-index---example). + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Naming convention +--- + +* Static-index class names follow a single naming convention: + Each `_` in the class name is translated to `/` in the index name on the server. +* In the above example, the index class name is `Orders_ByTotal`. + The name of the index that will be generated on the server will be: `Orders/ByTotal`. + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Customizing configuration +--- + +* You can set various [indexing configuration](../server/configuration/indexing-configuration) values within the index definition. +* Setting a configuration value within the index will override the matching indexing configuration values set at the server or database level. +{CODE:nodejs indexes_2@indexes/creating.js /} + +{CONTENT-FRAME/} +{PANEL/} + +{PANEL: Deploy a static-index} + +* To begin indexing data, the index must be deployed to the server. +* This section provides options for deploying indexes that inherit from `AbstractJavaScriptIndexCreationTask`. +* To create and deploy an index using the `IndexDefinition` class via `PutIndexesOperation`, + see [Creating a static-index - using an Operation](../indexes/creating-and-deploying#create-a-static-index---using-an-operation). + +--- + +{CONTENT-FRAME: } + +##### Deploy single index +--- + +* Use `execute()` or `executeIndex()` to deploy a single index. +* The following examples deploy index `Ordes/ByTotal` to the default database defined in your _DocumentStore_ object. + See the [syntax](../indexes/creating-and-deploying#deploy-syntax) section below for all available overloads. + +{CODE:nodejs indexes_3@indexes/creating.js /} +{CODE:nodejs indexes_4@indexes/creating.js /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Deploy multiple indexes +--- + +* Use `executeIndexes()` or `IndexCreation.createIndexes()` to deploy multiple indexes. +* The `IndexCreation.createIndexes` method attempts to create all indexes in a single request. + If it fails, it will repeat the execution by calling the `execute` method for each index, one by one, + in separate requests. +* The following examples deploy indexes `Ordes/ByTotal` and `Employees/ByLastName` to the default database defined in your _DocumentStore_ object. + See the [syntax](../indexes/creating-and-deploying#deploy-syntax) section below for all available overloads. + +{CODE:nodejs indexes_5@indexes/creating.js /} +{CODE:nodejs indexes_6@indexes/creating.js /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Deploy syntax +--- + +{CODE:nodejs syntax@indexes/creating.js /} + +| Parameter | Type | Description | +|--------------------|-----------------------|-------------------------------------------------------------------------------------------------------------------| +| **store** | `object` | Your document store object. | +| **conventions** | `DocumentConventions` | The [Conventions](../client-api/configuration/conventions) used by the document store. | +| **database** | `string` | The target database to deploy the index to. If not specified, the default database set on the store will be used. | +| **index** | `object` | The index object to deploy. | +| **indexes** | `object[]` | A list of index objects to deploy. | + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Deployment behavior +--- + +{INFO: } + +###### Deployment mode: +--- + +* When your database spans multiple nodes, + you can choose between **Rolling** index deployment or **Parallel** index deployment. +* Rolling deployment applies the index to one node at a time, + while Parallel deployment deploys the index on all nodes simultaneously. +* Learn more in [Rolling index deployment](../indexes/rolling-index-deployment). + +{INFO/} + +{SAFE: } + +###### When the index you are deploying already exists on the server: +--- + +* **If the index definition is updated**: + * RavenDB uses a side-by-side strategy for all index updates. + * When an existing index definition is modified, RavenDB creates a new index with the updated definition. + The new index will replace the existing index once it becomes non-stale. + * If you want to swap the indexes immediately, you can do so through the Studio. + For more details, see [Side by side indexing](../studio/database/indexes/indexes-list-view#indexes-list-view---side-by-side-indexing). +* **If the index definition is unchanged**: + * If the definition of the index being deployed is identical to the one on the server, + the existing index will not be overwritten. + * The indexed data will remain intact, and the indexing process will not restart. + +{SAFE/} +{CONTENT-FRAME/} +{PANEL/} + +{PANEL: Create a static-index - Example} + +{CODE:nodejs indexes_7@indexes/creating.js /} + +{PANEL/} + +{PANEL: Create a static-index - using an Operation} + +* An index can also be defined and deployed using the [PutIndexesOperation](../client-api/operations/maintenance/indexes/put-indexes) maintenance operation. + +* When using this operation: + + * Unlike the [naming convention](../indexes/creating-and-deploying#naming-convention) used with indexes inheriting from `AbstractJavaScriptIndexCreationTask`, + you can choose any string-based name for the index. + However, when querying, you must use that string-based name rather than the index class type. + + * You can also modify various low-level settings available in the [IndexDefinition](../client-api/operations/maintenance/indexes/put-indexes#put-indexes-operation-with-indexdefinition) class. + +* Consider using this operation only if inheriting from `AbstractJavaScriptIndexCreationTask` is not an option. + +* For a detailed explanation and examples, refer to the dedicated article: [Put Indexes Operation](../client-api/operations/maintenance/indexes/put-indexes). + +{PANEL/} + + +{PANEL: Creating auto-indexes} + +{CONTENT-FRAME: } + +##### Auto-indexes creation +--- + +* Indexes **created by the server** are called `dynamic` or `auto` indexes. +* Auto-indexes are created when all of the following conditions are met: + * A query is issued without specifying an index (a dynamic query). + * The query includes a filtering condition. + * No suitable auto-index exists that can satisfy the query. + * Creation of auto-indexes has not been disabled. +* For such queries, RavenDB's Query Optimizer searches for an existing auto-index that can satisfy the query. + If no suitable auto-index is found, RavenDB will either create a new auto-index or optimize an existing auto-index. + (Static-indexes are not taken into account when determining which auto-index should handle the query). +* Note: dynamic queries can be issued either when [querying](../studio/database/queries/query-view#query-view) or when [patching](../studio/database/documents/patch-view#patch-configuration). +* Over time, RavenDB automatically adjusts and merges auto-indexes to efficiently serve your queries. + For more details, see [Query a collection - with filtering (dynamic query)](../client-api/session/querying/how-to-query#dynamicQuery). + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Naming convention +--- + +* Auto-indexes are easily identified by their names, which start with the `Auto/` prefix. +* Their name also includes the name of the queried collection and a list of fields used in the query predicate to filter matching results. +* For example, issuing the following query: + {CODE-TABS} + {CODE-TAB:nodejs:Query indexes_8@indexes/creating.js /} + {CODE-TAB-BLOCK:sql:RQL} +from Employees +where FirstName = "Robert" and LastName = "King" + {CODE-TAB-BLOCK/} + {CODE-TABS/} + will result in the creation of an auto-index named `Auto/Employees/ByFirstNameAndLastName`. + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +##### Auto-index idle state +--- + +* To reduce server load, an auto-index is marked as `idle` when it hasn't been used for a while. + Specifically, if the time difference between the last time the auto-index was queried + and the last time a query was made on the database (using any index) exceeds the configured threshold (30 minutes by default), + the auto-index will be marked as `idle`. +* This is done in order to avoid marking indexes as idle for databases that were offline for a long period of time, + as well as for databases that were just restored from a snapshot or a backup. +* To set the time before marking an index as idle, use the + [Indexing.TimeToWaitBeforeDeletingAutoIndexMarkedAsIdleInHrs](../server/configuration/indexing-configuration#indexing.timetowaitbeforedeletingautoindexmarkedasidleinhrs) configuration key. + Setting this value too high is not recommended, as it may lead to performance degradation by causing unnecessary and redundant work for the indexes. +* An `idle` auto-index will resume its work and return to `normal` state upon its next query, + or when resetting the index. +* If not resumed, the idle auto-index will be deleted by the server after the time period defined in the + [Indexing.TimeToWaitBeforeDeletingAutoIndexMarkedAsIdleInHrs](../server/configuration/indexing-configuration#indexing.timetowaitbeforedeletingautoindexmarkedasidleinhrs) configuration key + (72 hours by default). + +{CONTENT-FRAME/} +{PANEL/} + +{PANEL: Disabling auto-indexes} + +**Why disable**: + +* Disabling auto-index creation prevents the accidental deployment of resource-consuming auto-indexes that may result from one-time, ad-hoc queries issued from the Studio. +* In production environments, disabling this feature helps avoid the creation and background execution of expensive indexes. + +**How to disable**: + +* You can disable auto-indexes by setting the [Indexing.DisableQueryOptimizerGeneratedIndexes](../server/configuration/indexing-configuration#indexing.disablequeryoptimizergeneratedindexes) configuration key. + This will affect all queries made both from the **Client API** and the **Studio**. + +* Alternatively, you can disable auto-indexes from the Studio. + However, this will affect queries made only from the **Studio**. + + * To disable auto-index creation for a specific query made from the query view, see these [Query settings](../studio/database/queries/query-view#query-settings). + * To disable auto-index creation for a specific query made from the patch view, see these [Patch settings](../studio/database/documents/patch-view#patch-settings). + * Disabling auto-index creation for ALL queries made on a database can be configured in the [Studio configuration view](../studio/database/settings/studio-configuration#disabling-auto-index-creation-on-studio-queries-or-patches). + +{PANEL/} + +## Related Articles + +### Indexes +- [What are Indexes](../indexes/what-are-indexes) +- [Indexing Basics](../indexes/indexing-basics) + +### Operations +- [Put Indexes Operation](../client-api/operations/maintenance/indexes/put-indexes) + +### Querying +- [Query Overview](../client-api/session/querying/how-to-query) +- [Querying an Index](../indexes/querying/query-index) + +### Studio +- [Indexes: Overview](../studio/database/indexes/indexes-overview#indexes-overview) +- [Studio Index List View](../studio/database/indexes/indexes-list-view) + +--- + +### Inside RavenDB +- [Working with Indexes](https://ravendb.net/learn/inside-ravendb-book/reader/4.0/12-working-with-indexes) + +--- + +### Code Walkthrough +- [See various indexing guides in the online DEMO](https://demo.ravendb.net/) diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/images/index-query-results-1.png b/Documentation/5.4/Raven.Documentation.Pages/indexes/images/index-query-results-1.png new file mode 100644 index 0000000000..0b9acecd0d Binary files /dev/null and b/Documentation/5.4/Raven.Documentation.Pages/indexes/images/index-query-results-1.png differ diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/images/index-query-results-2.png b/Documentation/5.4/Raven.Documentation.Pages/indexes/images/index-query-results-2.png new file mode 100644 index 0000000000..3044311080 Binary files /dev/null and b/Documentation/5.4/Raven.Documentation.Pages/indexes/images/index-query-results-2.png differ diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/map-reduce-indexes.dotnet.markdown b/Documentation/5.4/Raven.Documentation.Pages/indexes/map-reduce-indexes.dotnet.markdown index 2d24d5b7c3..5650275b98 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/indexes/map-reduce-indexes.dotnet.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/indexes/map-reduce-indexes.dotnet.markdown @@ -232,7 +232,7 @@ The map-reduce output documents are configured with these properties of | - | - | - | | **OutputReduceToCollection** | `string` | Collection name for the output documents. | | **PatternReferencesCollectionName** | `string` | Optional collection name for the reference documents - by default it is `/References`. | -| **PatternForOutputReduceToCollectionReferences** | `string` / `Expression>` | Document ID format for reference documents. This ID references the fields of the reduce function output, which determines how the output documents are aggregated. The type of this parameter is different depending on if the index is created using [IndexDefinition](../indexes/creating-and-deploying#using-maintenance-operations) or [AbstractIndexCreationTask](../indexes/creating-and-deploying#using-abstractindexcreationtask). | +| **PatternForOutputReduceToCollectionReferences** | `string` / `Expression>` | Document ID format for reference documents. This ID references the fields of the reduce function output, which determines how the output documents are aggregated. The type of this parameter is different depending on if the index is created using [IndexDefinition](../client-api/operations/maintenance/indexes/put-indexes#put-indexes-operation) or [AbstractIndexCreationTask](../indexes/creating-and-deploying#define-a-static-index-using-a-custom-class). | To index artificial documents in strongly typed syntax (LINQ), you will need the type of reference documents: diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/map-reduce-indexes.js.markdown b/Documentation/5.4/Raven.Documentation.Pages/indexes/map-reduce-indexes.js.markdown index 8b346c399b..10b9402758 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/indexes/map-reduce-indexes.js.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/indexes/map-reduce-indexes.js.markdown @@ -136,7 +136,7 @@ The map-reduce output documents are configured with these properties of | - | - | - | | **outputReduceToCollection** | `string` | Collection name for the output documents. | | **patternReferencesCollectionName** | `string` | Optional collection name for the reference documents - by default it is `/References`. | -| **patternForOutputReduceToCollectionReferences** | `string` | Document ID format for reference documents. This ID references the fields of the reduce function output, which determines how the output documents are aggregated. The type of this parameter is different depending on if the index is created using [IndexDefinition](../indexes/creating-and-deploying#using-maintenance-operations) or [AbstractIndexCreationTask](../indexes/creating-and-deploying#using-abstractindexcreationtask). | +| **patternForOutputReduceToCollectionReferences** | `string` | Document ID format for reference documents. This ID references the fields of the reduce function output, which determines how the output documents are aggregated. The type of this parameter is different depending on if the index is created using [IndexDefinition](../client-api/operations/maintenance/indexes/put-indexes#put-indexes-operation) or [AbstractJavaScriptIndexCreationTask](../indexes/creating-and-deploying#define-a-static-index-using-a-custom-class). | ### Examples diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.dotnet.markdown b/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.dotnet.markdown index 6eeb4dfcf9..d8057b5b92 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.dotnet.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.dotnet.markdown @@ -1,61 +1,203 @@ -# What Indexes Are +# What are Indexes --- {NOTE: } -* Indexes are server-side functions that define which fields and what values documents can be searched on. - They are _the only way_ to satisfy queries in RavenDB. +* Indexes in RavenDB enable efficient querying by processing raw data and providing fast query results without scanning the entire dataset each and every time. + Learn more in the [Indexes overview](../indexes/what-are-indexes#indexes-overview) below. -* The indexing process is performed in the background and is triggered whenever data is added or changed. - This approach allows the server to respond quickly even when large amounts of data have changed and avoid - costly table scan operations. - -* The core of every index is its mapping function, that utilizes LINQ-like syntax. - The result of this mapping is converted to a [Lucene](http://lucene.apache.org/) index entry - which is persisted for future use to avoid re-indexing each time the query is issued and keep - response times minimal. - -* This page shows a basic example for the creation, deployment, and usage of an index. - Additional examples are available, among other places, on the [Indexing Basics](../indexes/indexing-basics) - and [Map Indexes](../indexes/map-indexes) pages. +* This page provides a [Basic example](../indexes/what-are-indexes#basic-example) of creating, deploying, and querying an index. + Additional examples can be found in [Creating and deploying indexes](../indexes/creating-and-deploying), [Map Indexes](../indexes/map-indexes), + and many other articles under the "Indexes" menu. * In this page: - * [Basic Example](../indexes/what-are-indexes#basic-example) - * [Remarks](../indexes/what-are-indexes#remarks) + * [Indexes overview](../indexes/what-are-indexes#indexes-overview) + * [Types of indexes](../indexes/what-are-indexes#types-of-indexes) + * [Basic example](../indexes/what-are-indexes#basic-example) + * [Understanding index query results](../indexes/what-are-indexes#understanding-index-query-results) + * [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources) {NOTE/} -{PANEL: Basic Example} +{PANEL: Indexes overview} + +**Indexes are fundamental**: + +* Indexes are fundamental to RavenDB’s query execution, enabling efficient query performance by processing the underlying data and delivering faster results. +* ALL queries in RavenDB use an index to deliver results and ensure optimal performance. + Learn more in [Queries always provide results using an index](../client-api/session/querying/how-to-query#queries-always-provide-results-using-an-index). + +--- + +**Main concepts**: + +* When discussing indexes in RavenDB, three key components come into play: + * The index definition + * The indexing process + * The indexed data + +* Each of these components is described in detail in [Indexes - The moving parts](../studio/database/indexes/indexes-overview#indexes---the-moving-parts). + +--- + +**The indexing process**: + +* The indexing process iterates over the raw documents, creating an **index-entry** for each document that is processed. + (Usually a single index-entry is created per raw document, unless working with a [fanout index](../indexes/indexing-nested-data#fanout-index---multiple-index-entries-per-document)). +* Each index-entry contains **index-fields**, and each index-field contains content (**index-terms**) that was generated from the raw documents, + as defined by the index definition and depending on the [analyzer](../indexes/using-analyzers) used. +* A map is built between the indexed-terms and the documents they originated from, + enabling you to query for documents based on the indexed data. + +--- + +**Automatic data processing**: -In the below example we create an index that maps documents from the `Employees` -[collection](../client-api/faq/what-is-a-collection) and enables querying by `FirstName`, -`LastName`, or both. +* Once defined and deployed, an index will initially process the entire dataset. + After that, the index will only process documents that were modified, added or deleted. + This happens automatically without requiring direct user intervention. +* For example, if changes are made to documents in the "Orders" collection, + all indexes defined for the "Orders" collection will be triggered to update the index with the new data. +* This approach helps avoid costly table scans, allows the server to respond quickly, + and reduces the load on queries while optimizing machine resource usage. -- First, we need to create an index. - One way to create it is to use the `AbstractIndexCreationTask`, but there are other ways - available as well (you can read about them [here](../indexes/creating-and-deploying)). - {CODE indexes_1@Indexes/WhatAreIndexes.cs /} +--- + +**Background operation**: + +* RavenDB indexes are designed to run asynchronously in the background. +* The indexing process does not block or interrupt database operations, such as writing data or running queries, + though queries may temporarily return [stale results](../indexes/stale-indexes) until the index is fully updated. + +--- + +**Separate storage**: + +* Indexes store their processed data separately, ensuring that the raw data remains unaffected. + This separation helps maintain the integrity of the raw data while allowing the index to optimize query performance. + +* If system resources become strained due to indexing, it may require adjustments to the index design, hardware, or other factors. + Learn more in [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources). + +{PANEL/} + +{PANEL: Types of indexes} -- The next step is to send an index to a server (more about index deployment options can - be found [here](../indexes/creating-and-deploying)) so the indexing process can start indexing documents. - {CODE indexes_2@Indexes/WhatAreIndexes.cs /} +* Indexes in RavenDB are categorized along the following axes: + * **Auto** indexes -vs- **Static** indexes + * **Map** indexes -vs- **Map-Reduce** indexes + * **Single-Collection** (Single-Map) indexes -vs- **Multi-Collection** (Multi-Map) indexes -- Now, our index can be queried and indexed results can be returned. - {CODE indexes_3@Indexes/WhatAreIndexes.cs /} +* For a detailed description of each type, refer to section [Index types](../studio/database/indexes/indexes-overview#index-types). {PANEL/} -{PANEL: Remarks} +{PANEL: Basic example} + +In this example we create a static-index that indexes content from documents in the `Employees` [collection](../client-api/faq/what-is-a-collection). +This allows querying for _Employee_ documents by any of the index-fields (`FullName`, `LastName`, `Country`). + +{CONTENT-FRAME: } + +#### Define the index + +* The first step is to define the index. + One way to do this is by inheriting from `AbstractIndexCreationTask`. + Learn more in [Define a static-index using a custom class](../indexes/creating-and-deploying#define-a-static-index-using-a-custom-class). +* Other methods to create a static-index are: + * [Creating a static-index using an Operation](../client-api/operations/maintenance/indexes/put-indexes) + * [Creating a static-index from the Studio](../studio/database/indexes/indexes-list-view) + +{CODE indexes_1@Indexes/WhatAreIndexes.cs /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } -{WARNING:Remember} -A frequent mistake is to handle indexes as SQL Views, but they are not analogous. -The **result of a query for the given index is a full document**, not only the fields -that were indexed. +#### Deploy the index + +* The next step is to deploy the index to the RavenDB server. + One way to do this is by calling `Execute()` on the index instance. +* Additional methods for deploying static-indexes are described in [Deploy a static index](../indexes/creating-and-deploying#deploy-a-static-index). +* Once deployed, the indexing process will start indexing documents. + +{CODE indexes_2@Indexes/WhatAreIndexes.cs /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +#### Query the index + +* Now you can query the _Employees_ collection using the index. + In this example we query for _Employee_ documents, **filtering results based on index-fields** `LastName` and `Country`. + The results will include only the _Employee_ documents that match the query predicate. +* For detailed guidance on querying with an index, refer to the [Querying an index](../indexes/querying/query-index). + +{CODE indexes_3@Indexes/WhatAreIndexes.cs /} + +{CONTENT-FRAME/} +{PANEL/} + +{PANEL: Understanding index query results} + +{WARNING: } + +A common mistake is treating indexes like SQL Views, but they are not analogous. +The results of a query for a given index are the **full raw documents** that match the query predicate, +and not just the indexed fields. + +This behavior can be changed by applying [Projections](../indexes/querying/projections), +which let you project the query results into selected fields instead of returning the entire document. -This behavior can be altered by [storing](../indexes/storing-data-in-index) fields -and applying [projections](../indexes/querying/projections). {WARNING/} +--- + +#### Viewing the resulting documents: + +For example, the results shown in the following image are the **documents** that match the query predicate. + +![Index query results - documents](images/index-query-results-1.png "Query results - Documents") + +1. This is the index query. + The query predicate filters the resulting documents based on the content of the index-fields. +2. Each row in the results represents a **matching document**. +3. In this example, the `LastName`, `FirstName`, `Title`, etc., are the raw **document-fields**. + +--- + +#### Viewing the index-entries: + +If you wish to **view the index-entries** that compose the index itself, +you can enable the option to show "raw index entries" instead of the matching documents. + +![Index query results - index entries](images/index-query-results-2.png "Query results - Index Entries") + +1. Query the index (no filtering is applied in this example). +2. Click the "Settings" button and toggle on "Show raw index-entries instead of matching documents". +3. Each row in the results represents an **index-entry**. +4. In this example, the `Country`, `FullName`, and `LastName` columns are the **index-fields**, + which were defined in the index definition. +5. This a **term**. + In this example, `usa` is a term generated by the analyzer for index-field `Country` from document `employees/4-a`. + +{PANEL/} + +{PANEL: If indexes exhaust system resources} + +* The indexing process utilizes machine resources to keep the data up-to-date for queries. + +* If indexing drains system resources, it may indicate one or more of the following: + * Indexes may have been defined in a way that causes inefficient processing. + * The [license](https://ravendb.net/buy) may need to be upgraded, + * Your [cloud instance](../cloud/cloud-instances#a-production-cloud-cluster) (if used) may require optimization. + * Hardware upgrades may be necessary to better support your workload. + +* Refer to the [Indexing Performance View](../studio/database/indexes/indexing-performance) in the Studio to monitor and analyze the indexing process. + This view provides graphical representations and detailed statistics of all index activities at each stage. + +* Additionally, refer to the [Common indexing issues](../studio/database/indexes/indexing-performance#common-indexing-issues) section + for troubleshooting and resolving indexing challenges. + {PANEL/} ## Related Articles @@ -67,7 +209,6 @@ and applying [projections](../indexes/querying/projections). ### Indexes - [Creating and Deploying Indexes](../indexes/creating-and-deploying) -- [Indexing Basics](../indexes/indexing-basics) - [Query an Index](../indexes/querying/query-index) - [Stale Indexes](../indexes/stale-indexes) diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.java.markdown b/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.java.markdown index 51cd1a1985..666947a082 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.java.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.java.markdown @@ -1,61 +1,203 @@ -# What Indexes Are +# What are Indexes --- {NOTE: } -* Indexes are server-side functions that define which fields and what values documents can be searched on. - They are _the only way_ to satisfy queries in RavenDB. +* Indexes in RavenDB enable efficient querying by processing raw data and providing fast query results without scanning the entire dataset each and every time. + Learn more in the [Indexes overview](../indexes/what-are-indexes#indexes-overview) below. -* The indexing process is performed in the background and is triggered whenever data is added or changed. - This approach allows the server to respond quickly even when large amounts of data have changed and avoid - costly table scan operations. - -* The core of every index is its mapping function, that utilizes LINQ-like syntax. - The result of this mapping is converted to a [Lucene](http://lucene.apache.org/) index entry - which is persisted for future use to avoid re-indexing each time the query is issued and keep - response times minimal. - -* This page shows a basic example for the creation, deployment, and usage of an index. - Additional examples are available, among other places, on the [Indexing Basics](../indexes/indexing-basics) - and [Map Indexes](../indexes/map-indexes) pages. +* This page provides a [Basic example](../indexes/what-are-indexes#basic-example) of creating, deploying, and querying an index. + Additional examples can be found in [Creating and deploying indexes](../indexes/creating-and-deploying), [Map Indexes](../indexes/map-indexes), + and many other articles under the "Indexes" menu. * In this page: - * [Basic Example](../indexes/what-are-indexes#basic-example) - * [Remarks](../indexes/what-are-indexes#remarks) + * [Indexes overview](../indexes/what-are-indexes#indexes-overview) + * [Types of indexes](../indexes/what-are-indexes#types-of-indexes) + * [Basic example](../indexes/what-are-indexes#basic-example) + * [Understanding index query results](../indexes/what-are-indexes#understanding-index-query-results) + * [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources) {NOTE/} -{PANEL: Basic Example} +{PANEL: Indexes overview} + +**Indexes are fundamental**: + +* Indexes are fundamental to RavenDB’s query execution, enabling efficient query performance by processing the underlying data and delivering faster results. +* ALL queries in RavenDB use an index to deliver results and ensure optimal performance. + Learn more in [Queries always provide results using an index](../client-api/session/querying/how-to-query#queries-always-provide-results-using-an-index). + +--- + +**Main concepts**: + +* When discussing indexes in RavenDB, three key components come into play: + * The index definition + * The indexing process + * The indexed data + +* Each of these components is described in detail in [Indexes - The moving parts](../studio/database/indexes/indexes-overview#indexes---the-moving-parts). + +--- + +**The indexing process**: + +* The indexing process iterates over the raw documents, creating an **index-entry** for each document that is processed. + (Usually a single index-entry is created per raw document, unless working with a [fanout index](../indexes/indexing-nested-data#fanout-index---multiple-index-entries-per-document)). +* Each index-entry contains **index-fields**, and each index-field contains content (**index-terms**) that was generated from the raw documents, + as defined by the index definition and depending on the [analyzer](../indexes/using-analyzers) used. +* A map is built between the indexed-terms and the documents they originated from, + enabling you to query for documents based on the indexed data. + +--- + +**Automatic data processing**: + +* Once defined and deployed, an index will initially process the entire dataset. + After that, the index will only process documents that were modified, added or deleted. + This happens automatically without requiring direct user intervention. +* For example, if changes are made to documents in the "Orders" collection, + all indexes defined for the "Orders" collection will be triggered to update the index with the new data. +* This approach helps avoid costly table scans, allows the server to respond quickly, + and reduces the load on queries while optimizing machine resource usage. + +--- + +**Background operation**: + +* RavenDB indexes are designed to run asynchronously in the background. +* The indexing process does not block or interrupt database operations, such as writing data or running queries, + though queries may temporarily return [stale results](../indexes/stale-indexes) until the index is fully updated. + +--- + +**Separate storage**: + +* Indexes store their processed data separately, ensuring that the raw data remains unaffected. + This separation helps maintain the integrity of the raw data while allowing the index to optimize query performance. + +* If system resources become strained due to indexing, it may require adjustments to the index design, hardware, or other factors. + Learn more in [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources). + +{PANEL/} + +{PANEL: Types of indexes} + +* Indexes in RavenDB are categorized along the following axes: + * **Auto** indexes -vs- **Static** indexes + * **Map** indexes -vs- **Map-Reduce** indexes + * **Single-Collection** (Single-Map) indexes -vs- **Multi-Collection** (Multi-Map) indexes + +* For a detailed description of each type, refer to section [Index types](../studio/database/indexes/indexes-overview#index-types). + +{PANEL/} + +{PANEL: Basic example} + +In this example we create a static-index that indexes content from documents in the `Employees` [collection](../client-api/faq/what-is-a-collection). +This allows querying for _Employee_ documents by any of the index-fields (`FullName`, `LastName`, `Country`). + +{CONTENT-FRAME: } + +#### Define the index + +* The first step is to define the index. + One way to do this is by extending the `AbstractIndexCreationTask` class. + Learn more in [Define a static-index using a custom class](../indexes/creating-and-deploying#using-abstractindexcreationtask). +* Other methods to create a static-index are: + * [Creating a static-index using an Operation](../client-api/operations/maintenance/indexes/put-indexes) + * [Creating a static-index from the Studio](../studio/database/indexes/indexes-list-view) + +{CODE:java indexes_1@Indexes/WhatAreIndexes.java /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } -In the below example we create an index that maps documents from the `Employees` -[collection](../client-api/faq/what-is-a-collection) and enables querying by `FirstName`, -`LastName`, or both. +#### Deploy the index -- First, we need to create an index. - One way to create it is to use the `AbstractIndexCreationTask`, but there are other ways - available as well (you can read about them [here](../indexes/creating-and-deploying)). - {CODE:java indexes_1@Indexes/WhatAreIndexes.java /} +* The next step is to deploy the index to the RavenDB server. + One way to do this is by calling `execute()` on the index instance. +* Additional methods for deploying static-indexes are described in [Deploy a static index](../indexes/creating-and-deploying#deploy-a-static-index). +* Once deployed, the indexing process will start indexing documents. -- The next step is to send an index to a server (more about index deployment options can - be found [here](../indexes/creating-and-deploying)) so the indexing process can start indexing documents. - {CODE:java indexes_2@Indexes/WhatAreIndexes.java /} +{CODE:java indexes_2@Indexes/WhatAreIndexes.java /} -- Now, our index can be queried and indexed results can be returned. - {CODE:java indexes_3@Indexes/WhatAreIndexes.java /} +{CONTENT-FRAME/} +{CONTENT-FRAME: } +#### Query the index + +* Now you can query the _Employees_ collection using the index. + In this example we query for _Employee_ documents, **filtering results based on index-fields** `LastName` and `Country`. + The results will include only the _Employee_ documents that match the query predicate. +* For detailed guidance on querying with an index, refer to the [Querying an index](../indexes/querying/query-index). + +{CODE:java indexes_3@Indexes/WhatAreIndexes.java /} + +{CONTENT-FRAME/} {PANEL/} -{PANEL: Remarks} +{PANEL: Understanding index query results} + +{WARNING: } -{WARNING:Remember} -A frequent mistake is to handle indexes as SQL Views, but they are not analogous. -The **result of a query for the given index is a full document**, not only the fields -that were indexed. +A common mistake is treating indexes like SQL Views, but they are not analogous. +The results of a query for a given index are the **full raw documents** that match the query predicate, +and not just the indexed fields. + +This behavior can be changed by applying [Projections](../indexes/querying/projections), +which let you project the query results into selected fields instead of returning the entire document. -This behavior can be altered by [storing](../indexes/storing-data-in-index) fields -and applying [projections](../indexes/querying/projections). {WARNING/} +--- + +#### Viewing the resulting documents: + +For example, the results shown in the following image are the **documents** that match the query predicate. + +![Index query results - documents](images/index-query-results-1.png "Query results - Documents") + +1. This is the index query. + The query predicate filters the resulting documents based on the content of the index-fields. +2. Each row in the results represents a **matching document**. +3. In this example, the `LastName`, `FirstName`, `Title`, etc., are the raw **document-fields**. + +--- + +#### Viewing the index-entries: + +If you wish to **view the index-entries** that compose the index itself, +you can enable the option to show "raw index entries" instead of the matching documents. + +![Index query results - index entries](images/index-query-results-2.png "Query results - Index Entries") + +1. Query the index (no filtering is applied in this example). +2. Click the "Settings" button and toggle on "Show raw index-entries instead of matching documents". +3. Each row in the results represents an **index-entry**. +4. In this example, the `Country`, `FullName`, and `LastName` columns are the **index-fields**, + which were defined in the index definition. +5. This a **term**. + In this example, `usa` is a term generated by the analyzer for index-field `Country` from document `employees/4-a`. + +{PANEL/} + +{PANEL: If indexes exhaust system resources} + +* The indexing process utilizes machine resources to keep the data up-to-date for queries. + +* If indexing drains system resources, it may indicate one or more of the following: + * Indexes may have been defined in a way that causes inefficient processing. + * The [license](https://ravendb.net/buy) may need to be upgraded, + * Your [cloud instance](../cloud/cloud-instances#a-production-cloud-cluster) (if used) may require optimization. + * Hardware upgrades may be necessary to better support your workload. + +* Refer to the [Indexing Performance View](../studio/database/indexes/indexing-performance) in the Studio to monitor and analyze the indexing process. + This view provides graphical representations and detailed statistics of all index activities at each stage. + +* Additionally, refer to the [Common indexing issues](../studio/database/indexes/indexing-performance#common-indexing-issues) section + for troubleshooting and resolving indexing challenges. + {PANEL/} ## Related Articles @@ -67,7 +209,6 @@ and applying [projections](../indexes/querying/projections). ### Indexes - [Creating and Deploying Indexes](../indexes/creating-and-deploying) -- [Indexing Basics](../indexes/indexing-basics) - [Query an Index](../indexes/querying/query-index) - [Stale Indexes](../indexes/stale-indexes) diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.js.markdown b/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.js.markdown index 88803c1b08..7f91d3d882 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.js.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.js.markdown @@ -1,61 +1,203 @@ -# What Indexes Are +# What are Indexes --- {NOTE: } -* Indexes are server-side functions that define which fields and what values documents can be searched on. - They are _the only way_ to satisfy queries in RavenDB. +* Indexes in RavenDB enable efficient querying by processing raw data and providing fast query results without scanning the entire dataset each and every time. + Learn more in the [Indexes overview](../indexes/what-are-indexes#indexes-overview) below. -* The indexing process is performed in the background and is triggered whenever data is added or changed. - This approach allows the server to respond quickly even when large amounts of data have changed and avoid - costly table scan operations. - -* The core of every index is its mapping function, that utilizes LINQ-like syntax. - The result of this mapping is converted to a [Lucene](http://lucene.apache.org/) index entry - which is persisted for future use to avoid re-indexing each time the query is issued and keep - response times minimal. - -* This page shows a basic example for the creation, deployment, and usage of an index. - Additional examples are available, among other places, on the [Indexing Basics](../indexes/indexing-basics) - and [Map Indexes](../indexes/map-indexes) pages. +* This page provides a [Basic example](../indexes/what-are-indexes#basic-example) of creating, deploying, and querying an index. + Additional examples can be found in [Creating and deploying indexes](../indexes/creating-and-deploying), [Map Indexes](../indexes/map-indexes), + and many other articles under the "Indexes" menu. * In this page: - * [Basic Example](../indexes/what-are-indexes#basic-example) - * [Remarks](../indexes/what-are-indexes#remarks) + * [Indexes overview](../indexes/what-are-indexes#indexes-overview) + * [Types of indexes](../indexes/what-are-indexes#types-of-indexes) + * [Basic example](../indexes/what-are-indexes#basic-example) + * [Understanding index query results](../indexes/what-are-indexes#understanding-index-query-results) + * [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources) {NOTE/} -{PANEL: Basic Example} +{PANEL: Indexes overview} + +**Indexes are fundamental**: + +* Indexes are fundamental to RavenDB’s query execution, enabling efficient query performance by processing the underlying data and delivering faster results. +* ALL queries in RavenDB use an index to deliver results and ensure optimal performance. + Learn more in [Queries always provide results using an index](../client-api/session/querying/how-to-query#queries-always-provide-results-using-an-index). + +--- + +**Main concepts**: + +* When discussing indexes in RavenDB, three key components come into play: + * The index definition + * The indexing process + * The indexed data + +* Each of these components is described in detail in [Indexes - The moving parts](../studio/database/indexes/indexes-overview#indexes---the-moving-parts). + +--- + +**The indexing process**: + +* The indexing process iterates over the raw documents, creating an **index-entry** for each document that is processed. + (Usually a single index-entry is created per raw document, unless working with a [fanout index](../indexes/indexing-nested-data#fanout-index---multiple-index-entries-per-document)). +* Each index-entry contains **index-fields**, and each index-field contains content (**index-terms**) that was generated from the raw documents, + as defined by the index definition and depending on the [analyzer](../indexes/using-analyzers) used. +* A map is built between the indexed-terms and the documents they originated from, + enabling you to query for documents based on the indexed data. + +--- + +**Automatic data processing**: + +* Once defined and deployed, an index will initially process the entire dataset. + After that, the index will only process documents that were modified, added or deleted. + This happens automatically without requiring direct user intervention. +* For example, if changes are made to documents in the "Orders" collection, + all indexes defined for the "Orders" collection will be triggered to update the index with the new data. +* This approach helps avoid costly table scans, allows the server to respond quickly, + and reduces the load on queries while optimizing machine resource usage. + +--- + +**Background operation**: + +* RavenDB indexes are designed to run asynchronously in the background. +* The indexing process does not block or interrupt database operations, such as writing data or running queries, + though queries may temporarily return [stale results](../indexes/stale-indexes) until the index is fully updated. + +--- + +**Separate storage**: + +* Indexes store their processed data separately, ensuring that the raw data remains unaffected. + This separation helps maintain the integrity of the raw data while allowing the index to optimize query performance. + +* If system resources become strained due to indexing, it may require adjustments to the index design, hardware, or other factors. + Learn more in [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources). + +{PANEL/} + +{PANEL: Types of indexes} + +* Indexes in RavenDB are categorized along the following axes: + * **Auto** indexes -vs- **Static** indexes + * **Map** indexes -vs- **Map-Reduce** indexes + * **Single-Collection** (Single-Map) indexes -vs- **Multi-Collection** (Multi-Map) indexes + +* For a detailed description of each type, refer to section [Index types](../studio/database/indexes/indexes-overview#index-types). + +{PANEL/} + +{PANEL: Basic example} + +In this example we create a static-index that indexes content from documents in the `Employees` [collection](../client-api/faq/what-is-a-collection). +This allows querying for _Employee_ documents by any of the index-fields (`FullName`, `LastName`, `Country`). + +{CONTENT-FRAME: } + +#### Define the index + +* The first step is to define the index. + One way to do this is by extending `AbstractJavaScriptIndexCreationTask`. + Learn more in [Define a static-index using a custom class](../indexes/creating-and-deploying#define-a-static-index-using-a-custom-class). +* Other methods to create a static-index are: + * [Creating a static-index using an Operation](../client-api/operations/maintenance/indexes/put-indexes) + * [Creating a static-index from the Studio](../studio/database/indexes/indexes-list-view) + +{CODE:nodejs indexes_1@indexes/whatAreIndexes.js /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } -In the below example we create an index that maps documents from the `Employees` -[collection](../client-api/faq/what-is-a-collection) and enables querying by `FirstName`, -`LastName`, or both. +#### Deploy the index -- First, we need to create an index. - One way to create it is to use the `AbstractIndexCreationTask`, but there are other ways - available as well (you can read about them [here](../indexes/creating-and-deploying)). - {CODE:nodejs indexes_1@indexes/whatAreIndexes.js /} +* The next step is to deploy the index to the RavenDB server. + One way to do this is by calling `execute()` on the index instance. +* Additional methods for deploying static-indexes are described in [Deploy a static index](../indexes/creating-and-deploying#deploy-a-static-index). +* Once deployed, the indexing process will start indexing documents. -- The next step is to send an index to a server (more about index deployment options can - be found [here](../indexes/creating-and-deploying)) so the indexing process can start indexing documents. - {CODE:nodejs indexes_2@indexes/whatAreIndexes.js /} +{CODE:nodejs indexes_2@indexes/whatAreIndexes.js /} -- Now, our index can be queried and indexed results can be returned. - {CODE:nodejs indexes_3@indexes/whatAreIndexes.js /} +{CONTENT-FRAME/} +{CONTENT-FRAME: } +#### Query the index + +* Now you can query the _Employees_ collection using the index. + In this example we query for _Employee_ documents, **filtering results based on index-fields** `LastName` and `Country`. + The results will include only the _Employee_ documents that match the query predicate. +* For detailed guidance on querying with an index, refer to the [Querying an index](../indexes/querying/query-index). + +{CODE:nodejs indexes_3@indexes/whatAreIndexes.js /} + +{CONTENT-FRAME/} {PANEL/} -{PANEL: Remarks} +{PANEL: Understanding index query results} + +{WARNING: } -{WARNING:Remember} -A frequent mistake is to handle indexes as SQL Views, but they are not analogous. -The **result of a query for the given index is a full document**, not only the fields -that were indexed. +A common mistake is treating indexes like SQL Views, but they are not analogous. +The results of a query for a given index are the **full raw documents** that match the query predicate, +and not just the indexed fields. + +This behavior can be changed by applying [Projections](../indexes/querying/projections), +which let you project the query results into selected fields instead of returning the entire document. -This behavior can be altered by [storing](../indexes/storing-data-in-index) fields -and applying [projections](../indexes/querying/projections). {WARNING/} +--- + +#### Viewing the resulting documents: + +For example, the results shown in the following image are the **documents** that match the query predicate. + +![Index query results - documents](images/index-query-results-1.png "Query results - Documents") + +1. This is the index query. + The query predicate filters the resulting documents based on the content of the index-fields. +2. Each row in the results represents a **matching document**. +3. In this example, the `LastName`, `FirstName`, `Title`, etc., are the raw **document-fields**. + +--- + +#### Viewing the index-entries: + +If you wish to **view the index-entries** that compose the index itself, +you can enable the option to show "raw index entries" instead of the matching documents. + +![Index query results - index entries](images/index-query-results-2.png "Query results - Index Entries") + +1. Query the index (no filtering is applied in this example). +2. Click the "Settings" button and toggle on "Show raw index-entries instead of matching documents". +3. Each row in the results represents an **index-entry**. +4. In this example, the `Country`, `FullName`, and `LastName` columns are the **index-fields**, + which were defined in the index definition. +5. This a **term**. + In this example, `usa` is a term generated by the analyzer for index-field `Country` from document `employees/4-a`. + +{PANEL/} + +{PANEL: If indexes exhaust system resources} + +* The indexing process utilizes machine resources to keep the data up-to-date for queries. + +* If indexing drains system resources, it may indicate one or more of the following: + * Indexes may have been defined in a way that causes inefficient processing. + * The [license](https://ravendb.net/buy) may need to be upgraded, + * Your [cloud instance](../cloud/cloud-instances#a-production-cloud-cluster) (if used) may require optimization. + * Hardware upgrades may be necessary to better support your workload. + +* Refer to the [Indexing Performance View](../studio/database/indexes/indexing-performance) in the Studio to monitor and analyze the indexing process. + This view provides graphical representations and detailed statistics of all index activities at each stage. + +* Additionally, refer to the [Common indexing issues](../studio/database/indexes/indexing-performance#common-indexing-issues) section + for troubleshooting and resolving indexing challenges. + {PANEL/} ## Related Articles @@ -67,7 +209,6 @@ and applying [projections](../indexes/querying/projections). ### Indexes - [Creating and Deploying Indexes](../indexes/creating-and-deploying) -- [Indexing Basics](../indexes/indexing-basics) - [Query an Index](../indexes/querying/query-index) - [Stale Indexes](../indexes/stale-indexes) diff --git a/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.python.markdown b/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.python.markdown index 0cffaeb499..e230efb900 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.python.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/indexes/what-are-indexes.python.markdown @@ -1,61 +1,203 @@ -# What Indexes Are +# What are Indexes --- {NOTE: } -* Indexes are server-side functions that define which fields and what values documents can be searched on. - They are _the only way_ to satisfy queries in RavenDB. +* Indexes in RavenDB enable efficient querying by processing raw data and providing fast query results without scanning the entire dataset each and every time. + Learn more in the [Indexes overview](../indexes/what-are-indexes#indexes-overview) below. -* The indexing process is performed in the background and is triggered whenever data is added or changed. - This approach allows the server to respond quickly even when large amounts of data have changed and avoid - costly table scan operations. - -* The core of every index is its mapping function, that utilizes LINQ-like syntax. - The result of this mapping is converted to a [Lucene](http://lucene.apache.org/) index entry - which is persisted for future use to avoid re-indexing each time the query is issued and keep - response times minimal. - -* This page shows a basic example for the creation, deployment, and usage of an index. - Additional examples are available, among other places, on the [Indexing Basics](../indexes/indexing-basics) - and [Map Indexes](../indexes/map-indexes) pages. +* This page provides a [Basic example](../indexes/what-are-indexes#basic-example) of creating, deploying, and querying an index. + Additional examples can be found in [Creating and deploying indexes](../indexes/creating-and-deploying), [Map Indexes](../indexes/map-indexes), + and many other articles under the "Indexes" menu. * In this page: - * [Basic Example](../indexes/what-are-indexes#basic-example) - * [Remarks](../indexes/what-are-indexes#remarks) + * [Indexes overview](../indexes/what-are-indexes#indexes-overview) + * [Types of indexes](../indexes/what-are-indexes#types-of-indexes) + * [Basic example](../indexes/what-are-indexes#basic-example) + * [Understanding index query results](../indexes/what-are-indexes#understanding-index-query-results) + * [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources) {NOTE/} -{PANEL: Basic Example} +{PANEL: Indexes overview} + +**Indexes are fundamental**: + +* Indexes are fundamental to RavenDB’s query execution, enabling efficient query performance by processing the underlying data and delivering faster results. +* ALL queries in RavenDB use an index to deliver results and ensure optimal performance. + Learn more in [Queries always provide results using an index](../client-api/session/querying/how-to-query#queries-always-provide-results-using-an-index). + +--- + +**Main concepts**: + +* When discussing indexes in RavenDB, three key components come into play: + * The index definition + * The indexing process + * The indexed data + +* Each of these components is described in detail in [Indexes - The moving parts](../studio/database/indexes/indexes-overview#indexes---the-moving-parts). + +--- + +**The indexing process**: + +* The indexing process iterates over the raw documents, creating an **index-entry** for each document that is processed. + (Usually a single index-entry is created per raw document, unless working with a [fanout index](../indexes/indexing-nested-data#fanout-index---multiple-index-entries-per-document)). +* Each index-entry contains **index-fields**, and each index-field contains content (**index-terms**) that was generated from the raw documents, + as defined by the index definition and depending on the [analyzer](../indexes/using-analyzers) used. +* A map is built between the indexed-terms and the documents they originated from, + enabling you to query for documents based on the indexed data. + +--- + +**Automatic data processing**: + +* Once defined and deployed, an index will initially process the entire dataset. + After that, the index will only process documents that were modified, added or deleted. + This happens automatically without requiring direct user intervention. +* For example, if changes are made to documents in the "Orders" collection, + all indexes defined for the "Orders" collection will be triggered to update the index with the new data. +* This approach helps avoid costly table scans, allows the server to respond quickly, + and reduces the load on queries while optimizing machine resource usage. + +--- + +**Background operation**: + +* RavenDB indexes are designed to run asynchronously in the background. +* The indexing process does not block or interrupt database operations, such as writing data or running queries, + though queries may temporarily return [stale results](../indexes/stale-indexes) until the index is fully updated. + +--- + +**Separate storage**: + +* Indexes store their processed data separately, ensuring that the raw data remains unaffected. + This separation helps maintain the integrity of the raw data while allowing the index to optimize query performance. + +* If system resources become strained due to indexing, it may require adjustments to the index design, hardware, or other factors. + Learn more in [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources). + +{PANEL/} + +{PANEL: Types of indexes} + +* Indexes in RavenDB are categorized along the following axes: + * **Auto** indexes -vs- **Static** indexes + * **Map** indexes -vs- **Map-Reduce** indexes + * **Single-Collection** (Single-Map) indexes -vs- **Multi-Collection** (Multi-Map) indexes + +* For a detailed description of each type, refer to section [Index types](../studio/database/indexes/indexes-overview#index-types). + +{PANEL/} + +{PANEL: Basic example} + +In this example we create a static-index that indexes content from documents in the `Employees` [collection](../client-api/faq/what-is-a-collection). +This allows querying for _Employee_ documents by any of the index-fields (`FullName`, `LastName`, `Country`). + +{CONTENT-FRAME: } + +#### Define the index + +* The first step is to define the index. + One way to do this is by inheriting from `AbstractIndexCreationTask`. + Learn more in [Define a static-index using a custom class](../indexes/creating-and-deploying#define-a-static-index-using-a-custom-class). +* Other methods to create a static-index are: + * [Creating a static-index using an Operation](../client-api/operations/maintenance/indexes/put-indexes) + * [Creating a static-index from the Studio](../studio/database/indexes/indexes-list-view) + +{CODE:python indexes_1@Indexes/WhatAreIndexes.py /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } -In the below example we create an index that maps documents from the `Employees` -[collection](../client-api/faq/what-is-a-collection) and enables querying by `FirstName`, -`LastName`, or both. +#### Deploy the index -- First, we need to create an index. - One way to create it is to use the `AbstractIndexCreationTask`, but there are other ways - available as well (you can read about them [here](../indexes/creating-and-deploying)). - {CODE:python indexes_1@Indexes/WhatAreIndexes.py /} +* The next step is to deploy the index to the RavenDB server. + One way to do this is by calling `execute()` on the index instance. +* Additional methods for deploying static-indexes are described in [Deploy a static index](../indexes/creating-and-deploying#deploy-a-static-index). +* Once deployed, the indexing process will start indexing documents. -- The next step is to send an index to a server (more about index deployment options can - be found [here](../indexes/creating-and-deploying)) so the indexing process can start indexing documents. - {CODE:python indexes_2@Indexes/WhatAreIndexes.py /} +{CODE:python indexes_2@Indexes/WhatAreIndexes.py /} -- Now, our index can be queried and indexed results can be returned. - {CODE:python indexes_3@Indexes/WhatAreIndexes.py /} +{CONTENT-FRAME/} +{CONTENT-FRAME: } +#### Query the index + +* Now you can query the _Employees_ collection using the index. + In this example we query for _Employee_ documents, **filtering results based on index-fields** `LastName` and `Country`. + The results will include only the _Employee_ documents that match the query predicate. +* For detailed guidance on querying with an index, refer to the [Querying an index](../indexes/querying/query-index). + +{CODE:python indexes_3@Indexes/WhatAreIndexes.py /} + +{CONTENT-FRAME/} {PANEL/} -{PANEL: Remarks} +{PANEL: Understanding index query results} + +{WARNING: } -{WARNING:Remember} -A frequent mistake is to handle indexes as SQL Views, but they are not analogous. -The **result of a query for the given index is a full document**, not only the fields -that were indexed. +A common mistake is treating indexes like SQL Views, but they are not analogous. +The results of a query for a given index are the **full raw documents** that match the query predicate, +and not just the indexed fields. + +This behavior can be changed by applying [Projections](../indexes/querying/projections), +which let you project the query results into selected fields instead of returning the entire document. -This behavior can be altered by [storing](../indexes/storing-data-in-index) fields -and applying [projections](../indexes/querying/projections). {WARNING/} +--- + +#### Viewing the resulting documents: + +For example, the results shown in the following image are the **documents** that match the query predicate. + +![Index query results - documents](images/index-query-results-1.png "Query results - Documents") + +1. This is the index query. + The query predicate filters the resulting documents based on the content of the index-fields. +2. Each row in the results represents a **matching document**. +3. In this example, the `LastName`, `FirstName`, `Title`, etc., are the raw **document-fields**. + +--- + +#### Viewing the index-entries: + +If you wish to **view the index-entries** that compose the index itself, +you can enable the option to show "raw index entries" instead of the matching documents. + +![Index query results - index entries](images/index-query-results-2.png "Query results - Index Entries") + +1. Query the index (no filtering is applied in this example). +2. Click the "Settings" button and toggle on "Show raw index-entries instead of matching documents". +3. Each row in the results represents an **index-entry**. +4. In this example, the `Country`, `FullName`, and `LastName` columns are the **index-fields**, + which were defined in the index definition. +5. This a **term**. + In this example, `usa` is a term generated by the analyzer for index-field `Country` from document `employees/4-a`. + +{PANEL/} + +{PANEL: If indexes exhaust system resources} + +* The indexing process utilizes machine resources to keep the data up-to-date for queries. + +* If indexing drains system resources, it may indicate one or more of the following: + * Indexes may have been defined in a way that causes inefficient processing. + * The [license](https://ravendb.net/buy) may need to be upgraded, + * Your [cloud instance](../cloud/cloud-instances#a-production-cloud-cluster) (if used) may require optimization. + * Hardware upgrades may be necessary to better support your workload. + +* Refer to the [Indexing Performance View](../studio/database/indexes/indexing-performance) in the Studio to monitor and analyze the indexing process. + This view provides graphical representations and detailed statistics of all index activities at each stage. + +* Additionally, refer to the [Common indexing issues](../studio/database/indexes/indexing-performance#common-indexing-issues) section + for troubleshooting and resolving indexing challenges. + {PANEL/} ## Related Articles @@ -67,7 +209,6 @@ and applying [projections](../indexes/querying/projections). ### Indexes - [Creating and Deploying Indexes](../indexes/creating-and-deploying) -- [Indexing Basics](../indexes/indexing-basics) - [Query an Index](../indexes/querying/query-index) - [Stale Indexes](../indexes/stale-indexes) diff --git a/Documentation/5.4/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown b/Documentation/5.4/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown index 1f47441088..ccc4d0d2c3 100644 --- a/Documentation/5.4/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown +++ b/Documentation/5.4/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown @@ -5,7 +5,7 @@ * The below **indexing configuration keys** can be modified via any of the following methods: * As explained in the [Config overview](../../server/configuration/configuration-options) article - * Set a custom configuration per index from the [Client API](../../indexes/creating-and-deploying#creating-an-index-with-custom-configuration) + * Set a custom configuration per index from the [Client API](../../indexes/creating-and-deploying#customizing-configuration) * Set a custom configuration per index from the [Studio](../../studio/database/indexes/create-map-index#configuration) {NOTE/} diff --git a/Documentation/5.4/Samples/csharp/Raven.Documentation.Samples/Indexes/Creating.cs b/Documentation/5.4/Samples/csharp/Raven.Documentation.Samples/Indexes/Creating.cs new file mode 100644 index 0000000000..a001494399 --- /dev/null +++ b/Documentation/5.4/Samples/csharp/Raven.Documentation.Samples/Indexes/Creating.cs @@ -0,0 +1,256 @@ +using System.Collections.Generic; +using System.Linq; +using System.Reflection; +using System.Threading; +using System.Threading.Tasks; +using Raven.Client.Documents; +using Raven.Client.Documents.Conventions; +using Raven.Client.Documents.Indexes; +using Raven.Client.Documents.Session; +using Raven.Documentation.Samples.Orders; + +namespace Raven.Documentation.Samples.Indexes +{ + public class Creating + { + #region indexes_1 + // Define a static-index + // Inherit from 'AbstractIndexCreationTask' + public class Orders_ByTotal : AbstractIndexCreationTask + { + // ... + } + #endregion + + public async Task Sample() + { + using (var store = new DocumentStore()) + { + #region indexes_3 + // Call 'Execute' directly on the index instance + new Orders_ByTotal().Execute(store); + #endregion + + #region indexes_4 + // Call 'ExecuteAsync' directly on the index instance + await new Orders_ByTotal().ExecuteAsync(store); + #endregion + + #region indexes_5 + // Call 'ExecuteIndex' on your store object + store.ExecuteIndex(new Orders_ByTotal()); + #endregion + + #region indexes_6 + // Call 'ExecuteIndexAsync' on your store object + await store.ExecuteIndexAsync(new Orders_ByTotal()); + #endregion + + #region indexes_7 + var indexesToDeploy = new List + { + new Orders_ByTotal(), + new Employees_ByLastName() + }; + + // Call 'ExecuteIndexes' on your store object + store.ExecuteIndexes(indexesToDeploy); + #endregion + + /* + // The below is in a comment because I still want to show 'var indexesToDeploy' for each region in the UI. + #region indexes_8 + var indexesToDeploy = new List + { + new Orders_ByTotal(), + new Employees_ByLastName() + }; + + // Call 'ExecuteIndexesAsync' on your store object + await store.ExecuteIndexesAsync(indexesToDeploy); + #endregion + + #region indexes_9 + var indexesToDeploy = new List + { + new Orders_ByTotal(), + new Employees_ByLastName() + }; + + // Call the static method 'CreateIndexes' on the IndexCreation class + IndexCreation.CreateIndexes(indexesToDeploy, store); + #endregion + + #region indexes_10 + var indexesToDeploy = new List + { + new Orders_ByTotal(), + new Employees_ByLastName() + }; + + // Call the static method 'CreateIndexesAsync' on the IndexCreation class + await IndexCreation.CreateIndexesAsync(indexesToDeploy, store); + #endregion + */ + + #region indexes_11 + // Deploy ALL indexes from the assembly containing the `Orders_ByTotal` class + IndexCreation.CreateIndexes(typeof(Orders_ByTotal).Assembly, store); + #endregion + + #region indexes_12 + // Deploy ALL indexes from the assembly containing the `Orders_ByTotal` class + await IndexCreation.CreateIndexesAsync(typeof(Orders_ByTotal).Assembly, store); + #endregion + + using (var session = store.OpenSession()) + { + #region indexes_14 + List employees = session + .Query() + .Where(x => x.FirstName == "Robert" && x.LastName == "King") + .ToList(); + #endregion + } + } + } + } + + public class CreatingWithCustomConfiguration + { + #region indexes_2 + public class Orders_ByTotal : AbstractIndexCreationTask + { + public Orders_ByTotal() + { + // ... + // Set an indexing configuration value for this index: + Configuration["Indexing.MapTimeoutInSec"] = "30"; + } + } + #endregion + } + + class IndexExample + { + #region indexes_13 + // Define a static-index: + // ====================== + public class Orders_ByTotal : AbstractIndexCreationTask + { + public class IndexEntry + { + // The index-fields: + public string Employee { get; set; } + public string Company { get; set; } + public decimal Total { get; set; } + } + + public Orders_ByTotal() + { + Map = orders => from order in orders + select new IndexEntry + { + // Set the index-fields: + Employee = order.Employee, + Company = order.Company, + Total = order.Lines.Sum(l => + (l.Quantity * l.PricePerUnit) * (1 - l.Discount)) + }; + + // Customize the index as needed, for example: + DeploymentMode = IndexDeploymentMode.Rolling; + Configuration["Indexing.MapTimeoutInSec"] = "30"; + Indexes.Add(x => x.Company, FieldIndexing.Search); + // ... + } + } + + public static void Main(string[] args) + { + using (DocumentStore store = new DocumentStore + { + Urls = new[] { "http://localhost:8080" }, + Database = "Northwind" + }) + { + store.Initialize(); + + // Deploy the index: + // ================= + new Orders_ByTotal().Execute(store); + + using (IDocumentSession session = store.OpenSession()) + { + // Query the index: + // ================ + IList orders = session + .Query() + // Query for Order documents that have Total > 100 + .Where(x => x.Total > 100) + .OfType() + .ToList(); + } + } + } + #endregion + } + + // This index is used in the multiple indexes deployment overloads above + public class Employees_ByLastName : AbstractIndexCreationTask + { + public class IndexEntry + { + public string LastName { get; set; } + } + + public Employees_ByLastName() + { + Map = employees => from emp in employees + select new IndexEntry + { + LastName = emp.LastName + }; + } + } + + public interface IFoo + { + #region syntax_1 + // Call this method directly on the index instance + void Execute(IDocumentStore store, DocumentConventions conventions = null, + string database = null); + + // Call these methods on the store object + void ExecuteIndex(IAbstractIndexCreationTask index, string database = null); + void ExecuteIndexes(IEnumerable indexes, + string database = null); + + // Call these static methods on the IndexCreation class + void CreateIndexes(IEnumerable indexes, IDocumentStore store, + DocumentConventions conventions = null, string database = null); + void CreateIndexes(Assembly assemblyToScan, IDocumentStore store, + DocumentConventions conventions = null, string database = null); + #endregion + + #region syntax_2 + // Call this method directly on the index instance + Task ExecuteAsync(IDocumentStore store, DocumentConventions conventions = null, + string database = null, CancellationToken token = default); + + // Call these methods on the store object + Task ExecuteIndexAsync(IAbstractIndexCreationTask index, string database = null, + CancellationToken token = default(CancellationToken)); + Task ExecuteIndexesAsync(IEnumerable indexes, + string database = null, CancellationToken token = default(CancellationToken)); + + // Call these static methods on the IndexCreation class + Task CreateIndexesAsync(IEnumerable indexes, + IDocumentStore store, DocumentConventions conventions = null, string database = null, + CancellationToken token = default(CancellationToken)); + Task CreateIndexesAsync(Assembly assemblyToScan, IDocumentStore store, + DocumentConventions conventions = null, string database = null, + CancellationToken token = default(CancellationToken)); + #endregion + } +} diff --git a/Documentation/5.4/Samples/csharp/Raven.Documentation.Samples/Indexes/WhatAreIndexes.cs b/Documentation/5.4/Samples/csharp/Raven.Documentation.Samples/Indexes/WhatAreIndexes.cs index d0e9e9718f..ec6433c340 100644 --- a/Documentation/5.4/Samples/csharp/Raven.Documentation.Samples/Indexes/WhatAreIndexes.cs +++ b/Documentation/5.4/Samples/csharp/Raven.Documentation.Samples/Indexes/WhatAreIndexes.cs @@ -1,6 +1,5 @@ using System.Collections.Generic; using System.Linq; - using Raven.Client.Documents; using Raven.Client.Documents.Indexes; using Raven.Documentation.Samples.Orders; @@ -10,15 +9,28 @@ namespace Raven.Documentation.Samples.Indexes public class WhatAreIndexes { #region indexes_1 - public class Employees_ByFirstAndLastName : AbstractIndexCreationTask + // Define the index: + // ================= + + public class Employees_ByNameAndCountry : AbstractIndexCreationTask { - public Employees_ByFirstAndLastName() + public class IndexEntry + { + // The index-fields + public string LastName { get; set; } + public string FullName { get; set; } + public string Country { get; set; } + } + + public Employees_ByNameAndCountry() { Map = employees => from employee in employees - select new + select new IndexEntry() { - FirstName = employee.FirstName, - LastName = employee.LastName + // Define the content for each index-field + LastName = employee.LastName, + FullName = employee.FirstName + " " + employee.LastName, + Country = employee.Address.Country }; } } @@ -29,16 +41,24 @@ public WhatAreIndexes() using (var store = new DocumentStore()) { #region indexes_2 - // save index on server - new Employees_ByFirstAndLastName().Execute(store); + // Deploy the index to the server: + // =============================== + + new Employees_ByNameAndCountry().Execute(store); #endregion using (var session = store.OpenSession()) { #region indexes_3 - IList results = session - .Query() - .Where(x => x.FirstName == "Robert") + // Query the database using the index: + // =================================== + + IList employeesFromUK = session + .Query() + // Here we query for all Employee documents that are from the UK + // and have 'King' in their LastName field: + .Where(x => x.LastName == "King" && x.Country == "UK") + .OfType() .ToList(); #endregion } diff --git a/Documentation/5.4/Samples/java/src/test/java/net/ravendb/Indexes/Creating.java b/Documentation/5.4/Samples/java/src/test/java/net/ravendb/Indexes/Creating.java new file mode 100644 index 0000000000..078a9dd4c8 --- /dev/null +++ b/Documentation/5.4/Samples/java/src/test/java/net/ravendb/Indexes/Creating.java @@ -0,0 +1,168 @@ +import net.ravendb.client.documents.DocumentStore; +import net.ravendb.client.documents.IDocumentStore; +import net.ravendb.client.documents.indexes.*; +import net.ravendb.client.documents.operations.indexes.PutIndexesOperation; +import net.ravendb.client.documents.session.IDocumentSession; +import java.util.Collections; +import java.util.List; + +public class Creating { + + private static class Order { + } + + private static class Employee { + } + + //region indexes_8 + public static class Orders_Totals extends AbstractIndexCreationTask { + public static class Result { + private String employee; + private String company; + private double total; + + public String getEmployee() { + return employee; + } + + public void setEmployee(String employee) { + this.employee = employee; + } + + public String getCompany() { + return company; + } + + public void setCompany(String company) { + this.company = company; + } + + public double getTotal() { + return total; + } + + public void setTotal(double total) { + this.total = total; + } + } + + public Orders_Totals() { + map = "docs.Orders.Select(order => new { " + + " Employee = order.Employee, " + + " Company = order.Company, " + + " Total = Enumerable.Sum(order.Lines, l => ((decimal)((((decimal) l.Quantity) * l.PricePerUnit) * (1M - l.Discount)))) " + + "})"; + } + + public static void main(String[] args) { + try (IDocumentStore store = new DocumentStore(new String[]{ "http://localhost:8080" }, "Northwind")) { + store.initialize(); + + new Orders_Totals().execute(store); + + try (IDocumentSession session = store.openSession()) { + List orders = session + .query(Result.class, Orders_Totals.class) + .whereGreaterThan("Total", 100) + .ofType(Order.class) + .toList(); + } + } + } + } + //endregion + + private class Foo { + //region indexes_1 + public class Orders_Totals extends AbstractIndexCreationTask { + /// ... + } + //endregion + } + + public Creating() { + try (IDocumentStore store = new DocumentStore( new String[]{ "http://127.0.0.1:8080" }, "Northwind")) { + IndexCreation.createIndexes(Collections.singletonList(new Users_ByName()), store); + + //region indexes_6 + IndexDefinitionBuilder builder = new IndexDefinitionBuilder(); + builder.setMap( + "from order in docs.Orders \n" + + "select new \n" + + " {\n" + + " order.employee,\n" + + " order.company,\n" + + " total = order.lines.Sum(l => (l.quantity * l.pricePerUnit) * (1 - l.discount))\n" + + "}"); + + store.maintenance() + .send(new PutIndexesOperation(builder.toIndexDefinition(store.getConventions()))); + //endregion + } + + try (IDocumentStore store = new DocumentStore()) { + //region indexes_2 + // deploy index to database defined in `DocumentStore.getDatabase` method + // using default DocumentStore `conventions` + new Orders_Totals().execute(store); + //endregion + + //region indexes_3 + // deploy index to `Northwind` database + // using default DocumentStore `conventions` + new Orders_Totals().execute(store, store.getConventions(), "Northwind"); + //endregion + + //region indexes_5 + IndexDefinition indexDefinition = new IndexDefinition(); + indexDefinition.setName("Orders/Totals"); + indexDefinition.setMaps(Collections.singleton( + "from order in docs.Orders " + + " select new " + + " { " + + " order.employee, " + + " order.company, " + + " total = order.lines.Sum(l => (l.quantity * l.pricePerUnit) * (1 - l.discount)) " + + "}" + )); + + store + .maintenance() + .send(new PutIndexesOperation(indexDefinition)); + //endregion + + try (IDocumentSession session = store.openSession()) { + //region indexes_7 + List employees = session + .query(Employee.class) + .whereEquals("firstName", "Robert") + .andAlso() + .whereEquals("lastName", "King") + .toList(); + //endregion + } + + + + + } + } + + private class Foo2{ + IndexConfiguration configuration; + //region indexes_9 + public class Orders_Totals extends AbstractIndexCreationTask { + public Orders_Totals() { + // ... + configuration.put("MapTimeoutInSec","30"); + setConfiguration(configuration); + } + } + //endregion + } + + + private class Users_ByName extends AbstractIndexCreationTask { + + } +} diff --git a/Documentation/5.4/Samples/java/src/test/java/net/ravendb/Indexes/WhatAreIndexes.java b/Documentation/5.4/Samples/java/src/test/java/net/ravendb/Indexes/WhatAreIndexes.java index c13097569e..9beec2dd91 100644 --- a/Documentation/5.4/Samples/java/src/test/java/net/ravendb/Indexes/WhatAreIndexes.java +++ b/Documentation/5.4/Samples/java/src/test/java/net/ravendb/Indexes/WhatAreIndexes.java @@ -10,11 +10,15 @@ public class WhatAreIndexes { //region indexes_1 - public static class Employees_ByFirstAndLastName extends AbstractIndexCreationTask { - public Employees_ByFirstAndLastName() { - map = "docs.Employees.Select(employee => new {" + - " FirstName = employee.FirstName," + - " LastName = employee.LastName" + + // Define the index: + // ================= + + public static class Employees_ByNameAndCountry extends AbstractIndexCreationTask { + public Employees_ByNameAndCountry() { + map = "docs.Employees.Select(employee => new { " + + " LastName = employee.LastName, " + + " FullName = (employee.FirstName + \" \") + employee.LastName, " + + " Country = employee.Address.Country " + "})"; } } @@ -26,15 +30,23 @@ private static class Employee { public WhatAreIndexes() { try (IDocumentStore store = new DocumentStore()) { //region indexes_2 - // save index on server - new Employees_ByFirstAndLastName().execute(store); + // Deploy the index to the server: + // =============================== + + new Employees_ByNameAndCountry().execute(store); //endregion try (IDocumentSession session = store.openSession()) { //region indexes_3 - List results = session - .query(Employee.class, Employees_ByFirstAndLastName.class) - .whereEquals("FirstName", "Robert") + // Query the database using the index: + // =================================== + + List employeesFromUK = session + .query(Employee.class, Employees_ByNameAndCountry.class) + // Here we query for all Employee documents that are from the UK + // and have 'King' in their LastName field: + .whereEquals("LastName", "King") + .whereEquals("Country", "UK") .toList(); //endregion } diff --git a/Documentation/5.4/Samples/nodejs/indexes/creating.js b/Documentation/5.4/Samples/nodejs/indexes/creating.js new file mode 100644 index 0000000000..42a28d0132 --- /dev/null +++ b/Documentation/5.4/Samples/nodejs/indexes/creating.js @@ -0,0 +1,126 @@ +import { AbstractJavaScriptIndexCreationTask, DocumentStore} from "ravendb"; +const documentStore = new DocumentStore(); + +{ + //region indexes_1 + class Orders_ByTotal extends AbstractJavaScriptIndexCreationTask { + /// ... + } + //endregion +} +{ + //region indexes_2 + class Orders_ByTotal extends AbstractJavaScriptIndexCreationTask { + constructor() { + super(); + // ... + + // Set an indexing configuration value for this index: + this.configuration = { + "Indexing.MapTimeoutInSec": "30", + } + } + } + //endregion +} + +async function deployIndexes() { + //region indexes_3 + // Call 'execute' directly on the index instance + await new Orders_ByTotal().execute(documentStore); + //endregion + + //region indexes_4 + // Call 'executeIndex' on your store object + await documentStore.executeIndex(new Orders_ByTotal()); + //endregion + + //region indexes_5 + const indexesToDeploy = [new Orders_ByTotal(), new Employees_ByLastName()]; + // Call 'executeIndexes' on your store object + await documentStore.executeIndexes(indexesToDeploy); + //endregion + + //region indexes_6 + const indexesToDeploy = [new Orders_ByTotal(), new Employees_ByLastName()]; + // Call the static method 'createIndexes' on the IndexCreation class + await IndexCreation.createIndexes(indexesToDeploy, documentStore); + //endregion +} + +//region indexes_7 +// Define a static-index: +// ====================== +class Orders_ByTotal extends AbstractJavaScriptIndexCreationTask { + constructor() { + super(); + + this.map("Orders", order => { + return { + Employee: order.Employee, + Company: order.Company, + Total: order.Lines.reduce((sum, line) => + sum + (line.Quantity * line.PricePerUnit) * (1 - line.Discount), 0) + } + }); + + // Customize the index configuration + this.deploymentMode = "Rolling"; + this.configuration["Indexing.MapTimeoutInSec"] = "30"; + this.indexes.add(x => x.Company, "Search"); + // ... + } +} + +async function main() { + const documentStore = new DocumentStore("http://localhost:8080", "Northwind"); + documentStore.initialize(); + + // Deploy the index: + // ================= + const ordersByTotalIndex = new Orders_ByTotal(); + await ordersByTotalIndex.execute(documentStore); + + const session = documentStore.openSession() + + // Query the index: + // ================ + const myIndexName = ordersByTotalIndex.getIndexName(); + + const orders = await session + .query({ indexName: myIndexName }) + .whereGreaterThan("Total", 100) + .all(); +} +//endregion + +{ + const session = store.openSession(); + + //region indexes_8 + const employees = await session + .query({ collection: 'employees' }) + .whereEquals("FirstName", "Robert") + .whereEquals("LastName", "King") + .all(); + //endregion +} + +{ + //region syntax + // Call this method directly on the index instance + execute(store); + execute(store, conventions)>; + execute(store, conventions, database); + + // Call these methods on the store object + executeIndex(index); + executeIndex(index, database); + executeIndexes(indexes); + executeIndexes(indexes, database); + + // Call these methods on the IndexCreation class + createIndexes(indexes, store); + createIndexes(indexes, store, conventions); + //endregion +} diff --git a/Documentation/5.4/Samples/nodejs/indexes/whatAreIndexes.js b/Documentation/5.4/Samples/nodejs/indexes/whatAreIndexes.js index 870803469f..f0127a6488 100644 --- a/Documentation/5.4/Samples/nodejs/indexes/whatAreIndexes.js +++ b/Documentation/5.4/Samples/nodejs/indexes/whatAreIndexes.js @@ -1,20 +1,27 @@ import { DocumentStore, - AbstractIndexCreationTask, - IndexDefinition, - PutIndexesOperation + AbstractJavaScriptIndexCreationTask } from "ravendb"; const store = new DocumentStore(); //region indexes_1 - class Employees_ByFirstAndLastName extends AbstractIndexCreationTask { + // Define the index: + // ================= + + class Employees_ByNameAndCountry extends AbstractJavaScriptIndexCreationTask { constructor() { super(); - this.map = "docs.Employees.Select(employee => new {" + - " FirstName = employee.FirstName," + - " LastName = employee.LastName" + - "})"; + + this.map("Employees", employee => { + return { + // Define the content for each index-field: + // ======================================== + LastName: employee.LastName, + FullName: employee.FirstName + " " + employee.LastName, + Country: employee.Address.Country + }; + }); } } //endregion @@ -24,17 +31,25 @@ const store = new DocumentStore(); async function whatAreIndexes() { //region indexes_2 - // save index on server - const employeesIndex = new Employees_ByFirstAndLastName(); + // Deploy the index to the server: + // =============================== + + const employeesIndex = new Employees_ByNameAndCountry(); await employeesIndex.execute(store); //endregion { const session = store.openSession(); //region indexes_3 - const results = await session + // Query the database using the index: + // =================================== + + const employeesFromUK = await session .query({ indexName: employeesIndex.getIndexName() }) - .whereEquals("FirstName", "Robert") + // Here we query for all Employee documents that are from the UK + // and have 'King' in their LastName field: + .whereEquals("LastName", "King") + .whereEquals("Country", "UK") .all(); //endregion } diff --git a/Documentation/5.4/Samples/python/Indexes/WhatAreIndexes.py b/Documentation/5.4/Samples/python/Indexes/WhatAreIndexes.py index d9ee7ceecd..29d1c098e1 100644 --- a/Documentation/5.4/Samples/python/Indexes/WhatAreIndexes.py +++ b/Documentation/5.4/Samples/python/Indexes/WhatAreIndexes.py @@ -4,12 +4,21 @@ # region indexes_1 -class Employees_ByFirstAndLastName(AbstractIndexCreationTask): +# Define the index: +# ================= + +class Employees_ByNameAndCountry(AbstractIndexCreationTask): def __init__(self): super().__init__() - self.map = "from employee in docs.Employees select new {first_name = employee.FirstName, last_name = employee.LastName}" - - + + self.map = """ + from employee in docs.Employees + select new { + LastName = employee.LastName, + FullName = employee.FirstName + " " + employee.LastName, + Country = employee.Address.country + } + """ # endregion @@ -21,15 +30,22 @@ def test_what_are_indexes(self): with self.embedded_server.get_document_store("WhatAreIndexes") as store: with store.open_session() as session: # region indexes_2 - # save index on server - Employees_ByFirstAndLastName().execute(store) + # Deploy the index to the server: + # =============================== + + Employees_ByNameAndCountry().execute(store) # endregion with store.open_session() as session: # region indexes_3 - results = list( - session.query_index_type(Employees_ByFirstAndLastName, Employee).where_equals( - "first_name", "Robert" - ) + # Query the database using the index: + # =================================== + + employeesFromUK = list( + session.query_index_type(Employees_ByNameAndCountry, Employee) + # Here we query for all Employee documents that are from the UK + # and have 'King' in their LastName field: + .where_equals("Country", "UK") + .where_equals("LastName", "King") ) # endregion diff --git a/Documentation/6.0/Raven.Documentation.Pages/indexes/images/index-query-results-1.png b/Documentation/6.0/Raven.Documentation.Pages/indexes/images/index-query-results-1.png new file mode 100644 index 0000000000..0b9acecd0d Binary files /dev/null and b/Documentation/6.0/Raven.Documentation.Pages/indexes/images/index-query-results-1.png differ diff --git a/Documentation/6.0/Raven.Documentation.Pages/indexes/images/index-query-results-2.png b/Documentation/6.0/Raven.Documentation.Pages/indexes/images/index-query-results-2.png new file mode 100644 index 0000000000..3044311080 Binary files /dev/null and b/Documentation/6.0/Raven.Documentation.Pages/indexes/images/index-query-results-2.png differ diff --git a/Documentation/6.0/Raven.Documentation.Pages/indexes/map-reduce-indexes.dotnet.markdown b/Documentation/6.0/Raven.Documentation.Pages/indexes/map-reduce-indexes.dotnet.markdown index 663edf9baf..70bb83c1c4 100644 --- a/Documentation/6.0/Raven.Documentation.Pages/indexes/map-reduce-indexes.dotnet.markdown +++ b/Documentation/6.0/Raven.Documentation.Pages/indexes/map-reduce-indexes.dotnet.markdown @@ -230,7 +230,7 @@ The map-reduce output documents are configured with these properties of | - | - | - | | **OutputReduceToCollection** | `string` | Collection name for the output documents. | | **PatternReferencesCollectionName** | `string` | Optional collection name for the reference documents - by default it is `/References`. | -| **PatternForOutputReduceToCollectionReferences** | `string` / `Expression>` | Document ID format for reference documents. This ID references the fields of the reduce function output, which determines how the output documents are aggregated. The type of this parameter is different depending on if the index is created using [IndexDefinition](../indexes/creating-and-deploying#using-maintenance-operations) or [AbstractIndexCreationTask](../indexes/creating-and-deploying#using-abstractindexcreationtask). | +| **PatternForOutputReduceToCollectionReferences** | `string` / `Expression>` | Document ID format for reference documents. This ID references the fields of the reduce function output, which determines how the output documents are aggregated. The type of this parameter is different depending on if the index is created using [IndexDefinition](../client-api/operations/maintenance/indexes/put-indexes) or [AbstractIndexCreationTask](../indexes/creating-and-deploying#define-a-static-index-using-a-custom-class). | To index artificial documents in strongly typed syntax (LINQ), you will need the type of reference documents: diff --git a/Documentation/6.0/Raven.Documentation.Pages/indexes/map-reduce-indexes.js.markdown b/Documentation/6.0/Raven.Documentation.Pages/indexes/map-reduce-indexes.js.markdown index 958d93343a..4c7312ec7b 100644 --- a/Documentation/6.0/Raven.Documentation.Pages/indexes/map-reduce-indexes.js.markdown +++ b/Documentation/6.0/Raven.Documentation.Pages/indexes/map-reduce-indexes.js.markdown @@ -160,7 +160,7 @@ The map-reduce output documents are configured with these properties of | - | - | - | | **outputReduceToCollection** | `string` | Collection name for the output documents. | | **patternReferencesCollectionName** | `string` | Optional collection name for the reference documents - by default it is `/References`. | -| **patternForOutputReduceToCollectionReferences** | `string` | Document ID format for reference documents. This ID references the fields of the reduce function output, which determines how the output documents are aggregated. The type of this parameter is different depending on if the index is created using [IndexDefinition](../indexes/creating-and-deploying#using-maintenance-operations) or [AbstractIndexCreationTask](../indexes/creating-and-deploying#using-abstractindexcreationtask). | +| **patternForOutputReduceToCollectionReferences** | `string` | Document ID format for reference documents. This ID references the fields of the reduce function output, which determines how the output documents are aggregated. The type of this parameter is different depending on if the index is created using [IndexDefinition](../client-api/operations/maintenance/indexes/put-indexes#put-indexes-operation) or [AbstractJavaScriptIndexCreationTask](../indexes/creating-and-deploying#define-a-static-index-using-a-custom-class). | ### Examples diff --git a/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.dotnet.markdown b/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.dotnet.markdown index 8416a5f336..ceff469d5b 100644 --- a/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.dotnet.markdown +++ b/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.dotnet.markdown @@ -1,61 +1,203 @@ -# What Indexes Are +# What are Indexes --- {NOTE: } -* Indexes are server-side functions that define which fields and what values documents can be searched on. - They are _the only way_ to satisfy queries in RavenDB. +* Indexes in RavenDB enable efficient querying by processing raw data and providing fast query results without scanning the entire dataset each and every time. + Learn more in the [Indexes overview](../indexes/what-are-indexes#indexes-overview) below. -* The indexing process is performed in the background and is triggered whenever data is added or changed. - This approach allows the server to respond quickly even when large amounts of data have changed and avoid - costly table scan operations. - -* The core of every index is its mapping function, that utilizes LINQ-like syntax. - The result of this mapping is converted to a [Lucene](http://lucene.apache.org/) index entry - which is persisted for future use to avoid re-indexing each time the query is issued and keep - response times minimal. - -* This page shows a basic example for the creation, deployment, and usage of an index. - Additional examples are available, among other places, on the [Indexing Basics](../indexes/indexing-basics) - and [Map Indexes](../indexes/map-indexes) pages. +* This page provides a [Basic example](../indexes/what-are-indexes#basic-example) of creating, deploying, and querying an index. + Additional examples can be found in [Creating and deploying indexes](../indexes/creating-and-deploying), [Map Indexes](../indexes/map-indexes), + and many other articles under the "Indexes" menu. * In this page: - * [Basic Example](../indexes/what-are-indexes#basic-example) - * [Remarks](../indexes/what-are-indexes#remarks) + * [Indexes overview](../indexes/what-are-indexes#indexes-overview) + * [Types of indexes](../indexes/what-are-indexes#types-of-indexes) + * [Basic example](../indexes/what-are-indexes#basic-example) + * [Understanding index query results](../indexes/what-are-indexes#understanding-index-query-results) + * [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources) {NOTE/} -{PANEL: Basic Example} +{PANEL: Indexes overview} + +**Indexes are fundamental**: -In the below example we create an index that maps documents from the `Employees` -[collection](../client-api/faq/what-is-a-collection) and enables querying by `FirstName`, -`LastName`, or both. +* Indexes are fundamental to RavenDB’s query execution, enabling efficient query performance by processing the underlying data and delivering faster results. +* ALL queries in RavenDB use an index to deliver results and ensure optimal performance. + Learn more in [Queries always provide results using an index](../client-api/session/querying/how-to-query#queries-always-provide-results-using-an-index). + +--- -- First, we need to create an index. - One way to create it is to use the `AbstractIndexCreationTask`, but there are other ways - available as well (you can read about them [here](../indexes/creating-and-deploying)). - {CODE indexes_1@Indexes/WhatAreIndexes.cs /} +**Main concepts**: + +* When discussing indexes in RavenDB, three key components come into play: + * The index definition + * The indexing process + * The indexed data + +* Each of these components is described in detail in [Indexes - The moving parts](../studio/database/indexes/indexes-overview#indexes---the-moving-parts). + +--- + +**The indexing process**: + +* The indexing process iterates over the raw documents, creating an **index-entry** for each document that is processed. + (Usually a single index-entry is created per raw document, unless working with a [fanout index](../indexes/indexing-nested-data#fanout-index---multiple-index-entries-per-document)). +* Each index-entry contains **index-fields**, and each index-field contains content (**index-terms**) that was generated from the raw documents, + as defined by the index definition and depending on the [analyzer](../indexes/using-analyzers) used. +* A map is built between the indexed-terms and the documents they originated from, + enabling you to query for documents based on the indexed data. + +--- + +**Automatic data processing**: + +* Once defined and deployed, an index will initially process the entire dataset. + After that, the index will only process documents that were modified, added or deleted. + This happens automatically without requiring direct user intervention. +* For example, if changes are made to documents in the "Orders" collection, + all indexes defined for the "Orders" collection will be triggered to update the index with the new data. +* This approach helps avoid costly table scans, allows the server to respond quickly, + and reduces the load on queries while optimizing machine resource usage. + +--- -- The next step is to send an index to a server (more about index deployment options can - be found [here](../indexes/creating-and-deploying)) so the indexing process can start indexing documents. - {CODE indexes_2@Indexes/WhatAreIndexes.cs /} +**Background operation**: -- Now, our index can be queried and indexed results can be returned. - {CODE indexes_3@Indexes/WhatAreIndexes.cs /} +* RavenDB indexes are designed to run asynchronously in the background. +* The indexing process does not block or interrupt database operations, such as writing data or running queries, + though queries may temporarily return [stale results](../indexes/stale-indexes) until the index is fully updated. + +--- + +**Separate storage**: + +* Indexes store their processed data separately, ensuring that the raw data remains unaffected. + This separation helps maintain the integrity of the raw data while allowing the index to optimize query performance. + +* If system resources become strained due to indexing, it may require adjustments to the index design, hardware, or other factors. + Learn more in [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources). + +{PANEL/} + +{PANEL: Types of indexes} + +* Indexes in RavenDB are categorized along the following axes: + * **Auto** indexes -vs- **Static** indexes + * **Map** indexes -vs- **Map-Reduce** indexes + * **Single-Collection** (Single-Map) indexes -vs- **Multi-Collection** (Multi-Map) indexes + +* For a detailed description of each type, refer to section [Index types](../studio/database/indexes/indexes-overview#index-types). {PANEL/} -{PANEL: Remarks} +{PANEL: Basic example} + +In this example we create a static-index that indexes content from documents in the `Employees` [collection](../client-api/faq/what-is-a-collection). +This allows querying for _Employee_ documents by any of the index-fields (`FullName`, `LastName`, `Country`). + +{CONTENT-FRAME: } + +#### Define the index + +* The first step is to define the index. + One way to do this is by inheriting from `AbstractIndexCreationTask`. + Learn more in [Define a static-index using a custom class](../indexes/creating-and-deploying#define-a-static-index-using-a-custom-class). +* Other methods to create a static-index are: + * [Creating a static-index using an Operation](../client-api/operations/maintenance/indexes/put-indexes) + * [Creating a static-index from the Studio](../studio/database/indexes/indexes-list-view) + +{CODE indexes_1@Indexes/WhatAreIndexes.cs /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } -{WARNING:Remember} -A frequent mistake is to handle indexes as SQL Views, but they are not analogous. -The **result of a query for the given index is a full document**, not only the fields -that were indexed. +#### Deploy the index + +* The next step is to deploy the index to the RavenDB server. + One way to do this is by calling `Execute()` on the index instance. +* Additional methods for deploying static-indexes are described in [Deploy a static index](../indexes/creating-and-deploying#deploy-a-static-index). +* Once deployed, the indexing process will start indexing documents. + +{CODE indexes_2@Indexes/WhatAreIndexes.cs /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +#### Query the index + +* Now you can query the _Employees_ collection using the index. + In this example we query for _Employee_ documents, **filtering results based on index-fields** `LastName` and `Country`. + The results will include only the _Employee_ documents that match the query predicate. +* For detailed guidance on querying with an index, refer to the [Querying an index](../indexes/querying/query-index). + +{CODE indexes_3@Indexes/WhatAreIndexes.cs /} + +{CONTENT-FRAME/} +{PANEL/} + +{PANEL: Understanding index query results} + +{WARNING: } + +A common mistake is treating indexes like SQL Views, but they are not analogous. +The results of a query for a given index are the **full raw documents** that match the query predicate, +and not just the indexed fields. + +This behavior can be changed by applying [Projections](../indexes/querying/projections), +which let you project the query results into selected fields instead of returning the entire document. -This behavior can be altered by [storing](../indexes/storing-data-in-index) fields -and applying [projections](../indexes/querying/projections). {WARNING/} +--- + +#### Viewing the resulting documents: + +For example, the results shown in the following image are the **documents** that match the query predicate. + +![Index query results - documents](images/index-query-results-1.png "Query results - Documents") + +1. This is the index query. + The query predicate filters the resulting documents based on the content of the index-fields. +2. Each row in the results represents a **matching document**. +3. In this example, the `LastName`, `FirstName`, `Title`, etc., are the raw **document-fields**. + +--- + +#### Viewing the index-entries: + +If you wish to **view the index-entries** that compose the index itself, +you can enable the option to show "raw index entries" instead of the matching documents. + +![Index query results - index entries](images/index-query-results-2.png "Query results - Index Entries") + +1. Query the index (no filtering is applied in this example). +2. Click the "Settings" button and toggle on "Show raw index-entries instead of matching documents". +3. Each row in the results represents an **index-entry**. +4. In this example, the `Country`, `FullName`, and `LastName` columns are the **index-fields**, + which were defined in the index definition. +5. This a **term**. + In this example, `usa` is a term generated by the analyzer for index-field `Country` from document `employees/4-a`. + +{PANEL/} + +{PANEL: If indexes exhaust system resources} + +* The indexing process utilizes machine resources to keep the data up-to-date for queries. + +* If indexing drains system resources, it may indicate one or more of the following: + * Indexes may have been defined in a way that causes inefficient processing. + * The [license](https://ravendb.net/buy) may need to be upgraded, + * Your [cloud instance](../cloud/cloud-instances#a-production-cloud-cluster) (if used) may require optimization. + * Hardware upgrades may be necessary to better support your workload. + +* Refer to the [Indexing Performance View](../studio/database/indexes/indexing-performance) in the Studio to monitor and analyze the indexing process. + This view provides graphical representations and detailed statistics of all index activities at each stage. + +* Additionally, refer to the [Common indexing issues](../studio/database/indexes/indexing-performance#common-indexing-issues) section + for troubleshooting and resolving indexing challenges. + {PANEL/} ## Related Articles @@ -67,7 +209,6 @@ and applying [projections](../indexes/querying/projections). ### Indexes - [Creating and Deploying Indexes](../indexes/creating-and-deploying) -- [Indexing Basics](../indexes/indexing-basics) - [Query an Index](../indexes/querying/query-index) - [Stale Indexes](../indexes/stale-indexes) @@ -77,5 +218,6 @@ and applying [projections](../indexes/querying/projections). - [Studio Index List View](../studio/database/indexes/indexes-list-view) ### Sharding -- [Indexing in a Sharded DB](../sharding/indexing) + +- [Indexing in a Sharded DB](../sharding/indexing) - [Querying in a Sharded DB](../sharding/querying) diff --git a/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.java.markdown b/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.java.markdown index 51cd1a1985..b970a50743 100644 --- a/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.java.markdown +++ b/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.java.markdown @@ -1,61 +1,203 @@ -# What Indexes Are +# What are Indexes --- {NOTE: } -* Indexes are server-side functions that define which fields and what values documents can be searched on. - They are _the only way_ to satisfy queries in RavenDB. +* Indexes in RavenDB enable efficient querying by processing raw data and providing fast query results without scanning the entire dataset each and every time. + Learn more in the [Indexes overview](../indexes/what-are-indexes#indexes-overview) below. -* The indexing process is performed in the background and is triggered whenever data is added or changed. - This approach allows the server to respond quickly even when large amounts of data have changed and avoid - costly table scan operations. - -* The core of every index is its mapping function, that utilizes LINQ-like syntax. - The result of this mapping is converted to a [Lucene](http://lucene.apache.org/) index entry - which is persisted for future use to avoid re-indexing each time the query is issued and keep - response times minimal. - -* This page shows a basic example for the creation, deployment, and usage of an index. - Additional examples are available, among other places, on the [Indexing Basics](../indexes/indexing-basics) - and [Map Indexes](../indexes/map-indexes) pages. +* This page provides a [Basic example](../indexes/what-are-indexes#basic-example) of creating, deploying, and querying an index. + Additional examples can be found in [Creating and deploying indexes](../indexes/creating-and-deploying), [Map Indexes](../indexes/map-indexes), + and many other articles under the "Indexes" menu. * In this page: - * [Basic Example](../indexes/what-are-indexes#basic-example) - * [Remarks](../indexes/what-are-indexes#remarks) + * [Indexes overview](../indexes/what-are-indexes#indexes-overview) + * [Types of indexes](../indexes/what-are-indexes#types-of-indexes) + * [Basic example](../indexes/what-are-indexes#basic-example) + * [Understanding index query results](../indexes/what-are-indexes#understanding-index-query-results) + * [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources) {NOTE/} -{PANEL: Basic Example} +{PANEL: Indexes overview} + +**Indexes are fundamental**: + +* Indexes are fundamental to RavenDB’s query execution, enabling efficient query performance by processing the underlying data and delivering faster results. +* ALL queries in RavenDB use an index to deliver results and ensure optimal performance. + Learn more in [Queries always provide results using an index](../client-api/session/querying/how-to-query#queries-always-provide-results-using-an-index). + +--- + +**Main concepts**: + +* When discussing indexes in RavenDB, three key components come into play: + * The index definition + * The indexing process + * The indexed data + +* Each of these components is described in detail in [Indexes - The moving parts](../studio/database/indexes/indexes-overview#indexes---the-moving-parts). + +--- + +**The indexing process**: + +* The indexing process iterates over the raw documents, creating an **index-entry** for each document that is processed. + (Usually a single index-entry is created per raw document, unless working with a [fanout index](../indexes/indexing-nested-data#fanout-index---multiple-index-entries-per-document)). +* Each index-entry contains **index-fields**, and each index-field contains content (**index-terms**) that was generated from the raw documents, + as defined by the index definition and depending on the [analyzer](../indexes/using-analyzers) used. +* A map is built between the indexed-terms and the documents they originated from, + enabling you to query for documents based on the indexed data. + +--- + +**Automatic data processing**: + +* Once defined and deployed, an index will initially process the entire dataset. + After that, the index will only process documents that were modified, added or deleted. + This happens automatically without requiring direct user intervention. +* For example, if changes are made to documents in the "Orders" collection, + all indexes defined for the "Orders" collection will be triggered to update the index with the new data. +* This approach helps avoid costly table scans, allows the server to respond quickly, + and reduces the load on queries while optimizing machine resource usage. + +--- + +**Background operation**: + +* RavenDB indexes are designed to run asynchronously in the background. +* The indexing process does not block or interrupt database operations, such as writing data or running queries, + though queries may temporarily return [stale results](../indexes/stale-indexes) until the index is fully updated. + +--- + +**Separate storage**: + +* Indexes store their processed data separately, ensuring that the raw data remains unaffected. + This separation helps maintain the integrity of the raw data while allowing the index to optimize query performance. + +* If system resources become strained due to indexing, it may require adjustments to the index design, hardware, or other factors. + Learn more in [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources). + +{PANEL/} + +{PANEL: Types of indexes} + +* Indexes in RavenDB are categorized along the following axes: + * **Auto** indexes -vs- **Static** indexes + * **Map** indexes -vs- **Map-Reduce** indexes + * **Single-Collection** (Single-Map) indexes -vs- **Multi-Collection** (Multi-Map) indexes + +* For a detailed description of each type, refer to section [Index types](../studio/database/indexes/indexes-overview#index-types). + +{PANEL/} + +{PANEL: Basic example} + +In this example we create a static-index that indexes content from documents in the `Employees` [collection](../client-api/faq/what-is-a-collection). +This allows querying for _Employee_ documents by any of the index-fields (`FullName`, `LastName`, `Country`). + +{CONTENT-FRAME: } + +#### Define the index + +* The first step is to define the index. + One way to do this is by extending the `AbstractIndexCreationTask` class. + Learn more in [Define a static-index using a custom class](../indexes/creating-and-deploying#using-abstractindexcreationtask). +* Other methods to create a static-index are: + * [Creating a static-index using an Operation](../client-api/operations/maintenance/indexes/put-indexes) + * [Creating a static-index from the Studio](../studio/database/indexes/indexes-list-view) + +{CODE:java indexes_1@Indexes/WhatAreIndexes.java /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +#### Deploy the index + +* The next step is to deploy the index to the RavenDB server. + One way to do this is by calling `execute()` on the index instance. +* Additional methods for deploying static-indexes are described in [Deploy a static index](../indexes/creating-and-deploying#deploy-a-static-index). +* Once deployed, the indexing process will start indexing documents. -In the below example we create an index that maps documents from the `Employees` -[collection](../client-api/faq/what-is-a-collection) and enables querying by `FirstName`, -`LastName`, or both. +{CODE:java indexes_2@Indexes/WhatAreIndexes.java /} -- First, we need to create an index. - One way to create it is to use the `AbstractIndexCreationTask`, but there are other ways - available as well (you can read about them [here](../indexes/creating-and-deploying)). - {CODE:java indexes_1@Indexes/WhatAreIndexes.java /} +{CONTENT-FRAME/} +{CONTENT-FRAME: } -- The next step is to send an index to a server (more about index deployment options can - be found [here](../indexes/creating-and-deploying)) so the indexing process can start indexing documents. - {CODE:java indexes_2@Indexes/WhatAreIndexes.java /} +#### Query the index -- Now, our index can be queried and indexed results can be returned. - {CODE:java indexes_3@Indexes/WhatAreIndexes.java /} +* Now you can query the _Employees_ collection using the index. + In this example we query for _Employee_ documents, **filtering results based on index-fields** `LastName` and `Country`. + The results will include only the _Employee_ documents that match the query predicate. +* For detailed guidance on querying with an index, refer to the [Querying an index](../indexes/querying/query-index). +{CODE:java indexes_3@Indexes/WhatAreIndexes.java /} + +{CONTENT-FRAME/} {PANEL/} -{PANEL: Remarks} +{PANEL: Understanding index query results} + +{WARNING: } -{WARNING:Remember} -A frequent mistake is to handle indexes as SQL Views, but they are not analogous. -The **result of a query for the given index is a full document**, not only the fields -that were indexed. +A common mistake is treating indexes like SQL Views, but they are not analogous. +The results of a query for a given index are the **full raw documents** that match the query predicate, +and not just the indexed fields. + +This behavior can be changed by applying [Projections](../indexes/querying/projections), +which let you project the query results into selected fields instead of returning the entire document. -This behavior can be altered by [storing](../indexes/storing-data-in-index) fields -and applying [projections](../indexes/querying/projections). {WARNING/} +--- + +#### Viewing the resulting documents: + +For example, the results shown in the following image are the **documents** that match the query predicate. + +![Index query results - documents](images/index-query-results-1.png "Query results - Documents") + +1. This is the index query. + The query predicate filters the resulting documents based on the content of the index-fields. +2. Each row in the results represents a **matching document**. +3. In this example, the `LastName`, `FirstName`, `Title`, etc., are the raw **document-fields**. + +--- + +#### Viewing the index-entries: + +If you wish to **view the index-entries** that compose the index itself, +you can enable the option to show "raw index entries" instead of the matching documents. + +![Index query results - index entries](images/index-query-results-2.png "Query results - Index Entries") + +1. Query the index (no filtering is applied in this example). +2. Click the "Settings" button and toggle on "Show raw index-entries instead of matching documents". +3. Each row in the results represents an **index-entry**. +4. In this example, the `Country`, `FullName`, and `LastName` columns are the **index-fields**, + which were defined in the index definition. +5. This a **term**. + In this example, `usa` is a term generated by the analyzer for index-field `Country` from document `employees/4-a`. + +{PANEL/} + +{PANEL: If indexes exhaust system resources} + +* The indexing process utilizes machine resources to keep the data up-to-date for queries. + +* If indexing drains system resources, it may indicate one or more of the following: + * Indexes may have been defined in a way that causes inefficient processing. + * The [license](https://ravendb.net/buy) may need to be upgraded, + * Your [cloud instance](../cloud/cloud-instances#a-production-cloud-cluster) (if used) may require optimization. + * Hardware upgrades may be necessary to better support your workload. + +* Refer to the [Indexing Performance View](../studio/database/indexes/indexing-performance) in the Studio to monitor and analyze the indexing process. + This view provides graphical representations and detailed statistics of all index activities at each stage. + +* Additionally, refer to the [Common indexing issues](../studio/database/indexes/indexing-performance#common-indexing-issues) section + for troubleshooting and resolving indexing challenges. + {PANEL/} ## Related Articles @@ -67,7 +209,6 @@ and applying [projections](../indexes/querying/projections). ### Indexes - [Creating and Deploying Indexes](../indexes/creating-and-deploying) -- [Indexing Basics](../indexes/indexing-basics) - [Query an Index](../indexes/querying/query-index) - [Stale Indexes](../indexes/stale-indexes) @@ -75,3 +216,8 @@ and applying [projections](../indexes/querying/projections). - [Indexes: Overview](../studio/database/indexes/indexes-overview#indexes-overview) - [Studio Index List View](../studio/database/indexes/indexes-list-view) + +### Sharding + +- [Indexing in a Sharded DB](../sharding/indexing) +- [Querying in a Sharded DB](../sharding/querying) diff --git a/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.js.markdown b/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.js.markdown index 88803c1b08..478972c12d 100644 --- a/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.js.markdown +++ b/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.js.markdown @@ -1,61 +1,203 @@ -# What Indexes Are +# What are Indexes --- {NOTE: } -* Indexes are server-side functions that define which fields and what values documents can be searched on. - They are _the only way_ to satisfy queries in RavenDB. +* Indexes in RavenDB enable efficient querying by processing raw data and providing fast query results without scanning the entire dataset each and every time. + Learn more in the [Indexes overview](../indexes/what-are-indexes#indexes-overview) below. -* The indexing process is performed in the background and is triggered whenever data is added or changed. - This approach allows the server to respond quickly even when large amounts of data have changed and avoid - costly table scan operations. - -* The core of every index is its mapping function, that utilizes LINQ-like syntax. - The result of this mapping is converted to a [Lucene](http://lucene.apache.org/) index entry - which is persisted for future use to avoid re-indexing each time the query is issued and keep - response times minimal. - -* This page shows a basic example for the creation, deployment, and usage of an index. - Additional examples are available, among other places, on the [Indexing Basics](../indexes/indexing-basics) - and [Map Indexes](../indexes/map-indexes) pages. +* This page provides a [Basic example](../indexes/what-are-indexes#basic-example) of creating, deploying, and querying an index. + Additional examples can be found in [Creating and deploying indexes](../indexes/creating-and-deploying), [Map Indexes](../indexes/map-indexes), + and many other articles under the "Indexes" menu. * In this page: - * [Basic Example](../indexes/what-are-indexes#basic-example) - * [Remarks](../indexes/what-are-indexes#remarks) + * [Indexes overview](../indexes/what-are-indexes#indexes-overview) + * [Types of indexes](../indexes/what-are-indexes#types-of-indexes) + * [Basic example](../indexes/what-are-indexes#basic-example) + * [Understanding index query results](../indexes/what-are-indexes#understanding-index-query-results) + * [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources) {NOTE/} -{PANEL: Basic Example} +{PANEL: Indexes overview} + +**Indexes are fundamental**: + +* Indexes are fundamental to RavenDB’s query execution, enabling efficient query performance by processing the underlying data and delivering faster results. +* ALL queries in RavenDB use an index to deliver results and ensure optimal performance. + Learn more in [Queries always provide results using an index](../client-api/session/querying/how-to-query#queries-always-provide-results-using-an-index). + +--- + +**Main concepts**: + +* When discussing indexes in RavenDB, three key components come into play: + * The index definition + * The indexing process + * The indexed data + +* Each of these components is described in detail in [Indexes - The moving parts](../studio/database/indexes/indexes-overview#indexes---the-moving-parts). + +--- + +**The indexing process**: + +* The indexing process iterates over the raw documents, creating an **index-entry** for each document that is processed. + (Usually a single index-entry is created per raw document, unless working with a [fanout index](../indexes/indexing-nested-data#fanout-index---multiple-index-entries-per-document)). +* Each index-entry contains **index-fields**, and each index-field contains content (**index-terms**) that was generated from the raw documents, + as defined by the index definition and depending on the [analyzer](../indexes/using-analyzers) used. +* A map is built between the indexed-terms and the documents they originated from, + enabling you to query for documents based on the indexed data. + +--- + +**Automatic data processing**: + +* Once defined and deployed, an index will initially process the entire dataset. + After that, the index will only process documents that were modified, added or deleted. + This happens automatically without requiring direct user intervention. +* For example, if changes are made to documents in the "Orders" collection, + all indexes defined for the "Orders" collection will be triggered to update the index with the new data. +* This approach helps avoid costly table scans, allows the server to respond quickly, + and reduces the load on queries while optimizing machine resource usage. + +--- + +**Background operation**: + +* RavenDB indexes are designed to run asynchronously in the background. +* The indexing process does not block or interrupt database operations, such as writing data or running queries, + though queries may temporarily return [stale results](../indexes/stale-indexes) until the index is fully updated. + +--- + +**Separate storage**: + +* Indexes store their processed data separately, ensuring that the raw data remains unaffected. + This separation helps maintain the integrity of the raw data while allowing the index to optimize query performance. + +* If system resources become strained due to indexing, it may require adjustments to the index design, hardware, or other factors. + Learn more in [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources). + +{PANEL/} + +{PANEL: Types of indexes} + +* Indexes in RavenDB are categorized along the following axes: + * **Auto** indexes -vs- **Static** indexes + * **Map** indexes -vs- **Map-Reduce** indexes + * **Single-Collection** (Single-Map) indexes -vs- **Multi-Collection** (Multi-Map) indexes + +* For a detailed description of each type, refer to section [Index types](../studio/database/indexes/indexes-overview#index-types). + +{PANEL/} + +{PANEL: Basic example} + +In this example we create a static-index that indexes content from documents in the `Employees` [collection](../client-api/faq/what-is-a-collection). +This allows querying for _Employee_ documents by any of the index-fields (`FullName`, `LastName`, `Country`). + +{CONTENT-FRAME: } + +#### Define the index + +* The first step is to define the index. + One way to do this is by extending `AbstractJavaScriptIndexCreationTask`. + Learn more in [Define a static-index using a custom class](../indexes/creating-and-deploying#define-a-static-index-using-a-custom-class). +* Other methods to create a static-index are: + * [Creating a static-index using an Operation](../client-api/operations/maintenance/indexes/put-indexes) + * [Creating a static-index from the Studio](../studio/database/indexes/indexes-list-view) + +{CODE:nodejs indexes_1@indexes/whatAreIndexes.js /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +#### Deploy the index + +* The next step is to deploy the index to the RavenDB server. + One way to do this is by calling `execute()` on the index instance. +* Additional methods for deploying static-indexes are described in [Deploy a static index](../indexes/creating-and-deploying#deploy-a-static-index). +* Once deployed, the indexing process will start indexing documents. -In the below example we create an index that maps documents from the `Employees` -[collection](../client-api/faq/what-is-a-collection) and enables querying by `FirstName`, -`LastName`, or both. +{CODE:nodejs indexes_2@indexes/whatAreIndexes.js /} -- First, we need to create an index. - One way to create it is to use the `AbstractIndexCreationTask`, but there are other ways - available as well (you can read about them [here](../indexes/creating-and-deploying)). - {CODE:nodejs indexes_1@indexes/whatAreIndexes.js /} +{CONTENT-FRAME/} +{CONTENT-FRAME: } -- The next step is to send an index to a server (more about index deployment options can - be found [here](../indexes/creating-and-deploying)) so the indexing process can start indexing documents. - {CODE:nodejs indexes_2@indexes/whatAreIndexes.js /} +#### Query the index -- Now, our index can be queried and indexed results can be returned. - {CODE:nodejs indexes_3@indexes/whatAreIndexes.js /} +* Now you can query the _Employees_ collection using the index. + In this example we query for _Employee_ documents, **filtering results based on index-fields** `LastName` and `Country`. + The results will include only the _Employee_ documents that match the query predicate. +* For detailed guidance on querying with an index, refer to the [Querying an index](../indexes/querying/query-index). +{CODE:nodejs indexes_3@indexes/whatAreIndexes.js /} + +{CONTENT-FRAME/} {PANEL/} -{PANEL: Remarks} +{PANEL: Understanding index query results} + +{WARNING: } -{WARNING:Remember} -A frequent mistake is to handle indexes as SQL Views, but they are not analogous. -The **result of a query for the given index is a full document**, not only the fields -that were indexed. +A common mistake is treating indexes like SQL Views, but they are not analogous. +The results of a query for a given index are the **full raw documents** that match the query predicate, +and not just the indexed fields. + +This behavior can be changed by applying [Projections](../indexes/querying/projections), +which let you project the query results into selected fields instead of returning the entire document. -This behavior can be altered by [storing](../indexes/storing-data-in-index) fields -and applying [projections](../indexes/querying/projections). {WARNING/} +--- + +#### Viewing the resulting documents: + +For example, the results shown in the following image are the **documents** that match the query predicate. + +![Index query results - documents](images/index-query-results-1.png "Query results - Documents") + +1. This is the index query. + The query predicate filters the resulting documents based on the content of the index-fields. +2. Each row in the results represents a **matching document**. +3. In this example, the `LastName`, `FirstName`, `Title`, etc., are the raw **document-fields**. + +--- + +#### Viewing the index-entries: + +If you wish to **view the index-entries** that compose the index itself, +you can enable the option to show "raw index entries" instead of the matching documents. + +![Index query results - index entries](images/index-query-results-2.png "Query results - Index Entries") + +1. Query the index (no filtering is applied in this example). +2. Click the "Settings" button and toggle on "Show raw index-entries instead of matching documents". +3. Each row in the results represents an **index-entry**. +4. In this example, the `Country`, `FullName`, and `LastName` columns are the **index-fields**, + which were defined in the index definition. +5. This a **term**. + In this example, `usa` is a term generated by the analyzer for index-field `Country` from document `employees/4-a`. + +{PANEL/} + +{PANEL: If indexes exhaust system resources} + +* The indexing process utilizes machine resources to keep the data up-to-date for queries. + +* If indexing drains system resources, it may indicate one or more of the following: + * Indexes may have been defined in a way that causes inefficient processing. + * The [license](https://ravendb.net/buy) may need to be upgraded, + * Your [cloud instance](../cloud/cloud-instances#a-production-cloud-cluster) (if used) may require optimization. + * Hardware upgrades may be necessary to better support your workload. + +* Refer to the [Indexing Performance View](../studio/database/indexes/indexing-performance) in the Studio to monitor and analyze the indexing process. + This view provides graphical representations and detailed statistics of all index activities at each stage. + +* Additionally, refer to the [Common indexing issues](../studio/database/indexes/indexing-performance#common-indexing-issues) section + for troubleshooting and resolving indexing challenges. + {PANEL/} ## Related Articles @@ -67,7 +209,6 @@ and applying [projections](../indexes/querying/projections). ### Indexes - [Creating and Deploying Indexes](../indexes/creating-and-deploying) -- [Indexing Basics](../indexes/indexing-basics) - [Query an Index](../indexes/querying/query-index) - [Stale Indexes](../indexes/stale-indexes) @@ -75,3 +216,8 @@ and applying [projections](../indexes/querying/projections). - [Indexes: Overview](../studio/database/indexes/indexes-overview#indexes-overview) - [Studio Index List View](../studio/database/indexes/indexes-list-view) + +### Sharding + +- [Indexing in a Sharded DB](../sharding/indexing) +- [Querying in a Sharded DB](../sharding/querying) diff --git a/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.python.markdown b/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.python.markdown index 6d5c915a7d..c0747235df 100644 --- a/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.python.markdown +++ b/Documentation/6.0/Raven.Documentation.Pages/indexes/what-are-indexes.python.markdown @@ -1,61 +1,203 @@ -# What Indexes Are +# What are Indexes --- {NOTE: } -* Indexes are server-side functions that define which fields and what values documents can be searched on. - They are _the only way_ to satisfy queries in RavenDB. +* Indexes in RavenDB enable efficient querying by processing raw data and providing fast query results without scanning the entire dataset each and every time. + Learn more in the [Indexes overview](../indexes/what-are-indexes#indexes-overview) below. -* The indexing process is performed in the background and is triggered whenever data is added or changed. - This approach allows the server to respond quickly even when large amounts of data have changed and avoid - costly table scan operations. - -* The core of every index is its mapping function, that utilizes LINQ-like syntax. - The result of this mapping is converted to a [Lucene](http://lucene.apache.org/) index entry - which is persisted for future use to avoid re-indexing each time the query is issued and keep - response times minimal. - -* This page shows a basic example for the creation, deployment, and usage of an index. - Additional examples are available, among other places, on the [Indexing Basics](../indexes/indexing-basics) - and [Map Indexes](../indexes/map-indexes) pages. +* This page provides a [Basic example](../indexes/what-are-indexes#basic-example) of creating, deploying, and querying an index. + Additional examples can be found in [Creating and deploying indexes](../indexes/creating-and-deploying), [Map Indexes](../indexes/map-indexes), + and many other articles under the "Indexes" menu. * In this page: - * [Basic Example](../indexes/what-are-indexes#basic-example) - * [Remarks](../indexes/what-are-indexes#remarks) + * [Indexes overview](../indexes/what-are-indexes#indexes-overview) + * [Types of indexes](../indexes/what-are-indexes#types-of-indexes) + * [Basic example](../indexes/what-are-indexes#basic-example) + * [Understanding index query results](../indexes/what-are-indexes#understanding-index-query-results) + * [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources) {NOTE/} -{PANEL: Basic Example} +{PANEL: Indexes overview} + +**Indexes are fundamental**: -In the below example we create an index that maps documents from the `Employees` -[collection](../client-api/faq/what-is-a-collection) and enables querying by `FirstName`, -`LastName`, or both. +* Indexes are fundamental to RavenDB’s query execution, enabling efficient query performance by processing the underlying data and delivering faster results. +* ALL queries in RavenDB use an index to deliver results and ensure optimal performance. + Learn more in [Queries always provide results using an index](../client-api/session/querying/how-to-query#queries-always-provide-results-using-an-index). + +--- -- First, we need to create an index. - One way to create it is to use the `AbstractIndexCreationTask`, but there are other ways - available as well (you can read about them [here](../indexes/creating-and-deploying)). - {CODE:python indexes_1@Indexes/WhatAreIndexes.py /} +**Main concepts**: + +* When discussing indexes in RavenDB, three key components come into play: + * The index definition + * The indexing process + * The indexed data + +* Each of these components is described in detail in [Indexes - The moving parts](../studio/database/indexes/indexes-overview#indexes---the-moving-parts). + +--- + +**The indexing process**: + +* The indexing process iterates over the raw documents, creating an **index-entry** for each document that is processed. + (Usually a single index-entry is created per raw document, unless working with a [fanout index](../indexes/indexing-nested-data#fanout-index---multiple-index-entries-per-document)). +* Each index-entry contains **index-fields**, and each index-field contains content (**index-terms**) that was generated from the raw documents, + as defined by the index definition and depending on the [analyzer](../indexes/using-analyzers) used. +* A map is built between the indexed-terms and the documents they originated from, + enabling you to query for documents based on the indexed data. + +--- + +**Automatic data processing**: + +* Once defined and deployed, an index will initially process the entire dataset. + After that, the index will only process documents that were modified, added or deleted. + This happens automatically without requiring direct user intervention. +* For example, if changes are made to documents in the "Orders" collection, + all indexes defined for the "Orders" collection will be triggered to update the index with the new data. +* This approach helps avoid costly table scans, allows the server to respond quickly, + and reduces the load on queries while optimizing machine resource usage. + +--- -- The next step is to send an index to a server (more about index deployment options can - be found [here](../indexes/creating-and-deploying)) so the indexing process can start indexing documents. - {CODE:python indexes_2@Indexes/WhatAreIndexes.py /} +**Background operation**: -- Now, our index can be queried and indexed results can be returned. - {CODE:python indexes_3@Indexes/WhatAreIndexes.py /} +* RavenDB indexes are designed to run asynchronously in the background. +* The indexing process does not block or interrupt database operations, such as writing data or running queries, + though queries may temporarily return [stale results](../indexes/stale-indexes) until the index is fully updated. + +--- + +**Separate storage**: + +* Indexes store their processed data separately, ensuring that the raw data remains unaffected. + This separation helps maintain the integrity of the raw data while allowing the index to optimize query performance. + +* If system resources become strained due to indexing, it may require adjustments to the index design, hardware, or other factors. + Learn more in [If indexes exhaust system resources](../indexes/what-are-indexes#if-indexes-exhaust-system-resources). + +{PANEL/} + +{PANEL: Types of indexes} + +* Indexes in RavenDB are categorized along the following axes: + * **Auto** indexes -vs- **Static** indexes + * **Map** indexes -vs- **Map-Reduce** indexes + * **Single-Collection** (Single-Map) indexes -vs- **Multi-Collection** (Multi-Map) indexes + +* For a detailed description of each type, refer to section [Index types](../studio/database/indexes/indexes-overview#index-types). {PANEL/} -{PANEL: Remarks} +{PANEL: Basic example} + +In this example we create a static-index that indexes content from documents in the `Employees` [collection](../client-api/faq/what-is-a-collection). +This allows querying for _Employee_ documents by any of the index-fields (`FullName`, `LastName`, `Country`). + +{CONTENT-FRAME: } + +#### Define the index + +* The first step is to define the index. + One way to do this is by inheriting from `AbstractIndexCreationTask`. + Learn more in [Define a static-index using a custom class](../indexes/creating-and-deploying#define-a-static-index-using-a-custom-class). +* Other methods to create a static-index are: + * [Creating a static-index using an Operation](../client-api/operations/maintenance/indexes/put-indexes) + * [Creating a static-index from the Studio](../studio/database/indexes/indexes-list-view) + +{CODE:python indexes_1@Indexes/WhatAreIndexes.py /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } -{WARNING:Remember} -A frequent mistake is to handle indexes as SQL Views, but they are not analogous. -The **result of a query for the given index is a full document**, not only the fields -that were indexed. +#### Deploy the index + +* The next step is to deploy the index to the RavenDB server. + One way to do this is by calling `execute()` on the index instance. +* Additional methods for deploying static-indexes are described in [Deploy a static index](../indexes/creating-and-deploying#deploy-a-static-index). +* Once deployed, the indexing process will start indexing documents. + +{CODE:python indexes_2@Indexes/WhatAreIndexes.py /} + +{CONTENT-FRAME/} +{CONTENT-FRAME: } + +#### Query the index + +* Now you can query the _Employees_ collection using the index. + In this example we query for _Employee_ documents, **filtering results based on index-fields** `LastName` and `Country`. + The results will include only the _Employee_ documents that match the query predicate. +* For detailed guidance on querying with an index, refer to the [Querying an index](../indexes/querying/query-index). + +{CODE:python indexes_3@Indexes/WhatAreIndexes.py /} + +{CONTENT-FRAME/} +{PANEL/} + +{PANEL: Understanding index query results} + +{WARNING: } + +A common mistake is treating indexes like SQL Views, but they are not analogous. +The results of a query for a given index are the **full raw documents** that match the query predicate, +and not just the indexed fields. + +This behavior can be changed by applying [Projections](../indexes/querying/projections), +which let you project the query results into selected fields instead of returning the entire document. -This behavior can be altered by [storing](../indexes/storing-data-in-index) fields -and applying [projections](../indexes/querying/projections). {WARNING/} +--- + +#### Viewing the resulting documents: + +For example, the results shown in the following image are the **documents** that match the query predicate. + +![Index query results - documents](images/index-query-results-1.png "Query results - Documents") + +1. This is the index query. + The query predicate filters the resulting documents based on the content of the index-fields. +2. Each row in the results represents a **matching document**. +3. In this example, the `LastName`, `FirstName`, `Title`, etc., are the raw **document-fields**. + +--- + +#### Viewing the index-entries: + +If you wish to **view the index-entries** that compose the index itself, +you can enable the option to show "raw index entries" instead of the matching documents. + +![Index query results - index entries](images/index-query-results-2.png "Query results - Index Entries") + +1. Query the index (no filtering is applied in this example). +2. Click the "Settings" button and toggle on "Show raw index-entries instead of matching documents". +3. Each row in the results represents an **index-entry**. +4. In this example, the `Country`, `FullName`, and `LastName` columns are the **index-fields**, + which were defined in the index definition. +5. This a **term**. + In this example, `usa` is a term generated by the analyzer for index-field `Country` from document `employees/4-a`. + +{PANEL/} + +{PANEL: If indexes exhaust system resources} + +* The indexing process utilizes machine resources to keep the data up-to-date for queries. + +* If indexing drains system resources, it may indicate one or more of the following: + * Indexes may have been defined in a way that causes inefficient processing. + * The [license](https://ravendb.net/buy) may need to be upgraded, + * Your [cloud instance](../cloud/cloud-instances#a-production-cloud-cluster) (if used) may require optimization. + * Hardware upgrades may be necessary to better support your workload. + +* Refer to the [Indexing Performance View](../studio/database/indexes/indexing-performance) in the Studio to monitor and analyze the indexing process. + This view provides graphical representations and detailed statistics of all index activities at each stage. + +* Additionally, refer to the [Common indexing issues](../studio/database/indexes/indexing-performance#common-indexing-issues) section + for troubleshooting and resolving indexing challenges. + {PANEL/} ## Related Articles @@ -67,7 +209,6 @@ and applying [projections](../indexes/querying/projections). ### Indexes - [Creating and Deploying Indexes](../indexes/creating-and-deploying) -- [Indexing Basics](../indexes/indexing-basics) - [Query an Index](../indexes/querying/query-index) - [Stale Indexes](../indexes/stale-indexes) @@ -77,5 +218,6 @@ and applying [projections](../indexes/querying/projections). - [Studio Index List View](../studio/database/indexes/indexes-list-view) ### Sharding -- [Indexing in a Sharded DB](../sharding/indexing) + +- [Indexing in a Sharded DB](../sharding/indexing) - [Querying in a Sharded DB](../sharding/querying) diff --git a/Documentation/6.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown b/Documentation/6.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown index 69a15eb5e6..b55988c655 100644 --- a/Documentation/6.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown +++ b/Documentation/6.0/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown @@ -5,7 +5,7 @@ * The below **indexing configuration keys** can be modified via any of the following methods: * As explained in the [Config overview](../../server/configuration/configuration-options) article - * Set a custom configuration per index from the [Client API](../../indexes/creating-and-deploying#creating-an-index-with-custom-configuration) + * Set a custom configuration per index from the [Client API](../../indexes/creating-and-deploying#customizing-configuration) * Set a custom configuration per index from the [Studio](../../studio/database/indexes/create-map-index#configuration) {NOTE/} diff --git a/Documentation/6.0/Samples/csharp/Raven.Documentation.Samples/Indexes/WhatAreIndexes.cs b/Documentation/6.0/Samples/csharp/Raven.Documentation.Samples/Indexes/WhatAreIndexes.cs index 52274ad228..39fa6be19e 100644 --- a/Documentation/6.0/Samples/csharp/Raven.Documentation.Samples/Indexes/WhatAreIndexes.cs +++ b/Documentation/6.0/Samples/csharp/Raven.Documentation.Samples/Indexes/WhatAreIndexes.cs @@ -1,6 +1,5 @@ using System.Collections.Generic; using System.Linq; - using Raven.Client.Documents; using Raven.Client.Documents.Indexes; using Raven.Documentation.Samples.Orders; @@ -10,15 +9,28 @@ namespace Raven.Documentation.Samples.Indexes public class WhatAreIndexes { #region indexes_1 - public class Employees_ByFirstAndLastName : AbstractIndexCreationTask + // Define the index: + // ================= + + public class Employees_ByNameAndCountry : AbstractIndexCreationTask { - public Employees_ByFirstAndLastName() + public class IndexEntry + { + // The index-fields + public string LastName { get; set; } + public string FullName { get; set; } + public string Country { get; set; } + } + + public Employees_ByNameAndCountry() { Map = employees => from employee in employees - select new + select new IndexEntry() { - FirstName = employee.FirstName, - LastName = employee.LastName + // Define the content for each index-field + LastName = employee.LastName, + FullName = employee.FirstName + " " + employee.LastName, + Country = employee.Address.Country }; } } @@ -29,16 +41,24 @@ public WhatAreIndexes() using (var store = new DocumentStore()) { #region indexes_2 - // save index on server - new Employees_ByFirstAndLastName().Execute(store); + // Deploy the index to the server: + // =============================== + + new Employees_ByNameAndCountry().Execute(store); #endregion using (var session = store.OpenSession()) { #region indexes_3 - IList results = session - .Query() - .Where(x => x.FirstName == "Robert") + // Query the database using the index: + // =================================== + + IList employeesFromUK = session + .Query() + // Here we query for all Employee documents that are from the UK + // and have 'King' in their LastName field: + .Where(x => x.LastName == "King" && x.Country == "UK") + .OfType() .ToList(); #endregion } diff --git a/Documentation/6.0/Samples/java/src/test/java/net/ravendb/Indexes/WhatAreIndexes.java b/Documentation/6.0/Samples/java/src/test/java/net/ravendb/Indexes/WhatAreIndexes.java index c13097569e..9beec2dd91 100644 --- a/Documentation/6.0/Samples/java/src/test/java/net/ravendb/Indexes/WhatAreIndexes.java +++ b/Documentation/6.0/Samples/java/src/test/java/net/ravendb/Indexes/WhatAreIndexes.java @@ -10,11 +10,15 @@ public class WhatAreIndexes { //region indexes_1 - public static class Employees_ByFirstAndLastName extends AbstractIndexCreationTask { - public Employees_ByFirstAndLastName() { - map = "docs.Employees.Select(employee => new {" + - " FirstName = employee.FirstName," + - " LastName = employee.LastName" + + // Define the index: + // ================= + + public static class Employees_ByNameAndCountry extends AbstractIndexCreationTask { + public Employees_ByNameAndCountry() { + map = "docs.Employees.Select(employee => new { " + + " LastName = employee.LastName, " + + " FullName = (employee.FirstName + \" \") + employee.LastName, " + + " Country = employee.Address.Country " + "})"; } } @@ -26,15 +30,23 @@ private static class Employee { public WhatAreIndexes() { try (IDocumentStore store = new DocumentStore()) { //region indexes_2 - // save index on server - new Employees_ByFirstAndLastName().execute(store); + // Deploy the index to the server: + // =============================== + + new Employees_ByNameAndCountry().execute(store); //endregion try (IDocumentSession session = store.openSession()) { //region indexes_3 - List results = session - .query(Employee.class, Employees_ByFirstAndLastName.class) - .whereEquals("FirstName", "Robert") + // Query the database using the index: + // =================================== + + List employeesFromUK = session + .query(Employee.class, Employees_ByNameAndCountry.class) + // Here we query for all Employee documents that are from the UK + // and have 'King' in their LastName field: + .whereEquals("LastName", "King") + .whereEquals("Country", "UK") .toList(); //endregion } diff --git a/Documentation/6.0/Samples/nodejs/indexes/whatAreIndexes.js b/Documentation/6.0/Samples/nodejs/indexes/whatAreIndexes.js index bc3132e2dd..188d310478 100644 --- a/Documentation/6.0/Samples/nodejs/indexes/whatAreIndexes.js +++ b/Documentation/6.0/Samples/nodejs/indexes/whatAreIndexes.js @@ -1,43 +1,58 @@ -import { +import { DocumentStore, - AbstractIndexCreationTask, - IndexDefinition, - PutIndexesOperation + AbstractJavaScriptIndexCreationTask } from "ravendb"; const store = new DocumentStore(); - //region indexes_1 - class Employees_ByFirstAndLastName extends AbstractIndexCreationTask { - constructor() { - super(); - this.map = "docs.Employees.Select(employee => new {" + - " FirstName = employee.FirstName," + - " LastName = employee.LastName" + - "})"; - } +//region indexes_1 +// Define the index: +// ================= + +class Employees_ByNameAndCountry extends AbstractJavaScriptIndexCreationTask { + constructor() { + super(); + + this.map("Employees", employee => { + return { + // Define the content for each index-field: + // ======================================== + LastName: employee.LastName, + FullName: employee.FirstName + " " + employee.LastName, + Country: employee.Address.Country + }; + }); } +} +//endregion + +class Employee { } + +async function whatAreIndexes() { + + //region indexes_2 + // Deploy the index to the server: + // =============================== + + const employeesIndex = new Employees_ByNameAndCountry(); + await employeesIndex.execute(store); //endregion - class Employee { } - - async function whatAreIndexes() { - - //region indexes_2 - // save index on server - const employeesIndex = new Employees_ByFirstAndLastName(); - await employeesIndex.execute(store); - //endregion - - { - const session = store.openSession(); - //region indexes_3 - const results = await session - .query({ indexName: employeesIndex.getIndexName() }) - .whereEquals("FirstName", "Robert") - .all(); - //endregion - } - + { + const session = store.openSession(); + //region indexes_3 + // Query the database using the index: + // =================================== + + const employeesFromUK = await session + .query({ indexName: employeesIndex.getIndexName() }) + // Here we query for all Employee documents that are from the UK + // and have 'King' in their LastName field: + .whereEquals("LastName", "King") + .whereEquals("Country", "UK") + .all(); + //endregion } +} + diff --git a/Documentation/6.0/Samples/python/Indexes/WhatAreIndexes.py b/Documentation/6.0/Samples/python/Indexes/WhatAreIndexes.py index d9ee7ceecd..29d1c098e1 100644 --- a/Documentation/6.0/Samples/python/Indexes/WhatAreIndexes.py +++ b/Documentation/6.0/Samples/python/Indexes/WhatAreIndexes.py @@ -4,12 +4,21 @@ # region indexes_1 -class Employees_ByFirstAndLastName(AbstractIndexCreationTask): +# Define the index: +# ================= + +class Employees_ByNameAndCountry(AbstractIndexCreationTask): def __init__(self): super().__init__() - self.map = "from employee in docs.Employees select new {first_name = employee.FirstName, last_name = employee.LastName}" - - + + self.map = """ + from employee in docs.Employees + select new { + LastName = employee.LastName, + FullName = employee.FirstName + " " + employee.LastName, + Country = employee.Address.country + } + """ # endregion @@ -21,15 +30,22 @@ def test_what_are_indexes(self): with self.embedded_server.get_document_store("WhatAreIndexes") as store: with store.open_session() as session: # region indexes_2 - # save index on server - Employees_ByFirstAndLastName().execute(store) + # Deploy the index to the server: + # =============================== + + Employees_ByNameAndCountry().execute(store) # endregion with store.open_session() as session: # region indexes_3 - results = list( - session.query_index_type(Employees_ByFirstAndLastName, Employee).where_equals( - "first_name", "Robert" - ) + # Query the database using the index: + # =================================== + + employeesFromUK = list( + session.query_index_type(Employees_ByNameAndCountry, Employee) + # Here we query for all Employee documents that are from the UK + # and have 'King' in their LastName field: + .where_equals("Country", "UK") + .where_equals("LastName", "King") ) # endregion diff --git a/Documentation/6.2/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown b/Documentation/6.2/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown index a0c4690420..d495a2c2cf 100644 --- a/Documentation/6.2/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown +++ b/Documentation/6.2/Raven.Documentation.Pages/server/configuration/indexing-configuration.markdown @@ -5,7 +5,7 @@ * The below **indexing configuration keys** can be modified via any of the following methods: * As explained in the [Config overview](../../server/configuration/configuration-options) article - * Set a custom configuration per index from the [Client API](../../indexes/creating-and-deploying#creating-an-index-with-custom-configuration) + * Set a custom configuration per index from the [Client API](../../indexes/creating-and-deploying#customizing-configuration) * Set a custom configuration per index from the [Studio](../../studio/database/indexes/create-map-index#configuration) {NOTE/}