diff --git a/_api-reference/grpc-apis/bulk.md b/_api-reference/grpc-apis/bulk.md index 17cf8ae3e9f..59bd578b0ac 100644 --- a/_api-reference/grpc-apis/bulk.md +++ b/_api-reference/grpc-apis/bulk.md @@ -17,9 +17,9 @@ To submit gRPC requests, you must have a set of protobufs on the client side. Fo ## gRPC service and method -gRPC Document APIs reside in the [DocumentService](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/services/document_service.proto#L21). +gRPC Document APIs reside in the [DocumentService](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/services/document_service.proto#L22). -You can submit bulk requests by invoking the [`Bulk`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/services/document_service.proto#L23) gRPC method within the `DocumentService`. The method takes in a [`BulkRequest`](#bulkrequest-fields) and returns a [`BulkResponse`](#bulkresponsebody-fields). +You can submit bulk requests by invoking the [`Bulk`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/services/document_service.proto#L24) gRPC method within the `DocumentService`. The method takes a [`BulkRequest`](#bulkrequest-fields) and returns a [`BulkResponse`](#bulkresponsebody-fields). ## Document format @@ -40,63 +40,76 @@ For a gRPC Bulk API request, provide the same document in Base64 encoding: ## BulkRequest fields -The [`BulkRequest`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L16) message is the top-level container for a gRPC bulk operation. It accepts the following fields. +The [`BulkRequest`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L17) message is the top-level container for a gRPC bulk operation. It accepts the following fields. | Field | Protobuf type | Description | | :---- | :---- | :---- | -| `request_body` | `repeated `[`BulkRequestBody`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L53) | The list of bulk operations, each containing one of the operation types (`index`/`create`/`update`/`delete`). Required. | +| `request_body` | `repeated `[`BulkRequestBody`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L48) | The list of bulk operations, each containing one of the operation types (`index`/`create`/`update`/`delete`). Required. | | `index` | `string` | The default index for all operations unless overridden in `request_body`. Specifying the `index` in the `BulkRequest` means that you don't need to include it in the [BulkRequestBody](#bulkrequestbody-fields). Optional. | -| `source` | [`SourceConfigParam`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L154) | Controls whether to return the full `_source`, no `_source`, or only specific fields from `_source` in the response. Optional. | -| `source_excludes` | `repeated string` | Fields to exclude from `source`. Optional. | -| `source_includes` | `repeated string` | Fields to include from `source`. Optional. | +| `x_source` | [`SourceConfigParam`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L199) | Controls whether to return the full `_source`, no `_source`, or only specific fields from `_source` in the response. Optional. | +| `x_source_excludes` | `repeated string` | Fields to exclude from `source`. Optional. | +| `x_source_includes` | `repeated string` | Fields to include from `source`. Optional. | | `pipeline` | `string` | The preprocessing ingest pipeline ID. Optional. | -| `refresh` | [`Refresh`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L307) | Whether to refresh shards after indexing. Optional. | +| `refresh` | [`Refresh`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L264) | Whether to refresh shards after indexing. Optional. | | `require_alias` | `bool` | If `true`, actions must target an alias. Optional. | | `routing` | `string` | The routing value for shard assignment. Optional. | | `timeout` | `string` | The timeout duration (for example, `1m`). Optional. | | `type` (Deprecated) | `string` | The document type (always `_doc`). Optional. | -| `wait_for_active_shards` | [`WaitForActiveShards`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L13) | The minimum number of active shards to wait for. Optional. | +| `wait_for_active_shards` | [`WaitForActiveShards`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L23) | The minimum number of active shards to wait for. Optional. | +| `global_params` | [`GlobalParams`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L14) | Global parameters for the request. Optional. | ## BulkRequestBody fields -The [`BulkRequestBody`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L53) message represents a single document-level operation within a `BulkRequest`. It accepts the following fields. +The [`BulkRequestBody`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L48) message represents a single document-level operation within a `BulkRequest`. It accepts the following fields. + +| Field | Protobuf type | Description | +| :---- | :---- | :---- | +| `operation_container` | [`OperationContainer`](#operationcontainer-fields) | The operation to perform (`index`, `create`, `update`, or `delete`). Required. | +| `update_action` | [`UpdateAction`](#updateaction-fields) | Additional update-specific options. Optional. | +| `object` | `bytes` | The full document content used with `create` and `index` operations. Optional. | + +## OperationContainer fields + +The [`OperationContainer`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L57) message contains exactly one operation type. It accepts the following fields. | Field | Protobuf type | Description | | :---- | :---- | :---- | | `index` | [`IndexOperation`](#index) | Index a document. Replaces the document if it already exists. | -| `create` | [`CreateOperation`](#create) | Create a new document. Fails if the document already exists. | +| `create` | [`WriteOperation`](#create) | Create a new document. Fails if the document already exists. | | `update` | [`UpdateOperation`](#update) | Partially update a document or use upsert/script options. | | `delete` | [`DeleteOperation`](#delete) | Delete a document by ID. | + +## UpdateAction fields + +The [`UpdateAction`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L71) message provides additional options for update operations. It accepts the following fields. + +| Field | Protobuf type | Description | +| :---- | :---- | :---- | | `detect_noop` | `bool` | If `true`, skips the update if the document content hasn't changed. Optional. Default is `true`. | | `doc` | `bytes` | Partial or full document data for `update` operations. Optional. | | `doc_as_upsert` | `bool` | If `true`, treats the document as the full upsert document if the target document doesn't exist. Only valid for the `update` operation. Optional. | -| `script` | [`Script`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L27) | A script to apply to the document (used with `update`). Optional. | +| `script` | [`Script`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L36) | A script to apply to the document (used with `update`). Optional. | | `scripted_upsert` | `bool` | If `true`, executes the script whether or not the document exists. Optional. | -| `source` | [`SourceConfig`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L176) | Controls how the document source is fetched or filtered. Optional. | | `upsert` | `bytes` | The full document to use if the target does not exist. Used with `script`. Optional. | -| `object` | `bytes` | The full document content used with `create` operations. Optional. | +| `x_source` | [`SourceConfig`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L230) | Controls how the document source is fetched or filtered. Optional. | ### Create -`CreateOperation` adds a new document only if it doesn't already exist. +`WriteOperation` adds a new document only if it doesn't already exist. -The document itself must be provided in the `object` field, outside of the `CreateOperation` message. +The document itself must be provided in the `object` field of the `BulkRequestBody` message. The following optional fields can also be provided. | Field | Protobuf type | Description | | :---- | :---- | :---- | -| `id` | `string` | The document ID. If omitted, one is auto-generated. Optional. | -| `index` | `string` | The target index. Required if not set globally in the `BulkRequest`. Optional. | +| `x_id` | `string` | The document ID. If omitted, one is auto-generated. Optional. | +| `x_index` | `string` | The target index. Required if not set globally in the `BulkRequest`. Optional. | | `routing` | `string` | A custom routing value used to control shard placement. Optional. | -| `if_primary_term` | `int64` | Used for concurrency control. The operation only runs if the document's primary term matches this value. Optional. | -| `if_seq_no` | `int64` | Used for concurrency control. The operation only runs if the document's sequence number matches this value. Optional. | -| `version` | `int64` | The explicit document version for concurrency control. Optional. | -| `version_type` | [`VersionType`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L99) | Controls version matching behavior. Optional. | | `pipeline` | `string` | The preprocessing ingest pipeline ID. Optional. | -| `require_alias` | `bool` | Enforces the use of index aliases only. Optional. | +| `require_alias` | `bool` | If `true`, requires that all actions target an index alias rather than an index. Default is `false`. Optional. | #### Example request @@ -107,9 +120,11 @@ The following example shows a bulk request with a `create` operation. It creates "index": "movies", "request_body": [ { - "create": { - "index": "movies", - "id": "tt1375666" + "operation_container": { + "create": { + "x_index": "movies", + "x_id": "tt1375666" + } }, "object": "eyJ0aXRsZSI6ICJJbmNlcHRpb24iLCAieWVhciI6IDIwMTB9" } @@ -123,13 +138,13 @@ The `DeleteOperation` removes a document by ID. It accepts the following fields. | Field | Protobuf type | Description | | :---- | :---- | :---- | -| `id` | `string` | The ID of the document to delete. Required. | -| `index` | `string` | The target index. Required if not set globally in the `BulkRequest`. Optional. | +| `x_id` | `string` | The ID of the document to delete. Required. | +| `x_index` | `string` | The target index. Required if not set globally in the `BulkRequest`. Optional. | | `routing` | `string` | A custom routing value used to control shard placement. Optional. | | `if_primary_term` | `int64` | Used for concurrency control. The operation only runs if the document's primary term matches this value. Optional. | | `if_seq_no` | `int64` | Used for concurrency control. The operation only runs if the document's sequence number matches this value. Optional. | | `version` | `int64` | The explicit document version for concurrency control. Optional. | -| `version_type` | [`VersionType`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L99) | Controls version matching behavior. Optional. | +| `version_type` | [`VersionType`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L95) | Controls version matching behavior. Optional. | #### Example request @@ -140,9 +155,11 @@ The following example shows a bulk request with a `delete` operation. It deletes "index": "movies", "request_body": [ { - "delete": { - "index": "movies", - "id": "tt1392214" + "operation_container": { + "delete": { + "x_index": "movies", + "x_id": "tt1392214" + } } } ] @@ -154,22 +171,22 @@ The following example shows a bulk request with a `delete` operation. It deletes The `IndexOperation` creates or overwrites a document. If an ID is not provided, one is generated. -The document itself is provided in the `doc` field, outside of the `IndexOperation` message. +The document itself is provided in the `object` field of the `BulkRequestBody` message. The following optional fields can also be provided. | Field | Protobuf type | Description | | :---- | :---- | :---- | -| `id` | `string` | The document ID. If omitted, one is auto-generated. Optional. | -| `index` | `string` | The target index. Required only if not set globally in the `BulkRequest`. | +| `x_id` | `string` | The document ID. If omitted, one is auto-generated. Optional. | +| `x_index` | `string` | The target index. Required only if not set globally in the `BulkRequest`. | | `routing` | `string` | A custom routing value used to control shard placement. Optional. | | `if_primary_term` | `int64` | Used for concurrency control. The operation only runs if the document's primary term matches this value. Optional. | | `if_seq_no` | `int64` | Used for concurrency control. The operation only runs if the document's sequence number matches this value. Optional. | -| `op_type` | [`OpType`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L91) | The operation type. Controls the overwriting behavior. Valid values are `index` (default) and `create`. Optional. | +| `op_type` | [`OpType`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L89) | The operation type. Controls the overwriting behavior. Valid values are `index` (default) and `create`. Optional. | | `version` | `int64` | The explicit document version for concurrency control. Optional. | -| `version_type` | [`VersionType`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L99) | Controls version matching behavior. Optional. | +| `version_type` | [`VersionType`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L95) | Controls version matching behavior. Optional. | | `pipeline` | `string` | The preprocessing ingest pipeline ID. Optional. | -| `require_alias` | `bool` | Enforces the use of index aliases only. Optional. | +| `require_alias` | `bool` | If `true`, requires that all actions target an index alias rather than an index. Default is `false`. Optional. | #### Example request @@ -181,11 +198,13 @@ The following example shows a bulk request with an `index` operation. It indexes "index": "movies", "request_body": [ { - "index": { - "index": "movies", - "id": "tt0468569" + "operation_container": { + "index": { + "x_index": "movies", + "x_id": "tt0468569" + } }, - "doc": "eyJ0aXRsZSI6ICJUaGUgRGFyayBLbmlnaHQiLCAieWVhciI6IDIwMDh9" + "object": "eyJ0aXRsZSI6ICJUaGUgRGFyayBLbmlnaHQiLCAieWVhciI6IDIwMDh9" } ] } @@ -196,18 +215,18 @@ The following example shows a bulk request with an `index` operation. It indexes The `UpdateOperation` performs partial document updates. -The document itself is provided in the `doc` field within the `BulkRequestBody` message. +The update options are provided in the `update_action` field within the `BulkRequestBody` message. -All `UpdateOperation` fields, listed in the following table, are optional except for `id`. +All `UpdateOperation` fields, listed in the following table, are optional except for `x_id`. | Field | Protobuf type | Description | | :---- | :---- | :---- | -| `id` | `string` | The ID of the document to update. Required. | -| `index` | `string` | The target index. Required if not set globally in the `BulkRequest`. Optional. | +| `x_id` | `string` | The ID of the document to update. Required. | +| `x_index` | `string` | The target index. Required if not set globally in the `BulkRequest`. Optional. | | `routing` | `string` | A custom routing value used to control shard placement. Optional. | | `if_primary_term` | `int64` | Used for concurrency control. The operation only runs if the document's primary term matches this value. Optional. | | `if_seq_no` | `int64` | Used for concurrency control. The operation only runs if the document's sequence number matches this value. Optional. | -| `require_alias` | `bool` | Enforces the use of index aliases only. Optional. | +| `require_alias` | `bool` | If `true`, requires that all actions target an index alias rather than an index. Default is `false`. Optional. | | `retry_on_conflict` | `int32` | The number of times to retry the operation if a version conflict occurs. Optional. | @@ -220,12 +239,16 @@ The following example shows a bulk request with an `update` operation. It will u "index": "movies", "request_body": [ { - "update": { - "index": "movies", - "id": "tt1375666" + "operation_container": { + "update": { + "x_index": "movies", + "x_id": "tt1375666" + } }, - "doc": "eyJ5ZWFyIjogMjAxMX0=", - "detect_noop": true + "update_action": { + "doc": "eyJ5ZWFyIjogMjAxMX0=", + "detect_noop": true + } } ] } @@ -297,7 +320,7 @@ The gRPC Bulk API provides the following response fields. ### BulkResponseBody fields -The [`BulkResponse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L211) message wraps either a `BulkResponseBody` for successful requests or a `BulkErrorResponse` for failed requests. The `BulkResponseBody` provides a summary and per-item result of a bulk operation and contains the following fields. +The [`BulkResponse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L188) message wraps either a `BulkResponseBody` for successful requests or a `BulkErrorResponse` for failed requests. The `BulkResponseBody` provides a summary and per-item result of a bulk operation and contains the following fields. | Field | Protobuf type | Description | | :---- | :---- | :---- | @@ -326,17 +349,17 @@ Each `ResponseItem` corresponds to a single operation in the request. It contain | Field | Protobuf type | Description | | :---- | :---- | :---- | | `type` | `string` | The document type. | -| `id` | [`ResponseItem.Id`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L254) | The document ID associated with the operation. Can be `null`. | +| `id` | [`ResponseItem.Id`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L223) | The document ID associated with the operation. Can be `null`. | | `index` | `string` | The name of the index associated with the operation. If a data stream was targeted, this is the backing index. | | `status` | `int32` | The HTTP status code returned for the operation. *(Note: This field may be replaced with a gRPC code in the future.)* | -| `error` | [`ErrorCause`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L239) | Contains additional information about a failed operation. | +| `error` | [`ErrorCause`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L293) | Contains additional information about a failed operation. | | `primary_term` | `int64` | The primary term assigned to the document. | | `result` | `string` | The operation result. Valid values are `created`, `deleted`, and `updated`. | | `seq_no` | `int64` | A sequence number assigned to the document to maintain version order. | -| `shards` | [`ShardInfo`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L286) | Shard information for the operation (only returned for successful actions). | +| `shards` | [`ShardInfo`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L340) | Shard information for the operation (only returned for successful actions). | | `version` | `int64` | The document version (only returned for successful actions). | | `forced_refresh` | `bool` | If `true`, forces the document to become visible immediately after the operation. | -| `get` | [`InlineGetDictUserDefined`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/document.proto#L290) | Contains the document `source` returned from an inline get, if requested. | +| `get` | [`InlineGetDictUserDefined`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/document.proto#L266) | Contains the document `source` returned from an inline get, if requested. | ## Example response diff --git a/_api-reference/grpc-apis/index.md b/_api-reference/grpc-apis/index.md index 1e71c7dab66..af376778688 100644 --- a/_api-reference/grpc-apis/index.md +++ b/_api-reference/grpc-apis/index.md @@ -15,7 +15,7 @@ redirect_from: **Bulk and k-NN search generally available 3.2** {: .label .label-green } -Starting with OpenSearch version 3.2, the gRPC [Bulk API]({{site.url}}{{site.baseurl}}/api-reference/grpc-apis/bulk/) and [k-NN search queries]({{site.url}}{{site.baseurl}}/api-reference/grpc-apis/knn/) are generally available. These use [protobuf version 0.6.0](https://github.com/opensearch-project/opensearch-protobufs/releases/tag/0.6.0). However, expect updates to the protobuf structure as the feature matures in upcoming versions. Other gRPC search functionality remains experimental and not recommended for production use. For updates on the progress of these features or to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/OpenSearch/issues/16787). +Starting with OpenSearch version 3.2, the gRPC [Bulk API]({{site.url}}{{site.baseurl}}/api-reference/grpc-apis/bulk/) and [k-NN search queries]({{site.url}}{{site.baseurl}}/api-reference/grpc-apis/knn/) are generally available. These use [protobuf version 0.19.0](https://github.com/opensearch-project/opensearch-protobufs/releases/tag/0.19.0). However, expect updates to the protobuf structure as the feature matures in upcoming versions. Other gRPC search functionality remains experimental and not recommended for production use. For updates on the progress of these features or to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/OpenSearch/issues/16787). {: .note} The OpenSearch gRPC functionality provides an alternative, high-performance transport layer using [gRPC](https://grpc.io/) for communication with OpenSearch. It uses protocol buffers over gRPC for lower overhead and faster serialization. This reduces overhead, speeds up serialization, and improves request-side latency, based on initial benchmarking results. @@ -109,8 +109,8 @@ These settings are similar to the [HTTP Network settings]({{site.url}}{{site.bas To submit gRPC requests, you must have a set of protobufs on the client side. You can obtain the protobufs in the following ways: -- **Raw protobufs**: Download the raw protobuf schema from the [OpenSearch Protobufs GitHub repository (v0.6.0)](https://github.com/opensearch-project/opensearch-protobufs/releases/tag/0.6.0). You can then generate client-side code using the protocol buffer compilers for the [supported languages](https://grpc.io/docs/languages/). -- **Java client-side programs only**: Download the `opensearch-protobufs` jar from the [Maven Central repository](https://repo1.maven.org/maven2/org/opensearch/protobufs/0.6.0). +- **Raw protobufs**: Download the raw protobuf schema from the [OpenSearch Protobufs GitHub repository (v0.19.0)](https://github.com/opensearch-project/opensearch-protobufs/releases/tag/0.19.0). You can then generate client-side code using the protocol buffer compilers for the [supported languages](https://grpc.io/docs/languages/). +- **Java client-side programs only**: Download the `opensearch-protobufs` jar from the [Maven Central repository](https://repo1.maven.org/maven2/org/opensearch/protobufs/0.19.0). - **Python client-side programs only**: Download the `opensearch-protobufs` package from the [PyPI repository](https://pypi.org/project/opensearch-protobufs/). ## Supported APIs diff --git a/_api-reference/grpc-apis/knn.md b/_api-reference/grpc-apis/knn.md index 9964dedb3e6..d47dea4438b 100644 --- a/_api-reference/grpc-apis/knn.md +++ b/_api-reference/grpc-apis/knn.md @@ -22,15 +22,15 @@ To submit gRPC requests, you must have a set of protobufs on the client side. Fo ## gRPC service and method -gRPC k-NN APIs reside in the [`SearchService`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/services/search_service.proto#L22), the same service used for general search operations. +gRPC k-NN APIs reside in the [`SearchService`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/services/search_service.proto#L22), the same service used for general search operations. -You can submit k-NN search requests by invoking the [`Search`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/services/search_service.proto#L23) gRPC method within the `SearchService`, using a [`KnnQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1058) within the search request. The method takes a [`SearchRequest`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L18) and returns a [`SearchResponse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L317). +You can submit k-NN search requests by invoking the [`Search`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/services/search_service.proto#L23) gRPC method within the `SearchService`, using a [`KnnQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1262) within the search request. The method takes a [`SearchRequest`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L18) and returns a [`SearchResponse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L270). The gRPC implementation uses the same underlying k-NN functionality as the HTTP API while providing improved performance through protocol buffer serialization. ## KnnQuery fields -The gRPC k-NN API uses the [`KnnQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1058) message within a [`QueryContainer`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L341) for k-NN searches. The `KnnQuery` message accepts the following fields. +The gRPC k-NN API uses the [`KnnQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1262) message within a [`QueryContainer`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L395) for k-NN searches. The `KnnQuery` message accepts the following fields. | Field | Protobuf type | Description | | :---- | :---- | :---- | @@ -39,11 +39,11 @@ The gRPC k-NN API uses the [`KnnQuery`](https://github.com/opensearch-project/op | `k` | `int32` | The number of nearest neighbors to return as top hits. Optional. | | `min_score` | `float` | The minimum similarity score required for a neighbor to be considered a hit. Optional. | | `max_distance` | `float` | The maximum physical distance in vector space required for a neighbor to be considered a hit. Optional. | -| `filter` | [`QueryContainer`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L341) | Filters for the k-NN search query. See [Filter limitations](#filter-limitations). Optional. | +| `filter` | [`QueryContainer`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L395) | Filters for the k-NN search query. See [Filter limitations](#filter-limitations). Optional. | | `boost` | `float` | A boost value used to increase or decrease relevance scores. Default is 1.0. Optional. | | `underscore_name` | `string` | A query name for query tagging (JSON key: `_name`). Optional. | -| `method_parameters` | [`ObjectMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L76) | Algorithm-specific parameters (for example, `ef_search` or `nprobes`). Optional. | -| `rescore` | [`KnnQueryRescore`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1107) | A rescoring configuration for improved accuracy. Available in versions later than 2.17. Optional. | +| `method_parameters` | [`ObjectMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L111) | Algorithm-specific parameters (for example, `ef_search` or `nprobes`). Optional. | +| `rescore` | [`KnnQueryRescore`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1311) | A rescoring configuration for improved accuracy. Available in versions later than 2.17. Optional. | | `expand_nested_docs` | `bool` | When `true`, retrieves scores for all nested field documents within each parent document. Used with nested queries. Optional. | ## Example request @@ -117,7 +117,7 @@ public class KnnGrpcClient { ## Response fields -k-NN search requests return the same [`SearchResponse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L317) structure as regular search operations. For information about response fields, see [Search (gRPC) response fields]({{site.url}}{{site.baseurl}}/api-reference/grpc-apis/search/#response-fields). +k-NN search requests return the same [`SearchResponse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L270) structure as regular search operations. For information about response fields, see [Search (gRPC) response fields]({{site.url}}{{site.baseurl}}/api-reference/grpc-apis/search/#response-fields). The response includes the standard search metadata (`took`, `timed_out`, and `shards`) and a `hits` array containing the k-NN documents with their similarity scores. diff --git a/_api-reference/grpc-apis/search.md b/_api-reference/grpc-apis/search.md index 7322dd09dd6..94f98483312 100644 --- a/_api-reference/grpc-apis/search.md +++ b/_api-reference/grpc-apis/search.md @@ -21,12 +21,11 @@ To submit gRPC requests, you must have a set of protobufs on the client side. Fo ## gRPC service and method -gRPC Document APIs reside in the [`SearchService`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/services/search_service.proto#L22). +gRPC Document APIs reside in the [`SearchService`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/services/search_service.proto#L22). -You can submit search requests by invoking the [`Search`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/services/search_service.proto#L23) gRPC method within the `SearchService`. The method takes in a [`SearchRequest`](#searchrequest-fields) and returns a [`SearchResponse`](#searchresponse-fields). +You can submit search requests by invoking the [`Search`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/services/search_service.proto#L23) gRPC method within the `SearchService`. The method takes a [`SearchRequest`](#searchrequest-fields) and returns a [`SearchResponse`](#searchresponse-fields). -Currently, only the following basic queries are supported: [`match_all`](#match-all-query), [`term`](#term-query), -[`terms`](#terms-query), and [`match_none`](#match-none-query). Additional query types will be supported in future versions. +See [Supported Queries](#supported-queries) for currently supported search queries. Additional query types will be supported in future versions. {: .note} ## Request fields @@ -35,14 +34,14 @@ The gRPC Search API supports the following request fields. ### SearchRequest fields -The [`SearchRequest`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L18) message accepts the following fields. All fields are optional. +The [`SearchRequest`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L18) message accepts the following fields. All fields are optional. | Field | Protobuf type | Description | | :---- | :---- | :---- | | `index` | `repeated string` | A list of indexes to search. If not provided, defaults to all indexes. | -| `source` | [`SourceConfigParam`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L154) | Controls whether to return the full `_source`, no `_source`, or only specific fields from `_source` in the response. | -| `source_excludes` | `repeated string` | Fields to exclude from `_source`. Ignored if `source` is `false`. | -| `source_includes` | `repeated string` | Fields to include in `_source`. Ignored if `source` is `false`. | +| `x_source` | [`SourceConfigParam`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L199) | Controls whether to return the full `_source`, no `_source`, or only specific fields from `_source` in the response. | +| `x_source_excludes` | `repeated string` | Fields to exclude from `_source`. Ignored if `source` is `false`. | +| `x_source_includes` | `repeated string` | Fields to include in `_source`. Ignored if `source` is `false`. | | `allow_no_indices` | `bool` | Whether to ignore wildcards that match no indexes. Default is `true`. | | `allow_partial_search_results` | `bool` | Whether to return partial results upon an error or timeout. Default is `true`. | | `analyze_wildcard` | `bool` | Whether to analyze wildcard/prefix queries. Default is `false`. | @@ -50,10 +49,10 @@ The [`SearchRequest`](https://github.com/opensearch-project/opensearch-protobufs | `batched_reduce_size` | `int32` | The number of shards to reduce on a node. Default is `512`. | | `cancel_after_time_interval` | `string` | The time after which the request will be canceled. Default is `-1`. | | `ccs_minimize_roundtrips` | `bool` | Whether to minimize round trips between the node and remote clusters. Default is `true`. | -| `default_operator` | [`Operator`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L950) | The default operator for query strings. Valid values are `AND` or `OR`. Default is `OR`. | +| `default_operator` | [`Operator`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1155) | The default operator for query strings. Valid values are `AND` or `OR`. Default is `OR`. | | `df` | `string` | The default field for query strings without field prefixes. | | `docvalue_fields` | `repeated string` | The fields to return as doc values. | -| `expand_wildcards` | `repeated` [`ExpandWildcard`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L56) | Specifies the type of index that wildcard expressions can match. Valid values are `all` (match any index), `open` (match open, non-hidden indexes), `closed` (match closed, non-hidden indexes), `hidden` (match hidden indexes), and `none` (deny wildcard expressions). Default is `open`.| +| `expand_wildcards` | `repeated` [`ExpandWildcard`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L77) | Specifies the type of index that wildcard expressions can match. Valid values are `all` (match any index), `open` (match open, non-hidden indexes), `closed` (match closed, non-hidden indexes), `hidden` (match hidden indexes), and `none` (deny wildcard expressions). Default is `open`.| | `explain` | `bool` | Whether to return document score computation details. Default is `false`. | | `from` | `int32` | The starting index for paginated results. Default is `0`. | | `ignore_throttled` | `bool` | Whether to ignore frozen indexes when resolving aliases. Default is `true`. | @@ -70,24 +69,24 @@ The [`SearchRequest`](https://github.com/opensearch-project/opensearch-protobufs | `routing` | `repeated string` | The routing values used to direct requests to specific shards. | | `scroll` | `string` | The amount of time to keep the search context alive for scrolling. | | `search_pipeline` | `string` | The name of the search pipeline to use. | -| `search_type` | [`SearchType`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L109) | The method for calculating relevance scores. Valid values are `QUERY_THEN_FETCH` and `DFS_QUERY_THEN_FETCH`. Default is `QUERY_THEN_FETCH`. | +| `search_type` | [`SearchType`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L86) | The method for calculating relevance scores. Valid values are `QUERY_THEN_FETCH` and `DFS_QUERY_THEN_FETCH`. Default is `QUERY_THEN_FETCH`. | | `seq_no_primary_term` | `bool` | Whether to return the sequence number and primary term. | | `size` | `int32` | The number of results to return. | -| `sort` | `repeated` [`SortOrder`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L122) | The fields and directions by which to sort the results. Each `SortOrder` contains a `field` and optional `direction` (ASC/DESC). | | `stats` | `repeated string` | The tags to associate with the request for logging. | | `stored_fields` | `repeated string` | A list of stored fields to include in the response. | | `suggest_field` | `string` | The field on which to base suggestions. | -| `suggest_mode` | [`SuggestMode`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L145) | The suggestion mode (for example, `always`, `missing`, `popular`). | +| `suggest_mode` | [`SuggestMode`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L92) | The suggestion mode (for example, `always`, `missing`, `popular`). | | `suggest_size` | `int32` | The number of suggestions to return. | | `suggest_text` | `string` | The input text for generating suggestions. | | `terminate_after` | `int32` | The maximum number of matching documents (hits) to process before early termination. Default is `0`. | | `timeout` | `string` | The maximum amount of time to wait for query execution. Default is `1m`. | | `track_scores` | `bool` | Whether to return document scores. Default is `false`. | -| `track_total_hits` | [`TrackHits`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L309) | Whether to include total hits metadata. | +| `track_total_hits` | [`TrackHits`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L261) | Whether to include total hits metadata. | | `typed_keys` | `bool` | Whether to include type information in aggregation and suggestion keys. Default is `true`. | | `verbose_pipeline` | `bool` | Whether to enable verbose mode for the search pipeline. | | `version` | `bool` | Whether to return the document version in the response. | -| `request_body` | [`SearchRequestBody`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L176) | The main search request payload, including the query and filters. | +| `request_body` | [`SearchRequestBody`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L128) | The main search request payload, including the query and filters. | +| `global_params` | [`GlobalParams`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L14) | Global parameters for the request. Optional. | ### SearchRequestBody fields @@ -95,29 +94,29 @@ The `SearchRequestBody` message accepts the following fields. All fields are opt | Field | Protobuf type | Description | | :---- | :---- | :---- | -| `collapse` | [`FieldCollapse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L906) | Groups the results by a field. Returns only the top document per group. | +| `collapse` | [`FieldCollapse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1114) | Groups the results by a field. Returns only the top document per group. | | `explain` | `bool` | Returns scoring explanations for matched documents. | -| `ext` | [`ObjectMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L76) | Plugin-specific metadata, for example, for extensions like RAG. | +| `ext` | [`ObjectMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L111) | Plugin-specific metadata, for example, for extensions like RAG. | | `from` | `int32` | The starting index for paginated results. Default is `0`. | -| `highlight` | [`Highlight`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L516) | Highlights matched terms in the result snippets. | -| `track_total_hits` | [`TrackHits`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L309) | Whether to return the total hit count. | -| `indices_boost` | `repeated` [`NumberMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L747) | Per-index boost multipliers in the `: ` format. | -| `docvalue_fields` | `repeated` [`FieldAndFormat`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L893) | The fields returned using doc values. Optionally, this field can be formatted for readability. | +| `highlight` | [`Highlight`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L781) | Highlights matched terms in the result snippets. | +| `track_total_hits` | [`TrackHits`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L261) | Whether to return the total hit count. | +| `indices_boost` | `repeated` [`FloatMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L124) | Per-index boost multipliers in the `: ` format. | +| `docvalue_fields` | `repeated` [`FieldAndFormat`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1104) | The fields returned using doc values. Optionally, this field can be formatted for readability. | | `min_score` | `float` | The minimum score required in order for a document to be included in the results. | -| `post_filter` | [`QueryContainer`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L341) | Filters hits after aggregations are applied. | +| `post_filter` | [`QueryContainer`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L394) | Filters hits after aggregations are applied. | | `profile` | `bool` | Enables profiling to analyze query performance. | | `search_pipeline` | `string` | The name of the search pipeline to apply. | | `verbose_pipeline` | `bool` | Enables verbose logging in the search pipeline. | -| `query` | [`QueryContainer`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L341) | The query domain-specific language (DSL) for the search. | -| `rescore` | `repeated` [`Rescore`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L631) | Reranks the top N hits to improve precision. | -| `script_fields` | `map` | Custom fields whose values are computed by scripts. | -| `search_after` | `repeated` [`FieldValue`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1971) | Cursor-based pagination using values from the previous page. | +| `query` | [`QueryContainer`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L394) | The query domain-specific language (DSL) for the search. | +| `rescore` | `repeated` [`Rescore`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L561) | Reranks the top N hits to improve precision. | +| `script_fields` | `map` | Custom fields whose values are computed by scripts. | +| `search_after` | `repeated` [`FieldValue`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2168) | Cursor-based pagination using values from the previous page. | | `size` | `int32` | The number of results to return. Default is `10`. | -| `slice` | [`SlicedScroll`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L641) | Splits scroll context into slices for parallel processing. | -| `sort` | `repeated` [`SortCombinations`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L745) | The sorting rules (for example, by field, score, or custom order). | -| `source` | [`SourceConfig`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L176) | Controls whether to return the full `_source`, no `_source`, or only specific fields from `_source` in the response. | -| `fields` | `repeated` [`FieldAndFormat`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L893) | Additional fields to return, with formatting options. | -| `suggest` | [`Suggester`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L654) | Suggestion queries for autocomplete or corrections. | +| `slice` | [`SlicedScroll`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L571) | Splits scroll context into slices for parallel processing. | +| `sort` | `repeated` [`SortCombinations`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L740) | The sorting rules (for example, by field, score, or custom order). | +| `x_source` | [`SourceConfig`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L230) | Controls whether to return the full `_source`, no `_source`, or only specific fields from `_source` in the response. | +| `fields` | `repeated` [`FieldAndFormat`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1104) | Additional fields to return, with formatting options. | +| `suggest` | [`Suggester`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L584) | Suggestion queries for autocomplete or corrections. | | `terminate_after` | `int32` | The maximum number of matching documents (hits) to process before early termination. Default is `0`. | | `timeout` | `string` | The maximum amount of time to wait for query execution. | | `track_scores` | `bool` | Whether to return document scores in the results. | @@ -125,21 +124,21 @@ The `SearchRequestBody` message accepts the following fields. All fields are opt | `version` | `bool` | Whether to include the document version in the response. | | `seq_no_primary_term` | `bool` | Whether to include the sequence number and primary term for each hit. | | `stored_fields` | `repeated string` | The stored fields to return (excludes `_source` unless re-enabled). | -| `pit` | [`PointInTimeReference`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L752) | The Point in Time reference used to search a fixed snapshot. | +| `pit` | [`PointInTimeReference`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L684) | The Point in Time reference used to search a fixed snapshot. | | `stats` | `repeated string` | The tagging or logging fields to associate with the request. | -| `derived` | `map` | Dynamically computed fields returned in the response. | +| `derived` | `map` | Dynamically computed fields returned in the response. | ### DerivedField fields -The [`DerivedField`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L292) message is used for dynamically computed fields that are calculated during search execution. It accepts the following fields. +The [`DerivedField`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L244) message is used for dynamically computed fields that are calculated during search execution. It accepts the following fields. | Field | Protobuf type | Description | | :---- | :---- | :---- | | `name` | `string` | The name of the derived field. Required. | | `type` | `string` | The data type of the derived field. Required. | -| `script` | [`Script`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L27) | The script that computes the field value. Required. | +| `script` | [`Script`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L36) | The script that computes the field value. Required. | | `prefilter_field` | `string` | The field to use for prefiltering to optimize script execution. Optional. | -| `properties` | `map` | Additional properties for the derived field. Optional. | +| `properties` | `map` | Additional properties for the derived field. Optional. | | `ignore_malformed` | `bool` | Whether to ignore malformed values during field computation. Optional. | | `format` | `string` | The format to apply to the field value (for example, date formatting). Optional. | @@ -154,38 +153,47 @@ Note that some query types are currently unsupported. See [Supported queries](#s | Field | Protobuf type | Description | | :---- | :---- | :---- | -| `bool` | [`BoolQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1324) | A Boolean query that combines multiple clauses using `AND`/`OR`/`NOT` logic. Must be the only field set. | -| `boosting` | [`BoostingQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1360) | Boosts the results matching a positive query and demotes the results matching a negative query. Must be the only field set. | -| `constant_score` | [`ConstantScoreQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1377) | Wraps a filter and assigns a constant relevance score to all matching documents. Must be the only field set. | -| `dis_max` | [`DisMaxQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1389) | Returns documents matching any clause. Uses the highest score if multiple clauses match. Must be the only field set. | -| `function_score` | [`FunctionScoreQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1405) | Adjusts the scores of results using custom functions. Must be the only field set. | -| `exists` | [`ExistsQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1022) | Matches documents that contain a specific field. Must be the only field set. | -| `fuzzy` | [`FuzzyQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L2000) | Matches terms similar to the search term (fuzzy matching). Must be the only field set. | -| `ids` | [`IdsQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L2083) | Matches documents by `_id` values. Must be the only field set. | -| `prefix` | [`PrefixQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1592) | Matches terms with a specific prefix. Must be the only field set. | -| `range` | [`RangeQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1860) | Matches terms within a specified range. Must be the only field set. | -| `regexp` | [`RegexpQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1870) | Matches terms using regular expressions. Must be the only field set. | -| `term` | [`TermQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1732) | Matches exact terms (no analysis). Must be the only field set. | -| `terms` | [`TermsQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L462) | Matches any document containing one or more specified terms in a field. Must be the only field set. | -| `terms_set` | [`TermsSetQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1708) | Matches documents containing a minimum number of exact terms in a field. Must be the only field set. | -| `wildcard` | [`WildcardQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1083) | Matches terms using a wildcard pattern. Must be the only field set. | -| `match` | [`MatchQueryTypeless`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1198) | Full-text match on text or exact-value fields. Must be the only field set. | -| `match_bool_prefix` | [`MatchBoolPrefixQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L2150) | Matches full words and prefixes in a Boolean-style query. Must be the only field set. | -| `match_phrase` | [`MatchPhraseQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L2279) | Matches an exact phrase in order. Must be the only field set. | -| `match_phrase_prefix` | [`MatchPhrasePrefixQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L2246) | Matches a phrase in which the last term is treated as a prefix. Must be the only field set. | -| `multi_match` | [`MultiMatchQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L2309) | Searches multiple fields using a single query string. Must be the only field set. | -| `query_string` | [`QueryStringQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1753) | Parses advanced queries written as a single string. Must be the only field set. | -| `simple_query_string` | [`SimpleQueryStringQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1039) | A less strict syntax alternative to `query_string`. Ignores invalid syntax. Must be the only field set. | -| `intervals` | [`IntervalsQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1495) | Matches terms based on position/proximity. Must be the only field set. | -| `knn` | [`KnnQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1126) | A k-NN query across vector fields. Must be the only field set. | -| `match_all` | [`MatchAllQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L2142) | Matches all documents in the index. Must be the only field set. | -| `match_none` | [`MatchNoneQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L2228) | Matches no documents. Must be the only field set. | -| `script_score` | [`ScriptScoreQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1005) | Applies custom scoring using scripts. Must be the only field set. | -| `nested` | [`NestedQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L499) | Wraps a query targeting nested fields. Must be the only field set. | +| `bool` | [`BoolQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1416) | A Boolean query that combines multiple clauses using `AND`/`OR`/`NOT` logic. Must be the only field set. | +| `boosting` | [`BoostingQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1452) | Boosts the results matching a positive query and demotes the results matching a negative query. Must be the only field set. | +| `constant_score` | [`ConstantScoreQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1469) | Wraps a filter and assigns a constant relevance score to all matching documents. Must be the only field set. | +| `dis_max` | [`DisMaxQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1481) | Returns documents matching any clause. Uses the highest score if multiple clauses match. Must be the only field set. | +| `function_score` | [`FunctionScoreQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1497) | Adjusts the scores of results using custom functions. Must be the only field set. | +| `exists` | [`ExistsQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1144) | Matches documents that contain a specific field. Must be the only field set. | +| `fuzzy` | [`FuzzyQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2123) | Matches terms similar to the search term (fuzzy matching). Must be the only field set. | +| `ids` | [`IdsQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2178) | Matches documents by `_id` values. Must be the only field set. | +| `prefix` | [`PrefixQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1740) | Matches terms with a specific prefix. Must be the only field set. | +| `range` | [`RangeQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1981) | Matches terms within a specified range. Must be the only field set. | +| `regexp` | [`RegexpQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2097) | Matches terms using regular expressions. Must be the only field set. | +| `term` | [`TermQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1857) | Matches exact terms (no analysis). Must be the only field set. | +| `terms` | [`TermsQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L605) | Matches any document containing one or more specified terms in a field. Must be the only field set. | +| `terms_set` | [`TermsSetQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1836) | Matches documents containing a minimum number of exact terms in a field. Must be the only field set. | +| `wildcard` | [`WildcardQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1205) | Matches terms using a wildcard pattern. Must be the only field set. | +| `match` | [`MatchQueryTypeless`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1320) | Full-text match on text or exact-value fields. Must be the only field set. | +| `match_bool_prefix` | [`MatchBoolPrefixQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2245) | Matches full words and prefixes in a Boolean-style query. Must be the only field set. | +| `match_phrase` | [`MatchPhraseQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2353) | Matches an exact phrase in order. Must be the only field set. | +| `match_phrase_prefix` | [`MatchPhrasePrefixQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2320) | Matches a phrase in which the last term is treated as a prefix. Must be the only field set. | +| `multi_match` | [`MultiMatchQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2370) | Searches multiple fields using a single query string. Must be the only field set. | +| `query_string` | [`QueryStringQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1878) | Parses advanced queries written as a single string. Must be the only field set. | +| `simple_query_string` | [`SimpleQueryStringQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1161) | A less strict syntax alternative to `query_string`. Ignores invalid syntax. Must be the only field set. | +| `intervals` | [`IntervalsQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1582) | Matches terms based on position/proximity. Must be the only field set. | +| `knn` | [`KnnQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1263) | A k-NN query across vector fields. Must be the only field set. | +| `match_all` | [`MatchAllQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2237) | Matches all documents in the index. Must be the only field set. | +| `match_none` | [`MatchNoneQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2302) | Matches no documents. Must be the only field set. | +| `script_score` | [`ScriptScoreQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L1127) | Applies custom scoring using scripts. Must be the only field set. | +| `nested` | [`NestedQuery`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L619) | Wraps a query targeting nested fields. Must be the only field set. | ## Supported queries -The gRPC Search API supports the following queries. +The gRPC Search API supports the following query types: +* Term-level: `exists`, `ids`, `prefix`, `range`, `regexp`, `terms_set`, `wildcard` +* Full-text: `match`, `match_bool_prefix`, `match_phrase`, `multi_match` +* Compound queries: `bool` +* Geographic: `geo_bounding_box`, `geo_distance` +* Joining queries: `nested` +* Specialized queries: `script` + +For more information about these query types, see [Query DSL]({{site.url}}{{site.baseurl}}/query-dsl/). + All of the following examples show valid request payloads that can be sent to the `SearchService/Search` gRPC method. @@ -293,23 +301,9 @@ A `match_none` query matches none of the documents: {% include copy.html %} ## Response fields - -The gRPC Search API provides the following response fields. - ### SearchResponse fields -The following table lists the supported fields for the [`SearchResponse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L317) message. - -| Field | Protobuf type | Description | -| :---- | :---- | :---- | -| `response` | `oneof` | Contains one of the following response types: `response_body` (success), `error_4xx_response_body` (4xx error), or `error_5xx_response` (5xx error). | -| `response_body` | [`ResponseBody`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L329) | The actual payload of the search response (success case). | -| `error_4xx_response_body` | [`Error4xxResponseBody`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L309) | The 4xx error response body. | -| `error_5xx_response` | [`Error5xxResponse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L317) | The 5xx error response. | - -### ResponseBody fields - -The `ResponseBody` contains the following fields. +The following table lists the supported fields for the [`SearchResponse`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L270) message. The source documents are returned as bytes. Use Base64 decoding to read the `_source` field in the gRPC response. {: .note} @@ -318,16 +312,16 @@ The source documents are returned as bytes. Use Base64 decoding to read the `_so | :---- | :---- | :---- | | `took` | `int64` | The amount of time taken to process the search request, in milliseconds. | | `timed_out` | `bool` | Whether the search timed out. | -| `shards` | [`ShardStatistics`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L268) | The shard-level success/failure/total metadata. | -| `phase_took` | [`PhaseTook`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L394) | The phase-level `took` time values in the response. | -| `hits` | [`HitsMetadata`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L411) | The main document results and metadata. | -| `profile` | [`Profile`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L594) | Profiling data for query execution (debugging/performance insights). | -| `fields` | [`ObjectMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L76) | The top-level key-value field structure from the response (if any). | +| `x_shards` | [`ShardStatistics`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L321) | The shard-level success/failure/total metadata. | +| `phase_took` | [`PhaseTook`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L332) | The phase-level `took` time values in the response. | +| `hits` | [`HitsMetadata`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L363) | The main document results and metadata. | +| `processor_results` | `repeated` [`ProcessorExecutionDetail`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L315) | Processor execution details. | +| `x_clusters` | [`ClusterStatistics`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L467) | Information about the search on each cluster when searching remote clusters. | +| `fields` | [`ObjectMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L111) | Retrieved specific fields in the search response. | | `num_reduce_phases` | `int32` | The number of times that the coordinating node aggregates results from batches of shard responses. | -| `clusters` | [`ClusterStatistics`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L532) | Information about the search on each cluster when searching remote clusters. | -| `max_score` | `float` | The highest returned document score. Optional. | +| `profile` | [`Profile`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L529) | Profiling data for query execution (debugging/performance insights). | | `pit_id` | `string` | The Point in Time ID. | -| `scroll_id` | `string` | The identifier for the search and its search context. | +| `x_scroll_id` | `string` | The identifier for the search and its search context. | | `terminated_early` | `bool` | Whether the query was terminated early. | ### HitsMetadata fields @@ -336,28 +330,28 @@ The `HitsMetadata` object contains information about the search results, includi | Field | Protobuf type | Description | | :---- | :---- | :---- | -| `total` | [`HitsMetadataTotal`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L412) | Metadata about the total number of matching documents (value \+ relation). | -| `max_score` | [`HitsMetadataMaxScore`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L424) | The highest relevance score of the returned hits (may be `null`). | -| `hits` | `repeated` [`Hit`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L460) | The actual list of matched documents. Each hit includes core fields like `index`, `id`, `score`, and `source`, along with additional optional fields. | +| `total` | [`HitsMetadataTotal`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L349) | Metadata about the total number of matching documents (value \+ relation). | +| `max_score` | [`HitsMetadataMaxScore`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L356) | The highest relevance score of the returned hits (may be `null`). | +| `hits` | `repeated` [`HitsMetadataHitsInner`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L403) | The actual list of matched documents. Each hit includes core fields like `index`, `id`, `score`, and `source`, along with additional optional fields. | -### Hit fields +### HitsMetadataHitsInner fields -Each `Hit` represents a single document matched by the query and contains the following fields. +Each `HitsMetadataHitsInner` represents a single document matched by the query and contains the following fields. | Field | Protobuf type | Description | | :---- | :---- | :---- | | `type` | `string` | The document type. | | `index` | `string` | The name of the index containing the returned document. | | `id` | `string` | The unique ID for the document within the index. | -| `score` | [`HitUnderscoreScore`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L455) | The relevance score of the hit. | -| `explanation` | [`Explanation`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L944) | A text explanation of how the `_score` was calculated. | -| `fields` | [`ObjectMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L76) | The document field values. | -| `highlight` | `map` | The highlighted fields and fragments per hit. | -| `inner_hits` | `map` | The matching nested documents from a different scope that contributed to the overall query result. | +| `score` | [`HitXScore`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L396) | The relevance score of the hit. | +| `explanation` | [`Explanation`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L885) | A text explanation of how the `_score` was calculated. | +| `fields` | [`ObjectMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L111) | The document field values. | +| `highlight` | `map` | The highlighted fields and fragments per hit. | +| `inner_hits` | `map` | The matching nested documents from a different scope that contributed to the overall query result. | | `matched_queries` | `repeated string` | A list of query names matching the document. | -| `nested` | [`NestedIdentity`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/search.proto#L962) | The path to the inner nested object from which the hit originated. | +| `nested` | [`NestedIdentity`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/search.proto#L896) | The path to the inner nested object from which the hit originated. | | `ignored` | `repeated string` | A list of ignored fields. | -| `ignored_field_values` | `map` | Raw, unprocessed values from the document's original JSON. | +| `ignored_field_values` | `map` | Raw, unprocessed values from the document's original JSON. | | `shard` | `string` | The shard ID from which the hit was retrieved. | | `node` | `string` | The node ID from which the hit was retrieved. | | `routing` | `string` | The routing value used for custom shard routing. | @@ -365,8 +359,8 @@ Each `Hit` represents a single document matched by the query and contains the fo | `seq_no` | `int64` | The sequence number (used for indexing history and versioning). | | `primary_term` | `int64` | The primary term number (used for optimistic concurrency control). | | `version` | `int64` | The document version number. | -| `sort` | `repeated` [`FieldValue`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L1971) | The sort values used for result sorting. | -| `meta_fields` | [`ObjectMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.6.0/protos/schemas/common.proto#L76) | The metadata values for the document. | +| `sort` | `repeated` [`FieldValue`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L2168) | The sort values used for result sorting. | +| `meta_fields` | [`ObjectMap`](https://github.com/opensearch-project/opensearch-protobufs/blob/0.19.0/protos/schemas/common.proto#L111) | The metadata values for the document. | `source` is Base64 encoded and must be decoded to obtain the JSON document. {: .note}