Search request response format should not be coupled with query format

[jainankitk]

Is your feature request related to a problem? Please describe

While working on query rewriting, I noticed that the query response format in opensearch is tightly coupled with query format. It should just be data dependent allowing translation into more efficient queries without impacting the response format. For example:

The response for

"_source": false,
  "stored_fields": "_none_",
  "docvalue_fields": [
    {
      "field": "_id"
    }
  ],

is

{"took":8,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":null,"hits":[{"_index":"es-test","_type":"_doc","_score":null,"fields":{"_id":["123456789"]}}]}}

whereas for below query:

"_source": false,
<no stored fields or docvalue_fields specification>

it is:

{"took":8,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":null,"hits":[{"_index":"es-test","_type":"_doc","_score":null,"_id":"123456789"}]}}

While the difference in above seems minimal, this can easily break the end client.

Describe the solution you'd like

We should reevaluate all the query response types and ensure they are data oriented instead of being coupled with query format.

Related component

Search

Describe alternatives you've considered

No response

Additional context

No response

[msfroh]

@jainankitk -- Do you plan to work on this yourself? Can we assign it to you?

[jainankitk]

@msfroh - Yes I should own it, so you can assign to me. Just wanted to get feedback on the idea, and which version (probably major version) should we target