Introduce a new index setting/property for toggling a remote store index as "writable warm”

[andrross]

Is your feature request related to a problem? Please describe

The goal of this task to is come up with options for the API for enabling a “writeable remote index” as described by #7804 and #8891 and implement the API gated by a feature flag. The intent is to start building a minimal framework on which we can incrementally add the pieces described by #9987 and others and iterate on the feature.

Describe the solution you'd like

The simplest solution might be a new index setting. However, we'll want to think in detail about state transitions. Namely, toggling an index from "warm" to "hot" may require downloading a considerable amount of data similar to an index recovery option. A user will want to know that an index is in this transitioning state and be able to monitor the operation. We'll want to consider these details.

Related component

Search:Remote Search

[dblock]

However, we'll want to think in detail about state transitions. Namely, toggling an index from "warm" to "hot" may require downloading a considerable amount of data similar to an index recovery option.

A flexible approach could be to expose more options instead of shortcuts like "hot" or "warm".

storage: {
   init: {
       method: "bulk"
   },
   max_heap: "30%"
}