[FEATURE] Add CPU and jvm heap memory circuit breakers for Neural Search and Text Embedding processor

[joshpalis]

Is your feature request related to a problem?

When using remote models for text embedding during the execution of a neural query or the ingestion of documents, the request is stored in memory while the natural language query text is processed into vector embeddings. Depending on the latency of the response, it is possible that multiple requests can sit in memory and become detrimental to other tasks currently being handled by OpenSearch.

What solution would you like?

A circuit breaker for both memory and cpu utilization should be added to reject additional requests if the thresholds are breached

What alternatives have you considered?

n/a

Do you have any additional context?

Hybrid Search currently has circuit breakers integrated with the normalization processor. Since neural search does not require any use of search pipelines, we cannot follow the same pattern.

[heemin32]

Wouldn't it be handled by OpenSearch core level already? I think the queue for query request is 1,000 and more than that request will be rejected anyway?