[PROPOSAL] Integrate LTR With OpenSearch CI

[macohen]

What/Why

What are you proposing?

We want to be able to build against the OpenSearch CI system the same way as any other plugin in the opensearch-project repo. The first step is just to build the plugin with each release for self-install purposes. Next step would be to integrate with the open source release distribution if we think it's the right thing to do.

What users have asked for this feature?

We have no open source Learning to Rank repository for OpenSearch that is integrated. @gsingers asked for help in maintaining the build on this plugin and we took that on.

Are there any security considerations?

Yes, but these considerations are not 100% clear yet.

Are there any breaking changes to the API

No

Why should it be built? Any reason not to?

An OpenSearch supported LTR plugin will help us work with the community moving forward with additional features and enhancements. We are looking at ways to use LTR for hybrid (Vector + Lexical) search, for example.

What will it take to execute?

This list is to come, but most work can be found in https://github.com/opensearch-project/opensearch-plugins md files/

[macohen]

Putting in this checklist to track the process of upgrading and branching

• branching main to 2.5
• confirm successful build of 2.5
• branch main to 2.6
• confirm successful build of 2.6
• branch main to 2.8 - build against 2.8.1
• confirm successful build of 2.8.1
• branch main to 2.9
• confirm successful build of 2.9
• 2.9 becomes the main branch
• create a 2.x branch versioned for 2.9.1; main is 3.0.0
• remove direct lucene dependency if possible (should be transitive from OpenSearch version)
• ltrVersion - should it reflect the current OpenSearch version or the latest LTR version (e.g. ltr 2.1.0 is compatible with OpenSearch 2.5)

[jzonthemtn]

Related, and potentially a duplicate of #32.