Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update knn index default to always do approximate search in test #2332

Merged

Conversation

VijayanB
Copy link
Member

@VijayanB VijayanB commented Dec 12, 2024

Description

Update knn index default to always do approximate search in test cases

Related Issues

Resolves #2329

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@VijayanB VijayanB force-pushed the update-default-index-settings-test branch 2 times, most recently from cf1deca to 5701675 Compare December 12, 2024 22:42
@VijayanB VijayanB marked this pull request as ready for review December 12, 2024 22:42
navneet1v
navneet1v previously approved these changes Dec 13, 2024
@heemin32
Copy link
Collaborator

There might be test cases which use custom setting instead of default setting. We need to update those cases as well.

@VijayanB
Copy link
Member Author

There might be test cases which use custom setting instead of default setting. We need to update those cases as well.

like what? Can you share the test that you are referring to?

Signed-off-by: Vijayan Balasubramanian <[email protected]>
@VijayanB VijayanB force-pushed the update-default-index-settings-test branch from 850ddba to 691810a Compare December 16, 2024 19:45
Signed-off-by: Vijayan Balasubramanian <[email protected]>
Comment on lines +109 to +112
Settings settings = Settings.builder()
.put("index.knn", true)
.put(KNNSettings.KNN_ALGO_PARAM_EF_SEARCH, 512)
.put(KNNSettings.INDEX_KNN_ADVANCED_APPROXIMATE_THRESHOLD, 0)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for cases like this I think we should just move the code to use getDefaultIndexSettings so that in future we don't need to make the changes.

Copy link
Collaborator

@navneet1v navneet1v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good to me. Just 1 minor comment

@VijayanB
Copy link
Member Author

There might be test cases which use custom setting instead of default setting. We need to update those cases as well.
@heemin32 I realized what you mean by custom mapping. I searched for index.knn is set true and updated those index setting with approximate threshold parameter

@VijayanB VijayanB merged commit dc369e6 into opensearch-project:main Dec 18, 2024
31 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-2332-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 dc369e6d73beb068c4203db9f9375a6194d824f8
# Push it to GitHub
git push --set-upstream origin backport/backport-2332-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-2332-to-2.x.

owenhalpert pushed a commit to owenhalpert/k-NN that referenced this pull request Dec 19, 2024
…nsearch-project#2332)

* Update knn index default to always do approximate search

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Update custom mappings

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* update bwc

Signed-off-by: Vijayan Balasubramanian <[email protected]>

---------

Signed-off-by: Vijayan Balasubramanian <[email protected]>
VijayanB added a commit to VijayanB/k-NN-2 that referenced this pull request Dec 21, 2024
…nsearch-project#2332)

* Update knn index default to always do approximate search

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* Update custom mappings

Signed-off-by: Vijayan Balasubramanian <[email protected]>

* update bwc

Signed-off-by: Vijayan Balasubramanian <[email protected]>

---------

Signed-off-by: Vijayan Balasubramanian <[email protected]>
VijayanB added a commit that referenced this pull request Dec 23, 2024
…) (#2350)

* Update knn index default to always do approximate search



* Update custom mappings



* update bwc



---------

Signed-off-by: Vijayan Balasubramanian <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Integ test and futuer BWC test will be running only with exact search
3 participants