New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

use terms in set for concrete IPs keep disjunctions over ranges. fix #16200 #16202

Closed

mkhludnev wants to merge 4 commits into opensearch-project:main from mkhludnev:16200-terms-ip

Contributor

mkhludnev commented Oct 6, 2024

Description

Combines a many concrete IPs into set when querying IP field

Related Issues

Resolves #16200

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions bot added enhancement Search:Query Capabilities labels

Contributor Author

mkhludnev commented Oct 6, 2024

Please confirm my understanding, then I come up with thorough testing.
Presumably would be great to optimize union of ranges with specialized points query.

Contributor

github-actions bot commented Oct 6, 2024

❌ Gradle check result for e782068: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Oct 10, 2024

❌ Gradle check result for 0a1c3d0: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Oct 17, 2024

❌ Gradle check result for bebebac: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Oct 17, 2024

❌ Gradle check result for fbbb0bc: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          use terms in set for concrete IPs keep disjunctions over ranges.

da89a31

Signed-off-by: mikhail-khludnev <[email protected]>

mkhludnev force-pushed the 16200-terms-ip branch from 0586f06 to da89a31 Compare

October 17, 2024 22:05

Contributor

github-actions bot commented Oct 17, 2024

❌ Gradle check result for da89a31: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          forbidden api

33b8494

Signed-off-by: mikhail-khludnev <[email protected]>

Contributor

github-actions bot commented Oct 18, 2024

❌ Gradle check result for 33b8494: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          spotless

3c15413

Signed-off-by: mikhail-khludnev <[email protected]>

Contributor

github-actions bot commented Oct 18, 2024

❌ Gradle check result for 3c15413: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          mv Tests

853a344

Signed-off-by: mikhail-khludnev <[email protected]>

Contributor

github-actions bot commented Oct 18, 2024

❌ Gradle check result for 853a344: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor Author

mkhludnev commented Oct 18, 2024

»  java.lang.IllegalArgumentException: Could not load codec 'Lucene912'. Did you forget to add lucene-backward-codecs.jar?
»  Likely root cause: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene912' does not exist.  You need to add the corresponding JAR file supporting this SPI to your classpath.  The current classpath supports the following names: [Lucene80, Lucene84, Lucene86, Lucene87, Lucene70, Lucene90, Lucene91, Lucene92, Lucene94, Lucene95, Lucene99, Composite99Codec]
»  	at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:113)
»  	at org.apache.lucene.codecs.Codec.forName(Codec.java:118)

Task :qa:mixed-cluster:v2.18.0#mixedClusterTest FAILED

https://build.ci.opensearch.org/job/gradle-check/49705/consoleFull

I'm not sure what I can do with it.

mkhludnev commented

View reviewed changes

server/src/test/java/org/opensearch/search/SearchIpFieldTermsTests.java

+              import static org.opensearch.action.support.WriteRequest.RefreshPolicy.IMMEDIATE;
+              import static org.hamcrest.Matchers.equalTo;
+              public class SearchIpFieldTermsTests extends OpenSearchSingleNodeTestCase {

Contributor Author

mkhludnev Oct 18, 2024

It might be too heavy testing approach for the current change. However, such depth is necessary for the second stage (when ranges will be combined at PointsVisitor level).
There might be just a IpFieldTypeTest asserting resulting BooleanQuery has set of IP points. WDYT?

server/src/test/java/org/opensearch/search/SearchIpFieldTermsTests.java

+                          final String ip;
+                          final int prefix;
+                          if (IPv4_ONLY) {
+                              ip = generateRandomIPv4();

Contributor Author

mkhludnev Oct 18, 2024

it's just IP4 test, no IP6. Is IP6 necessary, is it worth to combine 6th and 4th in the single test?

mkhludnev marked this pull request as ready for review

October 18, 2024 14:35

mkhludnev requested review from anasalkouz, andrross, ashking94, Bukhtawar, CEHENKLE, dblock, dbwiddis, gbbafna, jed326, kotwanikunal and mch2 as code owners

October 18, 2024 14:35

mkhludnev requested review from msfroh, nknize, owaiskazi19, reta, Rishikesh1159, sachinpkale, saratvemulapalli, shwetathareja, sohami and VachaShah as code owners

October 18, 2024 14:35

mkhludnev mentioned this pull request

Support more than 1024 IP/masks with indexed field #16391

Merged

3 tasks

mkhludnev mentioned this pull request

[Feature Request] relax max Clauses Count limitation of termS query over IP field #16200

Closed

Contributor Author

mkhludnev commented Nov 16, 2024

closing in favor of #16391

mkhludnev closed this

mkhludnev deleted the 16200-terms-ip branch

November 28, 2024 06:34

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

anasalkouz Awaiting requested review from anasalkouz anasalkouz is a code owner

andrross Awaiting requested review from andrross andrross is a code owner

ashking94 Awaiting requested review from ashking94 ashking94 is a code owner

Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

CEHENKLE Awaiting requested review from CEHENKLE CEHENKLE is a code owner

dblock Awaiting requested review from dblock dblock is a code owner

dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

jed326 Awaiting requested review from jed326 jed326 is a code owner

kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

mch2 Awaiting requested review from mch2 mch2 is a code owner

msfroh Awaiting requested review from msfroh msfroh is a code owner

nknize Awaiting requested review from nknize nknize is a code owner

owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

reta Awaiting requested review from reta reta is a code owner

Rishikesh1159 Awaiting requested review from Rishikesh1159 Rishikesh1159 is a code owner

sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

shwetathareja Awaiting requested review from shwetathareja shwetathareja is a code owner

sohami Awaiting requested review from sohami sohami is a code owner

VachaShah Awaiting requested review from VachaShah VachaShah is a code owner

Labels

enhancement Search:Query Capabilities