Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] CoreWithSecurityClientYamlTestSuiteIT test {yaml=cluster.stats/10_basic/Sparse vector stats} failing #121246

Closed
elasticsearchmachine opened this issue Jan 29, 2025 · 2 comments · Fixed by #121541
Closed

[CI] CoreWithSecurityClientYamlTestSuiteIT test {yaml=cluster.stats/10_basic/Sparse vector stats} failing #121246

elasticsearchmachine opened this issue Jan 29, 2025 · 2 comments · Fixed by #121541
Assignees
@slobodanadamovic
Labels
:Data Management/Data streams Data streams and their lifecycles low-risk An open issue or test failure that is a low risk to future releases Team:Data Management Meta label for data/management team >test-failure Triaged test failures from CI

Comments

@elasticsearchmachine
Copy link
Collaborator

Build Scans:

Reproduction Line:

./gradlew ":x-pack:qa:core-rest-tests-with-security:yamlRestTest" --tests "org.elasticsearch.xpack.security.CoreWithSecurityClientYamlTestSuiteIT" -Dtests.method="test {yaml=cluster.stats/10_basic/Sparse vector stats}" -Dtests.seed=88AFEF83461AC20 -Dtests.locale=lo-LA -Dtests.timezone=Indian/Cocos -Druntime.java=23

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: Failure at [cluster.stats/10_basic:453]: 
Expected: <5>
     but: was <34>

Issue Reasons:

  • [main] 4 failures in test test {yaml=cluster.stats/10_basic/Sparse vector stats} (0.6% fail rate in 677 executions)
  • [main] 3 failures in step part-4 (0.9% fail rate in 337 executions)
  • [main] 3 failures in pipeline elasticsearch-pull-request (0.9% fail rate in 328 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.
@elasticsearchmachine elasticsearchmachine added :Data Management/Data streams Data streams and their lifecycles >test-failure Triaged test failures from CI labels Jan 29, 2025
elasticsearchmachine added a commit that referenced this issue Jan 29, 2025
@elasticsearchmachine
Mute org.elasticsearch.xpack.security.CoreWithSecurityClientYamlTestS…
d88ddca 
…uiteIT test {yaml=cluster.stats/10_basic/Sparse vector stats} #121246
@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch main

Mute Reasons:

  • [main] 4 failures in test test {yaml=cluster.stats/10_basic/Sparse vector stats} (0.6% fail rate in 677 executions)
  • [main] 3 failures in step part-4 (0.9% fail rate in 337 executions)
  • [main] 3 failures in pipeline elasticsearch-pull-request (0.9% fail rate in 328 executions)

Build Scans:

@elasticsearchmachine elasticsearchmachine added Team:Data Management Meta label for data/management team needs:risk Requires assignment of a risk label (low, medium, blocker) labels Jan 29, 2025
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-data-management (Team:Data Management)

@dakrone dakrone added low-risk An open issue or test failure that is a low risk to future releases and removed needs:risk Requires assignment of a risk label (low, medium, blocker) labels Jan 29, 2025
@slobodanadamovic slobodanadamovic self-assigned this Jan 30, 2025
@slobodanadamovic slobodanadamovic mentioned this issue Jan 30, 2025
Closed
slobodanadamovic added a commit to slobodanadamovic/elasticsearch that referenced this issue Feb 3, 2025
@slobodanadamovic
Disable queryable built-in roles feature for core and datastream YAML…
0644dbf 
… tests

Resolves elastic#121536
Resolves elastic#121513
Resolves elastic#121484
Resolves elastic#121478
Resolves elastic#121290
Resolves elastic#121246
Resolves elastic#121242
Resolves elastic#121238
Resolves elastic#121186
Resolves elastic#121131
Resolves elastic#121130
Resolves elastic#121128
Resolves elastic#121014
Resolves elastic#120965
Resolves elastic#120920
Resolves elastic#120890
@slobodanadamovic slobodanadamovic mentioned this issue Feb 3, 2025
Merged
slobodanadamovic added a commit to slobodanadamovic/elasticsearch that referenced this issue Feb 4, 2025
@slobodanadamovic
Disable queryable built-in roles feature for core and datastream YAML…
f845ba5 
… tests (elastic#121541)

This PR disables the "queryable built-in roles" feature for the `CoreWithSecurityClientYamlTestSuiteIT` and `DataStreamsClientYamlTestSuiteIT` YAML test suites.
The feature was enabled by default in the elastic#120323 PR, which asynchronously creates the `.security` index after cluster formation and indexes all built-in roles. The asynchronous creation of the `.security` index introduces non-deterministic behavior in our YAML tests.
Since these test suites are not intended to verify the queryable built-in roles functionality, having the feature enabled introduced flakiness and unnecessary complexity to handle `.security` in existing tests. These tests would have to exclude the `.security` index in some way (by adjusting permissions or API calls), and in the end cleanup (delete) the `.security` index. This simply adds overhead without much gain. The feature is already test covered by `XPackRestIT` and other integration/REST tests, disabling it here would not compromise test coverage. Instead, it ensures these suites remain deterministic and focused on the behaviors they were designed to verify.

Resolves elastic#121536
Resolves elastic#121513
Resolves elastic#121484
Resolves elastic#121478
Resolves elastic#121290
Resolves elastic#121246
Resolves elastic#121242
Resolves elastic#121238
Resolves elastic#121186
Resolves elastic#121131
Resolves elastic#121130
Resolves elastic#121128
Resolves elastic#121014
Resolves elastic#120965
Resolves elastic#120920
Resolves elastic#120890

(cherry picked from commit d1beb01)

# Conflicts:
#	muted-tests.yml
slobodanadamovic added a commit to slobodanadamovic/elasticsearch that referenced this issue Feb 4, 2025
@slobodanadamovic
Disable queryable built-in roles feature for core and datastream YAML…
206eb0d 
… tests (elastic#121541)

This PR disables the "queryable built-in roles" feature for the `CoreWithSecurityClientYamlTestSuiteIT` and `DataStreamsClientYamlTestSuiteIT` YAML test suites.
The feature was enabled by default in the elastic#120323 PR, which asynchronously creates the `.security` index after cluster formation and indexes all built-in roles. The asynchronous creation of the `.security` index introduces non-deterministic behavior in our YAML tests.
Since these test suites are not intended to verify the queryable built-in roles functionality, having the feature enabled introduced flakiness and unnecessary complexity to handle `.security` in existing tests. These tests would have to exclude the `.security` index in some way (by adjusting permissions or API calls), and in the end cleanup (delete) the `.security` index. This simply adds overhead without much gain. The feature is already test covered by `XPackRestIT` and other integration/REST tests, disabling it here would not compromise test coverage. Instead, it ensures these suites remain deterministic and focused on the behaviors they were designed to verify.

Resolves elastic#121536
Resolves elastic#121513
Resolves elastic#121484
Resolves elastic#121478
Resolves elastic#121290
Resolves elastic#121246
Resolves elastic#121242
Resolves elastic#121238
Resolves elastic#121186
Resolves elastic#121131
Resolves elastic#121130
Resolves elastic#121128
Resolves elastic#121014
Resolves elastic#120965
Resolves elastic#120920
Resolves elastic#120890

(cherry picked from commit d1beb01)

# Conflicts:
#	muted-tests.yml
slobodanadamovic added a commit to slobodanadamovic/elasticsearch that referenced this issue Feb 4, 2025
@slobodanadamovic
Disable queryable built-in roles feature for core and datastream YAML…
a78b152 
… tests (elastic#121541)

This PR disables the "queryable built-in roles" feature for the `CoreWithSecurityClientYamlTestSuiteIT` and `DataStreamsClientYamlTestSuiteIT` YAML test suites.
The feature was enabled by default in the elastic#120323 PR, which asynchronously creates the `.security` index after cluster formation and indexes all built-in roles. The asynchronous creation of the `.security` index introduces non-deterministic behavior in our YAML tests.
Since these test suites are not intended to verify the queryable built-in roles functionality, having the feature enabled introduced flakiness and unnecessary complexity to handle `.security` in existing tests. These tests would have to exclude the `.security` index in some way (by adjusting permissions or API calls), and in the end cleanup (delete) the `.security` index. This simply adds overhead without much gain. The feature is already test covered by `XPackRestIT` and other integration/REST tests, disabling it here would not compromise test coverage. Instead, it ensures these suites remain deterministic and focused on the behaviors they were designed to verify.

Resolves elastic#121536
Resolves elastic#121513
Resolves elastic#121484
Resolves elastic#121478
Resolves elastic#121290
Resolves elastic#121246
Resolves elastic#121242
Resolves elastic#121238
Resolves elastic#121186
Resolves elastic#121131
Resolves elastic#121130
Resolves elastic#121128
Resolves elastic#121014
Resolves elastic#120965
Resolves elastic#120920
Resolves elastic#120890

(cherry picked from commit d1beb01)

# Conflicts:
#	muted-tests.yml
elasticsearchmachine pushed a commit that referenced this issue Feb 4, 2025
@slobodanadamovic
Disable queryable built-in roles feature for core and datastream YAML…
a0f0cea 
… tests (#121541) (#121663)

This PR disables the "queryable built-in roles" feature for the `CoreWithSecurityClientYamlTestSuiteIT` and `DataStreamsClientYamlTestSuiteIT` YAML test suites.
The feature was enabled by default in the #120323 PR, which asynchronously creates the `.security` index after cluster formation and indexes all built-in roles. The asynchronous creation of the `.security` index introduces non-deterministic behavior in our YAML tests.
Since these test suites are not intended to verify the queryable built-in roles functionality, having the feature enabled introduced flakiness and unnecessary complexity to handle `.security` in existing tests. These tests would have to exclude the `.security` index in some way (by adjusting permissions or API calls), and in the end cleanup (delete) the `.security` index. This simply adds overhead without much gain. The feature is already test covered by `XPackRestIT` and other integration/REST tests, disabling it here would not compromise test coverage. Instead, it ensures these suites remain deterministic and focused on the behaviors they were designed to verify.

Resolves #121536
Resolves #121513
Resolves #121484
Resolves #121478
Resolves #121290
Resolves #121246
Resolves #121242
Resolves #121238
Resolves #121186
Resolves #121131
Resolves #121130
Resolves #121128
Resolves #121014
Resolves #120965
Resolves #120920
Resolves #120890

(cherry picked from commit d1beb01)

# Conflicts:
#	muted-tests.yml
elasticsearchmachine pushed a commit that referenced this issue Feb 4, 2025
@slobodanadamovic
Disable queryable built-in roles feature for core and datastream YAML…
9ca2890 
… tests (#121541) (#121658)

This PR disables the "queryable built-in roles" feature for the `CoreWithSecurityClientYamlTestSuiteIT` and `DataStreamsClientYamlTestSuiteIT` YAML test suites.
The feature was enabled by default in the #120323 PR, which asynchronously creates the `.security` index after cluster formation and indexes all built-in roles. The asynchronous creation of the `.security` index introduces non-deterministic behavior in our YAML tests.
Since these test suites are not intended to verify the queryable built-in roles functionality, having the feature enabled introduced flakiness and unnecessary complexity to handle `.security` in existing tests. These tests would have to exclude the `.security` index in some way (by adjusting permissions or API calls), and in the end cleanup (delete) the `.security` index. This simply adds overhead without much gain. The feature is already test covered by `XPackRestIT` and other integration/REST tests, disabling it here would not compromise test coverage. Instead, it ensures these suites remain deterministic and focused on the behaviors they were designed to verify.

Resolves #121536
Resolves #121513
Resolves #121484
Resolves #121478
Resolves #121290
Resolves #121246
Resolves #121242
Resolves #121238
Resolves #121186
Resolves #121131
Resolves #121130
Resolves #121128
Resolves #121014
Resolves #120965
Resolves #120920
Resolves #120890

(cherry picked from commit d1beb01)

# Conflicts:
#	muted-tests.yml
elasticsearchmachine pushed a commit that referenced this issue Feb 4, 2025
@slobodanadamovic
Disable queryable built-in roles feature for core and datastream YAML…
d11c3b5 
… tests (#121541) (#121664)

This PR disables the "queryable built-in roles" feature for the `CoreWithSecurityClientYamlTestSuiteIT` and `DataStreamsClientYamlTestSuiteIT` YAML test suites.
The feature was enabled by default in the #120323 PR, which asynchronously creates the `.security` index after cluster formation and indexes all built-in roles. The asynchronous creation of the `.security` index introduces non-deterministic behavior in our YAML tests.
Since these test suites are not intended to verify the queryable built-in roles functionality, having the feature enabled introduced flakiness and unnecessary complexity to handle `.security` in existing tests. These tests would have to exclude the `.security` index in some way (by adjusting permissions or API calls), and in the end cleanup (delete) the `.security` index. This simply adds overhead without much gain. The feature is already test covered by `XPackRestIT` and other integration/REST tests, disabling it here would not compromise test coverage. Instead, it ensures these suites remain deterministic and focused on the behaviors they were designed to verify.

Resolves #121536
Resolves #121513
Resolves #121484
Resolves #121478
Resolves #121290
Resolves #121246
Resolves #121242
Resolves #121238
Resolves #121186
Resolves #121131
Resolves #121130
Resolves #121128
Resolves #121014
Resolves #120965
Resolves #120920
Resolves #120890

(cherry picked from commit d1beb01)

# Conflicts:
#	muted-tests.yml
fzowl pushed a commit to voyage-ai/elasticsearch that referenced this issue Feb 4, 2025
@slobodanadamovic @fzowl
Disable queryable built-in roles feature for core and datastream YAML…
a1ec29f 
… tests (elastic#121541)

This PR disables the "queryable built-in roles" feature for the `CoreWithSecurityClientYamlTestSuiteIT` and `DataStreamsClientYamlTestSuiteIT` YAML test suites. 
The feature was enabled by default in the elastic#120323 PR, which asynchronously creates the `.security` index after cluster formation and indexes all built-in roles. The asynchronous creation of the `.security` index introduces non-deterministic behavior in our YAML tests. 
Since these test suites are not intended to verify the queryable built-in roles functionality, having the feature enabled introduced flakiness and unnecessary complexity to handle `.security` in existing tests. These tests would have to exclude the `.security` index in some way (by adjusting permissions or API calls), and in the end cleanup (delete) the `.security` index. This simply adds overhead without much gain. The feature is already test covered by `XPackRestIT` and other integration/REST tests, disabling it here would not compromise test coverage. Instead, it ensures these suites remain deterministic and focused on the behaviors they were designed to verify.

Resolves elastic#121536 
Resolves elastic#121513 
Resolves elastic#121484 
Resolves elastic#121478 
Resolves elastic#121290 
Resolves elastic#121246 
Resolves elastic#121242 
Resolves elastic#121238 
Resolves elastic#121186 
Resolves elastic#121131 
Resolves elastic#121130 
Resolves elastic#121128 
Resolves elastic#121014 
Resolves elastic#120965 
Resolves elastic#120920 
Resolves elastic#120890
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@slobodanadamovic slobodanadamovic

Labels
:Data Management/Data streams Data streams and their lifecycles low-risk An open issue or test failure that is a low risk to future releases Team:Data Management Meta label for data/management team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
3 participants
@dakrone @slobodanadamovic @elasticsearchmachine