[Index Management] Test LogsDb Index Template Modifications #206548

SoniaSanzV · 2025-01-14T08:30:39Z

Part of #203716

Summary

This PR introduces a new test case for LogsDB in both Stateful and Serverless:

Verify that users can override LogsDB index settings including: ignore_above, ignore_malformed, ignore_dynamic_beyond_limit, subobjects and timestamp format.

For modify subobjects and timestamp format it must be done from the mappings tab. For ignore_above, ignore_malformed, ignore_dynamic_beyond_limit the configuration is done in the Settings tab.

It also introduces a test case only for Stateful (enableMappingsSourceFieldSection is disabled for serverless)

Verify that users cannot disable synthetic source for a LogsDB index.

…logsdb index

elasticmachine · 2025-01-14T08:30:41Z

Pinging @elastic/kibana-management (Team:Kibana Management)

ElenaStoeva

Thanks for working on this @SoniaSanzV! Overall, it looks good, but I have a few comments/questions. Let me know what you think.

x-pack/test/functional/apps/index_management/index_templates_tab/index_template_tab.ts

ElenaStoeva · 2025-01-14T10:58:13Z

x-pack/test/functional/apps/index_management/index_templates_tab/index_template_tab.ts

@@ -79,5 +79,96 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => {
      // Close detail tab
      await testSubjects.click('closeDetailsButton');
    });
+
+    it('can modify ignore_above, ignore_malformed, ignore_dynamic_beyond_limit, subobjects and timestamp format in an index template with logsdb index mode', async () => {


This test verifies that we can create an index template with the mentioned index/mappings settings, but doesn't the test flow require overriding them? I'm wondering if we should also verify that we can edit these settings after the initial creation.

You are right! I can create the index template from ES and then modify it in this test.

ElenaStoeva · 2025-01-14T11:09:12Z

...test_serverless/functional/test_suites/common/management/index_management/index_templates.ts

@@ -128,6 +128,94 @@ export default ({ getPageObjects, getService }: FtrProviderContext) => {
        expect(await testSubjects.getVisibleText('indexModeValue')).to.be('LogsDB');
        await testSubjects.click('closeDetailsButton');
      });
+
+      it('can modify ignore_above, ignore_malformed, ignore_dynamic_beyond_limit, subobjects and timestamp format in an index template with logsdb index mode', async () => {


For serverless FTRs, we usually add tests for the main functionalities unless some logic is different from stateful - otherwise, there is a lot of code duplication and too heavy tests. Since the related index templates logic is same for both stateful and serverless, I wonder whether it's necessary to add this test here as well. 🤔 Wdyt?

I added both because it was specified in the Issue description, but I see your point. I don't have a strong opinion.

I see...let's leave it then to make sure it works correctly in serverless too.

ElenaStoeva

Thanks for addressing my feedback @SoniaSanzV! I added a few more comments. Also, could you run the flaky test runner to check for flakiness?

ElenaStoeva · 2025-01-15T17:22:07Z

x-pack/test/functional/apps/index_management/index_templates_tab/index_template_tab.ts

+          data_stream: {},
+          template: {
+            settings: {
+              mode: 'logsdb',


I think the setting should be index.mode instead of just mode. See https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html

It seems that both are accepted. I'll keep yours since is more verbose.

ElenaStoeva · 2025-01-15T17:23:14Z

x-pack/test/functional/apps/index_management/index_templates_tab/index_template_tab.ts

+        await testSubjects.setValue(
+          'kibanaCodeEditor',
+          JSON.stringify({
+            index: {


I wonder if this overwrites the index.mode setting to standard since we don't specify it explicitly and it might be reset to the default value 🤔

x-pack/test/functional/apps/index_management/index_templates_tab/index_template_tab.ts

ElenaStoeva · 2025-01-15T17:25:27Z

x-pack/test/functional/apps/index_management/index_templates_tab/index_template_tab.ts

+          '_source can not be disabled in index using [logsdb] index mode'
+        );
+
+        // Comeback to templates tab


Why do we need this step?

After all the test we delete the index template and and click reloadButton to avoid refreshing the page. I'll add it in the afterEach section.

ElenaStoeva · 2025-01-15T17:26:26Z

x-pack/test/functional/apps/index_management/index_templates_tab/index_template_tab.ts

-      // Fill out required fields
-      await testSubjects.setValue('nameField', INDEX_TEMPLATE_NAME);
-      await testSubjects.setValue('indexPatternsField', 'logsdb-test-index-pattern');
+    describe('index modification', () => {


did you mean "index template modification"?

ElenaStoeva · 2025-01-15T17:27:20Z

...test_serverless/functional/test_suites/common/management/index_management/index_templates.ts

+        });
+
+        // Close detail tab
+        await testSubjects.click('closeDetailsButton');


This step could be added in a before() if it's needed for the next test.

Added in the afterEach section.

elasticmachine · 2025-01-16T10:37:05Z

💚 Build Succeeded

Buildkite Build
Commit: 293809f

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`indexManagement`	713.2KB	713.2KB	+80.0B

History

💛 Build #266813 was flaky f034a40

cc @SoniaSanzV

kibanamachine · 2025-01-16T11:19:33Z

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7734

[✅] x-pack/test/functional/apps/index_management/config.ts: 200/200 tests passed.

see run history

SoniaSanzV · 2025-01-16T11:51:10Z

Thanks for addressing my feedback @SoniaSanzV! I added a few more comments. Also, could you run the flaky test runner to check for flakiness?

Ran and passed!

ElenaStoeva

Thanks for addressing my comments @SoniaSanzV! Latest changes lgtm 👍

kibanamachine · 2025-01-17T06:15:40Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12823753517

…206548) Part of elastic#203716 ## Summary This PR introduces a new test case for LogsDB in both Stateful and Serverless: > Verify that users can override LogsDB index settings including: ignore_above, ignore_malformed, ignore_dynamic_beyond_limit, subobjects and timestamp format. For modify `subobjects` and `timestamp` format it must be done from the mappings tab. For `ignore_above`, `ignore_malformed`, `ignore_dynamic_beyond_limit` the configuration is done in the Settings tab. It also introduces a test case only for Stateful (enableMappingsSourceFieldSection [is disabled](https://github.com/elastic/kibana/blob/9c6de6aabced9d180bf1d68ec42708c27e5616d6/config/serverless.yml#L112) for serverless) > Verify that users cannot disable synthetic source for a LogsDB index. (cherry picked from commit 75e1866)

kibanamachine · 2025-01-17T06:20:51Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

…06548) (#207019) # Backport This will backport the following commits from `main` to `8.x`: - [[Index Management] Test LogsDb Index Template Modifications (#206548)](#206548)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Sonia Sanz Vivas <[email protected]>

…206548) Part of elastic#203716 ## Summary This PR introduces a new test case for LogsDB in both Stateful and Serverless: > Verify that users can override LogsDB index settings including: ignore_above, ignore_malformed, ignore_dynamic_beyond_limit, subobjects and timestamp format. For modify `subobjects` and `timestamp` format it must be done from the mappings tab. For `ignore_above`, `ignore_malformed`, `ignore_dynamic_beyond_limit` the configuration is done in the Settings tab. It also introduces a test case only for Stateful (enableMappingsSourceFieldSection [is disabled](https://github.com/elastic/kibana/blob/9c6de6aabced9d180bf1d68ec42708c27e5616d6/config/serverless.yml#L112) for serverless) > Verify that users cannot disable synthetic source for a LogsDB index.

Create FTs for verify that user can change certain paramethers for a …

f034a40

…logsdb index

SoniaSanzV self-assigned this Jan 14, 2025

SoniaSanzV requested a review from a team as a code owner January 14, 2025 08:30

SoniaSanzV mentioned this pull request Jan 14, 2025

LogsDB functional tests #203716

Open

11 tasks

ElenaStoeva reviewed Jan 14, 2025

View reviewed changes

SoniaSanzV added 2 commits January 14, 2025 15:39

Address PR comments

005c68e

Merge branch 'main' into logsdb_index_templates_fts

8dcd145

SoniaSanzV requested a review from ElenaStoeva January 15, 2025 07:19

SoniaSanzV added 2 commits January 15, 2025 14:52

Test that syntethic source cannot be disabled for a logsdb index

c17862f

Merge branch 'main' into logsdb_index_templates_fts

a8e53c6

ElenaStoeva reviewed Jan 15, 2025

View reviewed changes

SoniaSanzV added 2 commits January 16, 2025 10:06

Add some fixes

4210c38

Merge branch 'main' into logsdb_index_templates_fts

293809f

SoniaSanzV changed the title ~~[Index Management] Create FTs for verify that user can change certain paramethers for a …~~ [Index Management] Test LogsDb Index Template Modifications Jan 16, 2025

ElenaStoeva approved these changes Jan 16, 2025

View reviewed changes

SoniaSanzV merged commit 75e1866 into elastic:main Jan 17, 2025
8 checks passed

SoniaSanzV deleted the logsdb_index_templates_fts branch January 17, 2025 06:15

kibanamachine added the v9.0.0 label Jan 17, 2025

kibanamachine mentioned this pull request Jan 17, 2025

[8.x] [Index Management] Test LogsDb Index Template Modifications (#206548) #207019

Merged

kibanamachine added the v8.18.0 label Jan 17, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Index Management] Test LogsDb Index Template Modifications #206548

[Index Management] Test LogsDb Index Template Modifications #206548

SoniaSanzV commented Jan 14, 2025 •

edited by kibanamachine

Loading

elasticmachine commented Jan 14, 2025

ElenaStoeva left a comment

ElenaStoeva Jan 14, 2025

SoniaSanzV Jan 14, 2025

ElenaStoeva Jan 14, 2025

SoniaSanzV Jan 14, 2025

ElenaStoeva Jan 14, 2025

ElenaStoeva left a comment

ElenaStoeva Jan 15, 2025

SoniaSanzV Jan 16, 2025

ElenaStoeva Jan 15, 2025

ElenaStoeva Jan 15, 2025

SoniaSanzV Jan 16, 2025

ElenaStoeva Jan 15, 2025

ElenaStoeva Jan 15, 2025

SoniaSanzV Jan 16, 2025

elasticmachine commented Jan 16, 2025

kibanamachine commented Jan 16, 2025

SoniaSanzV commented Jan 16, 2025

ElenaStoeva left a comment

kibanamachine commented Jan 17, 2025

kibanamachine commented Jan 17, 2025

[Index Management] Test LogsDb Index Template Modifications #206548

[Index Management] Test LogsDb Index Template Modifications #206548

Conversation

SoniaSanzV commented Jan 14, 2025 • edited by kibanamachine Loading

Summary

elasticmachine commented Jan 14, 2025

ElenaStoeva left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ElenaStoeva left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elasticmachine commented Jan 16, 2025

💚 Build Succeeded

Metrics [docs]

Async chunks

History

kibanamachine commented Jan 16, 2025

Flaky Test Runner Stats

🎉 All tests passed! - kibana-flaky-test-suite-runner#7734

SoniaSanzV commented Jan 16, 2025

ElenaStoeva left a comment

Choose a reason for hiding this comment

kibanamachine commented Jan 17, 2025

kibanamachine commented Jan 17, 2025

💚 All backports created successfully

Questions ?

SoniaSanzV commented Jan 14, 2025 •

edited by kibanamachine

Loading