Skip to content

Map unmapped fields in ES monitoring template #133058

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

philippkahr
Copy link
Contributor

@philippkahr philippkahr commented Aug 18, 2025

Closes:

  1. I run a 8.19 cluster in cloud
  2. Activated monitoring
  3. Waited a bit and checked the unmapped fields in Kibana which was over ~60 fields.
  4. Turned on dynamic mapping with a strings as keyword dynamic template.
  5. Rolled over and waited a bit.
  6. Copied the created template, cleaned it up from unnecessary fields [1]
  7. Copied the created template over the .monitoring-es-mb template.
  8. Deleted all the data streams
  9. Watch data come in.
  10. Kibana reports a total of 580 fields mapped [2]

[1]: Per default there will be a new object created for each node id. Thus an unnecessary field such as elasticsearch.cluster.stats.state.nodes. <node-id>.attributes.logical_availability_zone will be mapped.
[2]: image

  • Have you signed the contributor license agreement?
  • Have you followed the contributor guidelines?
  • If submitting code, have you built your formula locally prior to submission with gradle check?
  • If submitting code, is your pull request against main? Unless there is a good reason otherwise, we prefer pull requests against main and will backport as needed.
  • If submitting code, have you checked that your submission is for an OS and architecture that we support?
  • If you are submitting this code for a class then read our policy for that.

@philippkahr philippkahr requested a review from a team as a code owner August 18, 2025 06:55
@elasticsearchmachine
Copy link
Collaborator

@philippkahr please enable the option "Allow edits and access to secrets by maintainers" on your PR. For more information, see the documentation.

@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label external-contributor Pull request authored by a developer outside the Elasticsearch team labels Aug 18, 2025
@philippkahr philippkahr changed the title elasticsearch.node.stats.indices.shard_stats.total_count missing in ES monitoring template Map unmapped fields in ES monitoring template Aug 18, 2025
@miltonhultgren miltonhultgren removed their request for review August 18, 2025 08:05
@szybia szybia added Team:Stack Monitoring and removed needs:triage Requires assignment of a team area label labels Aug 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external-contributor Pull request authored by a developer outside the Elasticsearch team Team:Stack Monitoring v9.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants