Skip to content

Commit

Permalink
Fix handling of indexes without any properties mapped
Browse files Browse the repository at this point in the history
It's possible to have an index (or mapping type pre ES7) without any
properties mapped (yet).

Previously, on ES7+ the exporter would assume this meant the server was
actually pre ES7 and try and parse the mappings as mapping types, causing
errors if there were any non-property settings in the mappings.

On pre ES7, the exporter would assume every mapping type had properties,
causing errors if any type didn't.

Detection of pre ES7 mapping responses has been improved to avoid these
errors, and indexes/mapping types without properties are skipped.
  • Loading branch information
braedon committed Nov 12, 2020
1 parent 275f7f9 commit 262f24f
Show file tree
Hide file tree
Showing 2 changed files with 33 additions and 3 deletions.
18 changes: 15 additions & 3 deletions prometheus_es_exporter/indices_mappings_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,17 +47,29 @@ def parse_index(index, mappings, metric=None):
if 'properties' in mappings:
counts = count_object_fields(mappings)

# Older Elasticsearch versions had the concept of mapping types, so the root maps from mapping
# type to the object mappings for that type. We have to count the fields the types separately.
else:
# No properties defined, so this could be a pre ES7 Elasticsearch instance.
# Before ES7, Elasticsearch had the concept of mapping types, so the mapping root maps from the
# mapping type to the object mappings for the type.
#
# However, this could also be ES7+, and the index just doesn't have any properties mapped yet.
#
# If any value is an dict with properties, we know it's pre ES7, and we have to count the
# fields of the types separately.
elif any(isinstance(value, dict) and 'properties' in value for value in mappings.values()):
counts = {}
for mapping_type, type_mappings in mappings.items():
if mapping_type == '_default_':
# Skip the default mapping type - it's a template used for new mapping types, not an
# actual type itself.
continue
if 'properties' not in type_mappings:
# Skip any empty mapping types that don't have any properties.
continue
counts = count_object_fields(type_mappings, counts=counts)

else:
counts = {}

metrics = []
for field_type, count in counts.items():
metrics.append((metric, '', merge_dicts_ordered(labels, field_type=field_type), count))
Expand Down
18 changes: 18 additions & 0 deletions tests/test_indices_mappings_parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,24 @@ def test_endpoint(self):
result = convert_result(parse_response(response))
self.assertEqual(expected, result)

# Sample response generated by querying the endpoint on a Elasticsearch
# server populated with the following data (http command = Httpie utility):
# > http -v PUT localhost:9200/foo
# > http -v PUT localhost:9200/foo/_mapping date_detection:=false
def test_endpoint_no_properties(self):
# Endpoint: /_mappings?pretty
response = {
"foo": {
"mappings": {
"date_detection": False
}
}
}

expected = {}
result = convert_result(parse_response(response))
self.assertEqual(expected, result)

# Pre ES7 version with mapping types.
# Sample response generated by querying the endpoint on a Elasticsearch
# server populated with the following data (http command = Httpie utility):
Expand Down

0 comments on commit 262f24f

Please sign in to comment.