[Feature Request] Add _timestamp field for each document

[razoliver]

Is your feature request related to a problem? Please describe

When indexing new documents the current options for adding a timestamp are:

1. Using ingest pipeline ('set' or 'date' processors) and setting it as the default for an index or using it in every index request.
2. Using OSI (Ingestion Pipelines) and using a 'Date' processor to add a timestamp

Describe the solution you'd like

Every document that is indexed could have a _timestamp field by default when it's indexed. Upon update/upsert this _timestamp can be updated.

Related component

Indexing

Describe alternatives you've considered

No response

Additional context

No response

[peternied]

[Triage - attendees 1 2]
@razoliver Thanks for creating this issue; however, it isn't being accepted due that the caller can include their own timestamp in the document. Please feel free to re-open after addressing the reason.

[razoliver]

@peternied
If OS is a part of a pipeline it can be very useful to have a timestamp upon indexing which will be different than the timestamp on different parts of the pipeline

[peternied]

very useful

@razoliver Could you describe a scenario where a timestamp added at document creation wouldn't be useful, whereas this proposal timestamp would be?

[razoliver]

@peternied
If you want to calculate data-freshness in your pipeline you'd need both timestamps.

[peternied]

@razoliver For metrics there is a plugin Performance Analyzer that supports a large number of dimensions and granularities. The use case you are after might already by supported with this plugin.

Performance Analyzer is a plugin that contains an agent and REST API that allow you to query numerous cluster performance metrics, including aggregations of those metrics.