figure out metrics expiry

[dgryski]

No description provided.

[kanatohodets]

The general plan is to change the write buffer to store a time.Time in the text metric set (perhaps along with the hashed metric value), then filter out old metrics from the write buffer data structures before giving them to indexes to materialize.

This plan was consciously broken for the text index when the active/standby bloom index bit was implemented. If we stick with bloom indexes, they'll just need to be regenerated from scratch every so often.

On Nov 20, 2016, at 10:12, Damian Gryski [email protected] wrote:

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[dgryski]

For the bloom index (and optionally the new one I'm working on too), we can partition by time and expire old indexes after a certain period. For example, create one new index every minute, keep 10 in memory, and expire the oldest.