You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
The eventhistory service keeps events for 2 weeks (configurable). However, there is no limit on the amount of events that could happen during those 2 weeks. We could consider 1 million or more events that we need to handle during those 2 weeks.
This is also a problem for the activitylog service, which not only would need to store the ids of 1 million events, but also need to transfer the data of those events from the eventhistory service. Include additional processing such as sorting the list of events or ids.
Describe the solution you'd like
The eventhistory service should have a (configurable) limit on the number of events it can store. Once that limit is reached, older events should be removed to leave space to newer events. We can assume that, if the events are ordered by timestamp, it's unlikely that people would want to go through more than 10k events, so a default of 10k events should be good enough.
Note that the current TTL of 2 weeks should be kept. This storage size limit should be applied on top.
Describe alternatives you've considered
A fully persistent eventhistory would require to store data either in files or DB in order to have virtually unlimited storage. Combined with a proper search mechanism, we could check any event that happened 5 years ago because we wouldn't need to delete data (or at least we could storage a much larger event number).
So far, the implementation leans towards showing a partial list of events (the events that happened in the last 2 weeks), so it could make sense to also have a limit on the number of events we can store and show.
Additional context
Taking into account that most of our store implementations don't have an explicit size limit, we could provide a "soft" size limit ourselves. This means that we won't enforce the size limit (the number of events might be higher than the limit at any given time), but we'll try to keep it within the limit (if we detect we're passed the limit, we'll remove events until we're back on track).
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
The eventhistory service keeps events for 2 weeks (configurable). However, there is no limit on the amount of events that could happen during those 2 weeks. We could consider 1 million or more events that we need to handle during those 2 weeks.
This is also a problem for the activitylog service, which not only would need to store the ids of 1 million events, but also need to transfer the data of those events from the eventhistory service. Include additional processing such as sorting the list of events or ids.
Describe the solution you'd like
The eventhistory service should have a (configurable) limit on the number of events it can store. Once that limit is reached, older events should be removed to leave space to newer events. We can assume that, if the events are ordered by timestamp, it's unlikely that people would want to go through more than 10k events, so a default of 10k events should be good enough.
Note that the current TTL of 2 weeks should be kept. This storage size limit should be applied on top.
Describe alternatives you've considered
A fully persistent eventhistory would require to store data either in files or DB in order to have virtually unlimited storage. Combined with a proper search mechanism, we could check any event that happened 5 years ago because we wouldn't need to delete data (or at least we could storage a much larger event number).
So far, the implementation leans towards showing a partial list of events (the events that happened in the last 2 weeks), so it could make sense to also have a limit on the number of events we can store and show.
Additional context
Taking into account that most of our store implementations don't have an explicit size limit, we could provide a "soft" size limit ourselves. This means that we won't enforce the size limit (the number of events might be higher than the limit at any given time), but we'll try to keep it within the limit (if we detect we're passed the limit, we'll remove events until we're back on track).
The text was updated successfully, but these errors were encountered: