You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, when a new consumer type registers in the Outbox, it's implicitly assumed consumption boundary is 0.
This effectively means processing all historically accumulated events of the connected producer starting from the very beginning.
This might be a good approach to completely rebuild new consumer's state solely from events.
In some cases, like recent DatasetEntryService,this behavior is undesirable, because there is an explicit initialization job, DatasetEntryIndexer, which is building the initial state of the service from an external source (storage system's state).
Such cases do not require consuming historically accumulated events with overlapping semantics, so consumption should start from the current producer's latest message.
In summary, create a mechanism in Outbox system so that consumer can specify what to do if it's first time registered in the system:
start from event 0 (full history reconstruction)
start from the latest event of the producer (custom initialization procedure, like indexing).
Currently, when a new consumer type registers in the Outbox, it's implicitly assumed consumption boundary is 0.
This effectively means processing all historically accumulated events of the connected producer starting from the very beginning.
This might be a good approach to completely rebuild new consumer's state solely from events.
In some cases, like recent
DatasetEntryService
,this behavior is undesirable, because there is an explicit initialization job,DatasetEntryIndexer
, which is building the initial state of the service from an external source (storage system's state).Such cases do not require consuming historically accumulated events with overlapping semantics, so consumption should start from the current producer's latest message.
In summary, create a mechanism in Outbox system so that consumer can specify what to do if it's first time registered in the system:
I.e.:
#[meta(MessageConsumerMeta {
consumer_name: MESSAGE_CONSUMER_KAMU_DATASET_ENTRY_SERVICE,
initial_consumption_boundary: InitialConsumptionBoundary::LatestProduced
...
})]
impl DatasetEntryService {
The text was updated successfully, but these errors were encountered: