Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KafkaSupervisorIOConfig's RejectionPeriod config #507

Open
zhaojiandong opened this issue Sep 4, 2018 · 0 comments
Open

KafkaSupervisorIOConfig's RejectionPeriod config #507

zhaojiandong opened this issue Sep 4, 2018 · 0 comments

Comments

@zhaojiandong
Copy link
Contributor

Does this 2 config mixed up ?
lateMessageRejectionPeriod and earlyMessageRejectionPeriod

http://druid.io/docs/latest/development/extensions-core/kafka-ingestion.html#KafkaSupervisorIOConfig

Field Type Description Required
lateMessageRejectionPeriod ISO8601 Period Configure tasks to reject messages with timestamps earlier than this period before the task was created; for example if this is set to PT1H and the supervisor creates a task at 2016-01-01T12:00Z, messages with timestamps earlier than 2016-01-01T11:00Z will be dropped. This may help prevent concurrency issues if your data stream has late messages and you have multiple pipelines that need to operate on the same segments (e.g. a realtime and a nightly batch ingestion pipeline). no (default == none)
earlyMessageRejectionPeriod ISO8601 Period Configure tasks to reject messages with timestamps later than this period after the task reached its taskDuration; for example if this is set to PT1H, the taskDuration is set to PT1H and the supervisor creates a task at 2016-01-01T12:00Z, messages with timestamps later than 2016-01-01T14:00Z will be dropped. no (default == none)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant