You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Druid operator should be extended to be able to deploy ingestion specs from definitions provided in CRDs.
Ingestion specs can be defined by the user via a customresource, which is watched by a controller in the Druid operator that then provisions these specs.
The crd will contain at least the following:
type of source (stream, flat file, ...)
failure behavior
none
retry
delete and retry
specification of the ingest job
inline
reference to configmap
reference to pvc (or file therein)
These objects will initially be considered read-only, so changes to them will not be propagated to Druid by the controller.
The initial implementation will not be a perfect ingestion task management solution, but rather a first attempt to offer something useful to our users.
The user needs to decide themselves what the appropriate failure behavior is for the spec they provide to the operator, whether duplicate data might be created by retrying etc.
The defined failure options should offer simple solutions for all scenarios:
task is too complex -> none , the user will investigate themselves
no duplicates expected -> retry , task is idempotent and can be retried
duplicates possible -> delete and retry , not idempotent, to be on the safe side delete the target before retrying
The text was updated successfully, but these errors were encountered:
Implementation ticket for #168
The Druid operator should be extended to be able to deploy ingestion specs from definitions provided in CRDs.
Ingestion specs can be defined by the user via a customresource, which is watched by a controller in the Druid operator that then provisions these specs.
The crd will contain at least the following:
These objects will initially be considered read-only, so changes to them will not be propagated to Druid by the controller.
The initial implementation will not be a perfect ingestion task management solution, but rather a first attempt to offer something useful to our users.
The user needs to decide themselves what the appropriate failure behavior is for the spec they provide to the operator, whether duplicate data might be created by retrying etc.
The defined failure options should offer simple solutions for all scenarios:
The text was updated successfully, but these errors were encountered: