Skip to content

Latest commit

 

History

History
99 lines (87 loc) · 2.81 KB

README.md

File metadata and controls

99 lines (87 loc) · 2.81 KB
Raw

melonade-process-manager

Workflow manager that implemented SAGA, written in typescript

To start

  nvm install v10.16.3
  nvm use

  npm install
  npm run start:watch

Docker .env

melonade.namespace=docker-compose
runners.max=1
kafka.conf.bootstrap.servers=localhost:29092
server.hostname=0.0.0.0
server.port=8080
server.enabled=true
topic.kafka.number_partitions=10
topic.kafka.replication_factor=1
task-definition.type=ZOOKEEPER
task-definition.zookeeper.connections=localhost:2181
workflow-definition.type=ZOOKEEPER
workflow-definition.zookeeper.connections=localhost:2181
task-instance.type=REDIS
task-instance.redis.host=localhost
task-instance.redis.port=16379
workflow-instance.type=REDIS
workflow-instance.redis.host=localhost
workflow-instance.redis.port=16379
transaction-instance.type=REDIS
transaction-instance.redis.host=localhost
transaction-instance.redis.port=16379

Recomended kafka's brokers config

offsets.topic.replication.factor=3 default.replication.factor=3 min.insync.replicas=2 offsets.retention.minutes=5000000 unclean.leader.election.enable=false auto.create.topics.enable=false

Tasks

  • Workflow Definition
  • Task Definition
  • Workflow Instance
  • Task Instance
  • State Translation
  • Store
  • Refacetor code, move to their's domain
  • Dispatcher
  • Consumer
  • Config
  • Logger
  • Use custom error
  • Clean up completed workflows/tasks
  • Delay dispatcher
  • Cron job transaction support
  • Failed workflow handling
  • Timeout workflow handling
  • Cancel workflow
  • Event store
  • Compensate workflow
  • Publish event to kafka
  • Update Workflow/Transaction's output
  • Time keeper
  • Refactor redundant action in "STATE"
  • Watch for workflowDefinition changed
  • Rewrite all the test XD
  • Instance delete retention => clean up when transaction finished
  • Graceful shutdown
  • Procress system task immediately intread of dispatch to itself
  • Use json schema
  • Document
  • Remove sub-workflow
  • Test parallel task inside another parallel task
  • Standardise the error, so it easilier to summary.
  • List running transaction redis not work correctly (paginates)
  • Pause workflow

Known issues

  • parallel tasks can be empty
  • Sub workflow won't get compensate => Not support sub-workflow anymore
  • Task/Workflow data send as string's ISO time format instead of number
  • Workflow Definition can have task that not existed
  • MongoDB not fast enough for 5000 concurent (Lag about 1 min before task updated) => Added redis store
  • Transaction did not cancelled if compensating
  • Parent task did not failed if child failed
  • Delay retry task sometime did not retry until limit
  • Transaction will failed if send cancel transaction while on compensate workflow