Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alternative workflow processor behavior when order is used #755

Open
advancedlogic opened this issue May 1, 2021 · 1 comment
Open

Alternative workflow processor behavior when order is used #755

advancedlogic opened this issue May 1, 2021 · 1 comment
Labels
enhancement processors Any tasks or issues relating specifically to processors

Comments

@advancedlogic
Copy link

advancedlogic commented May 1, 2021

Currently the workflow processor runs branches trying to infere the DAG automatically or using order. The behavior of order prioritizes parallelism over sequential so in [ [ A, B], [C] ], A and B are processed simultaneously and then C is executed. When the graph becomes complex, it's hard to keep track of every parallel processing and it would be easier to describe the order as a graph with nodes and edges.
Considering the example in the documentation:


      /--> B -------------|--> D
     /                   /
A --|          /--> E --|
     \--> C --|          \
               \----------|--> F

described as:
[ [ A ], [ B, C ], [ E ], [ D, F ] ]

could become a list of edges:

[ [ A, B], [A, C], [B, D], [C, E ], [ C, F ], [ E, F ] ]

@Jeffail Jeffail added enhancement processors Any tasks or issues relating specifically to processors labels May 1, 2021
@jem-davies
Copy link

jem-davies commented May 28, 2024

#2599 - possible overlap with this issue

As part of the PR that proposes a solution to the issue 2599 - the way the DAG is represented in Benthos Config has been altered to :

          B:
            dependency_list: ["A"]
            processors:
              ...

So each Node in the DAG has a list of it's direct dependents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement processors Any tasks or issues relating specifically to processors
Projects
None yet
Development

No branches or pull requests

3 participants