The possibility of assembling multi-algorithm node data pipelines in a dynamic and visual way

[meua]

1. Background

When creating a complex data pipeline composed of multiple input sources and multiple operator nodes by writing a declarative yaml file, the relationship is chaotic, and the successive dependencies are like a mess, and the reference relationship between each other cannot be well displayed. It takes a lot of money for a novice to master It takes a lot of time to clarify the relationship between different nodes. Therefore, if a visual way to edit the data flow graph of the entire DAG can be provided, it will greatly reduce the entry threshold for ordinary developers. The visual data flow graph and the corresponding declarative Yaml can be transformed into each other, and the ease of use will be significantly improved by N times.

2. Functional interaction draft/visual draft

https://q67z5z.csb.app/

There are many open source projects for the same example:
https://github.com/logspace-ai/langflow
[#https://flowiseai.com/]{https://github.com/FlowiseAI/Flowise}
https://github.com/newcat/baklavajs/
https://github.com/antvis/X6
https://github.com/alibaba/butterfly
https://github.com/orchest/orchest

3. Detailed demand logic description

• Declarative yaml files can generate visual data flow graphs, and vice versa
• Assemble a more complex DAG diagram by dragging the Node on the sidebar to the editor
• Standardize the access of Node nodes and define a unified access standard (for example: when adding a new Node, you must provide a detailed description of the request/response parameters corresponding to interfaces such as http/gRPC and import the node to automatically generate the node)
• Service management of algorithm nodes (such as the number of running algorithm instances, flexible destruction of algorithm instances that are no longer needed, etc.)
• Zero-intrusion access method for external algorithms

4. Adaptation range requirements

Developers can create corresponding pipelines according to different perception task types, and define pipeline tasks through configuration files. Compared with before, the running process of each task is clearer, and it is also convenient for expansion. In addition, developers can also choose different algorithm plug-ins according to their needs. For example, the Apollo perception module provides 4 detectors. Developers can choose different detectors according to the configuration file to verify the detection effect. Through algorithm plug-ins, algorithm engineers are more focused Focus on the algorithm itself, without paying too much attention to the implementation of the framework.

[heyong4725]

Indeed this could be Dora Dev IDE, which will be part of Dora control plane

[meua]

If this function can be realized, it can greatly improve the solubility of dora.

[haixuanTao]

Can you provide the source code for demo you have attached?

Thanks

[meua]

Demo address: https://github.com/meua/dora-baklavajs-v2-example.git
This is just an example of a very simple front-end visualization, there are still many things to do if it is to be implemented.
Here is the framework documentation for front-end dependencies: https://v2.baklava.tech/

[meua]

other examples：ComfyUI + SeargeSDXL
https://github.com/comfyanonymous/ComfyUI
https://github.com/SeargeDP/SeargeSDXL