Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(web): Add DAG variables to web flow #1981

Merged
merged 95 commits into from
Sep 10, 2024
Merged
Changes from 1 commit
Commits
Show all changes
95 commits
Select commit Hold shift + click to select a range
725e557
feat: Support some higher-order operators
fangyinc Aug 28, 2024
1f676b9
feat: Support endpoint placeholder
fangyinc Aug 28, 2024
439b5b3
feat: Support mappers in inputs and outputs
fangyinc Aug 29, 2024
494eb58
feat: Support variables query API
fangyinc Aug 29, 2024
147051c
feat: Support query file metadatas
fangyinc Aug 29, 2024
08749a0
feat: Support dynamic parameters
fangyinc Aug 29, 2024
f1e00a7
feat: Support rag flow template
fangyinc Aug 30, 2024
c67b500
chore: Merge latest code
fangyinc Aug 30, 2024
bf63a96
feat: Support endpoint placeholder
fangyinc Aug 28, 2024
0e71991
chore: Merge latest code
fangyinc Aug 30, 2024
93527e0
feat: Support variables query API
fangyinc Aug 29, 2024
0219f57
feat: Support query file metadatas
fangyinc Aug 29, 2024
e9155f0
feat: Support dynamic parameters
fangyinc Aug 29, 2024
d152017
feat: Support rag flow template
fangyinc Aug 30, 2024
dbd1a4a
Merge branch 'flow2.0-operators' into chore/update-api
Aug 30, 2024
08b0c78
Merge branch 'flow2.0-operators' of https://github.com/eosphoros-ai/D…
Aug 30, 2024
3e0a243
Merge branch 'flow2.0-operators' into chore/update-api
Aug 30, 2024
f978a50
chore: update flow api (#1931)
Dreammy23 Aug 30, 2024
d07cce6
chore: update SaveFlowModal component to use useRouter hook and add u…
Aug 30, 2024
2393842
fix: update SaveFlowModal component to use useRouter hook and add use…
Dreammy23 Aug 30, 2024
0392ab6
feat:update deploy state in SaveFlowModal component
Aug 30, 2024
161d04d
feat: update deploy state in SaveFlowModal component (#1934)
Dreammy23 Aug 30, 2024
6030fe1
chore: Fix import flow error
fangyinc Aug 30, 2024
b1ca247
Merge branch 'flow2.0-operators' of https://github.com/eosphoros-ai/D…
fangyinc Aug 30, 2024
293fbb0
chore: Remove unnecessary code in Flow component
Aug 30, 2024
8d64679
chore: Remove unnecessary code in Flow component (#1936)
Dreammy23 Aug 30, 2024
f8789c0
feat: Add AddFlowVariable component for managing flow variables
Sep 2, 2024
ea71c49
feat: Add AddFlowVariable component for managing flow variables (#1944)
Dreammy23 Sep 2, 2024
a09587d
Merge branch 'flow2.0-operators' into chore/update-api-new
Sep 2, 2024
03e3f93
chore: update api (#1945)
Dreammy23 Sep 2, 2024
4edc64a
feat: import update canvas flow
yunfeng1993 Sep 2, 2024
1bc77f9
feat: Update AddFlowVariable component to include parameter management
Sep 2, 2024
4bd4d0a
chore: Update AddFlowVariable component to remove unnecessary code in…
Sep 2, 2024
7661de8
chore: Update AddFlowVariable component to remove unnecessary code i…
Dreammy23 Sep 3, 2024
230ba69
chore: Update getFlowNodes function to accept optional tags parameter
Sep 3, 2024
e14eb25
chore: Update AddNodesSider component to toggle between all nodes and…
Sep 3, 2024
72299da
chore: Remove commented out code for advanced and default modes in fl…
Sep 3, 2024
9c1f788
chore: Remove commented out code for advanced and default modes in fl…
Dreammy23 Sep 3, 2024
907897c
chore: Update AddFlowVariableModal component to include parameter man…
Sep 3, 2024
46cb6ac
chore: Update AddFlowVariableModal component to include parameter man…
Dreammy23 Sep 3, 2024
bc87dec
feat: upload metadata 、ui size settings 276 320 530 、import flow
yunfeng1993 Sep 3, 2024
37aed53
feat:Optimized code
Sep 4, 2024
d96e1ec
fix: upload loading
Sep 4, 2024
839a946
chore: Update AddFlowVariableModal component to include parameter man…
Sep 4, 2024
bb97721
Update AddFlowVariableModal component to include parameter management
Sep 4, 2024
59b0e68
Update globals.css
yunfeng1993 Sep 4, 2024
75788f1
Update ctx-axios.ts
yunfeng1993 Sep 4, 2024
9b9030c
Delete web/utils/ctx-axios.ts
yunfeng1993 Sep 4, 2024
ecc9c42
fix: axios file
Sep 4, 2024
b87b42f
feat: upload metadata 、ui size settings 276 320 530 、import flow (#1965)
Dreammy23 Sep 4, 2024
04af499
Update AddFlowVariableModal component to include parameter management
Sep 4, 2024
ed12ee0
Merge branch 'feat/sprint-flow-remaining-requirements' into feat/dag
Sep 4, 2024
af09d14
feat: Update AddFlowVariableModal component to include parameter mana…
Dreammy23 Sep 4, 2024
fbfb851
feat:Eslint formatting
Sep 4, 2024
9a4e6b6
feat: Add support for retrieving keys and variables by key in Flow API
Sep 4, 2024
f6bc56a
feat: Add support for retrieving keys and variables by key in Flow AP…
Dreammy23 Sep 4, 2024
25944c7
feat: Refactor AddFlowVariableModal to use buildVariableString from u…
Sep 4, 2024
5a35640
refactor: Refactor AddFlowVariableModal to use buildVariableString fr…
Dreammy23 Sep 4, 2024
f8ce7d4
feat: Check serialization for AWEL operator function
fangyinc Sep 4, 2024
8fb595a
feat: Update AddNodesSider component to use Switch instead of Tag for…
Sep 4, 2024
1794069
feat: Update AddNodesSider component to use Switch instead of Tag for…
Dreammy23 Sep 4, 2024
6ca1381
Merge branch 'flow2.0-operators' into chore/update-api
Sep 4, 2024
cbfd3d2
chore: update api (#1972)
Dreammy23 Sep 4, 2024
00d9e51
refactor: Simplify AppContent component by removing unnecessary div a…
Sep 4, 2024
ef04432
fix: set flow variable default value
Sep 4, 2024
ef624b0
fix: set flow variable default value (#1973)
Dreammy23 Sep 4, 2024
5617385
feat: Add helper function to render control component in AddFlowVaria…
Sep 4, 2024
19b8d45
feat: Add helper function to render control component in AddFlowVaria…
Dreammy23 Sep 4, 2024
70dc475
feat: Bring In Template
yunfeng1993 Sep 4, 2024
3b6a5bd
feat: lang setting
yunfeng1993 Sep 4, 2024
62b051c
refactor: Update AddFlowVariableModal to use buildVariableString from…
Sep 4, 2024
f3fe4cd
refactor: Update AddFlowVariableModal to use buildVariableString from…
Dreammy23 Sep 4, 2024
45dec7a
refactor: Improve input handling in AddFlowVariableModal
Sep 4, 2024
b501920
refactor: Improve input handling in AddFlowVariableModal (#1977)
Dreammy23 Sep 4, 2024
a8b6b82
refactor: Update AddFlowVariableModal to use buildVariableString from…
Sep 4, 2024
104b7d0
refactor: Update AddFlowVariableModal to set a minimum height for the…
Sep 4, 2024
58051df
refactor: Update AddFlowVariableModal to set a minimum height for the…
Dreammy23 Sep 4, 2024
4d4aac4
refactor: Update AddFlowVariableModal to include hover effect on vari…
Sep 4, 2024
f214572
refactor: Update AddFlowVariableModal styles and functionality
Sep 4, 2024
7feb1c3
refactor: Update AddFlowVariableModal styles and functionality (#1979)
Dreammy23 Sep 4, 2024
21146bb
Merge branch 'feat/sprint-flow-remaining-requirements' into feat/spri…
Sep 4, 2024
95e119f
refactor: Update FlowTemplateModal component to use correct prop names
Sep 4, 2024
59bbe9b
feat: Bring In Template (#1975)
Dreammy23 Sep 4, 2024
b60dd3b
style: Rename template-flow-modal to flow-template-modal and update i…
Sep 4, 2024
595c1f2
Merge branch 'main' into chore/merge-main
Sep 5, 2024
2aa791c
chore: merge main (#1982)
Dreammy23 Sep 5, 2024
a3a9da8
feat: flow template add label 、 description
yunfeng1993 Sep 8, 2024
713714f
Update flow-template-modal.tsx
yunfeng1993 Sep 9, 2024
c39a213
Update flow-template-modal.tsx
yunfeng1993 Sep 9, 2024
f6b9dfa
Update flow.ts
yunfeng1993 Sep 9, 2024
bc717c4
Update flow.ts
yunfeng1993 Sep 9, 2024
cadc59b
feat: flow template add label 、 description (#1985)
Dreammy23 Sep 9, 2024
2e99116
chore: Merge latest code
fangyinc Sep 9, 2024
a5145aa
fix: Eslint formatting
Sep 9, 2024
f273cb4
fix: Eslint formatting (#1987)
Dreammy23 Sep 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: Support rag flow template
fangyinc committed Aug 30, 2024
commit d15201780086531715194bab8d3c69ee26e10d43
Empty file added dbgpt/serve/dbgpts/__init__.py
Empty file.
29 changes: 27 additions & 2 deletions dbgpt/serve/flow/api/endpoints.py
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@
from dbgpt.util import PaginationResult

from ..config import APP_NAME, SERVE_SERVICE_COMPONENT_NAME, ServeConfig
from ..service.service import Service
from ..service.service import Service, _parse_flow_template_from_json
from ..service.variables_service import VariablesService
from .schemas import (
FlowDebugRequest,
@@ -517,7 +517,7 @@ async def import_flow(
raise HTTPException(
status_code=400, detail="invalid json file, missing 'flow' key"
)
flow = ServeRequest.parse_obj(json_dict["flow"])
flow = _parse_flow_template_from_json(json_dict["flow"])
elif file_extension == "zip":
from ..service.share_utils import _parse_flow_from_zip_file

@@ -536,6 +536,31 @@ async def import_flow(
return Result.succ(flow)


@router.get(
"/flow/templates",
response_model=Result[PaginationResult[ServerResponse]],
dependencies=[Depends(check_api_key)],
)
async def query_flow_templates(
user_name: Optional[str] = Query(default=None, description="user name"),
sys_code: Optional[str] = Query(default=None, description="system code"),
page: int = Query(default=1, description="current page"),
page_size: int = Query(default=20, description="page size"),
service: Service = Depends(get_service),
) -> Result[PaginationResult[ServerResponse]]:
"""Query Flow templates."""

res = await blocking_func_to_async(
global_system_app,
service.get_flow_templates,
user_name,
sys_code,
page,
page_size,
)
return Result.succ(res)


def init_endpoints(system_app: SystemApp) -> None:
"""Initialize the endpoints"""
from .variables_provider import (
59 changes: 59 additions & 0 deletions dbgpt/serve/flow/service/service.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import json
import logging
import os
from typing import AsyncIterator, List, Optional, cast

import schedule
@@ -399,6 +400,47 @@ def get_list_by_page(
item.metadata = metadata.to_dict()
return page_result

def get_flow_templates(
self,
user_name: Optional[str] = None,
sys_code: Optional[str] = None,
page: int = 1,
page_size: int = 20,
) -> PaginationResult[ServerResponse]:
"""Get a list of Flow templates

Args:
user_name (Optional[str]): The user name
sys_code (Optional[str]): The system code
page (int): The page number
page_size (int): The page size
Returns:
List[ServerResponse]: The response
"""
local_file_templates = self._get_flow_templates_from_files()
return PaginationResult.build_from_all(local_file_templates, page, page_size)

def _get_flow_templates_from_files(self) -> List[ServerResponse]:
"""Get a list of Flow templates from files"""
user_lang = self._system_app.config.get_current_lang(default="en")
# List files in current directory
parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
template_dir = os.path.join(parent_dir, "templates", user_lang)
default_template_dir = os.path.join(parent_dir, "templates", "en")
if not os.path.exists(template_dir):
template_dir = default_template_dir
templates = []
for root, _, files in os.walk(template_dir):
for file in files:
if file.endswith(".json"):
try:
with open(os.path.join(root, file), "r") as f:
data = json.load(f)
templates.append(_parse_flow_template_from_json(data))
except Exception as e:
logger.warning(f"Load template {file} error: {str(e)}")
return templates

async def chat_stream_flow_str(
self, flow_uid: str, request: CommonLLMHttpRequestBody
) -> AsyncIterator[str]:
@@ -638,3 +680,20 @@ async def _wrapper_chat_stream_flow_str(
break
else:
yield f"data:{text}\n\n"


def _parse_flow_template_from_json(json_dict: dict) -> ServerResponse:
"""Parse the flow from json

Args:
json_dict (dict): The json dict

Returns:
ServerResponse: The flow
"""
flow_json = json_dict["flow"]
flow_json["editable"] = False
del flow_json["uid"]
flow_json["state"] = State.INITIALIZING
flow_json["dag_id"] = None
return ServerResponse(**flow_json)
Loading