Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/lint service front #56

Merged
merged 38 commits into from
Jul 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
4c4646c
fix: Let docker work
Ramimashkouk Jun 7, 2024
6f035aa
chore: Push bun.lockb
Ramimashkouk Jun 7, 2024
ed204b3
refactor: Rename run_backend -> run_app
Ramimashkouk Jun 10, 2024
dbf36dc
feat: Add endpoint to get dff cnds
Ramimashkouk Jun 10, 2024
b7e6e2e
fix: edit condition priority + delete condition fns
mxerf Jun 10, 2024
52d5bdd
fix: change api path get
mxerf Jun 10, 2024
31c41a5
feature: add save point to start/fallback label handler
mxerf Jun 10, 2024
21a18fa
feat: Support fallback node in backend
Ramimashkouk Jun 11, 2024
4f525ef
Merge branch 'working-docker2' of https://github.com/deeppavlov/dialo…
Ramimashkouk Jun 11, 2024
6857a61
fix: Correct index refreshing behaviour
Ramimashkouk Jun 17, 2024
8e1ea47
feature: combine build&run button
mxerf Jun 18, 2024
01a7195
feature: update fallback flag logic
mxerf Jun 18, 2024
9fc291d
fix deletion: mouse detection on pane
mxerf Jun 18, 2024
e0cc91c
fix: change ws url to dynamic
mxerf Jun 18, 2024
724c739
feature: autocomplete plugin init
mxerf Jun 18, 2024
7fd6623
Merge branch 'working-docker2' into feat/dff_services-front
mxerf Jun 18, 2024
153693b
feat: Add endpoint for linting a python service
Ramimashkouk Jun 19, 2024
503b645
feature: add reset chat btn
mxerf Jun 19, 2024
a4b33b0
feature: add dff index autocomplete
mxerf Jun 19, 2024
1861b08
feature: add manager mode
mxerf Jun 19, 2024
8212fe9
fix: unselect main text
mxerf Jun 19, 2024
ca14df7
fix: don't initiate deletion when modal is open
mxerf Jun 21, 2024
c6266cd
fix: fix in chat node deletion
mxerf Jun 21, 2024
be64bd9
Merge branch 'feat/lint-services' into feat/lint-service-front
mxerf Jun 21, 2024
c31e8d7
fix: Copy imports with their aliases
Ramimashkouk Jun 25, 2024
bb42c51
Merge branch 'dev' into feat/lint-services
Ramimashkouk Jun 25, 2024
6df74e4
Merge branch 'feat/lint-services' into feat/lint-service-front
mxerf Jun 25, 2024
c94308b
lint ready
mxerf Jun 25, 2024
219b805
fix: change editor first line color
mxerf Jun 26, 2024
fb7138f
update: delete save blocking when action is not valid
mxerf Jun 26, 2024
0b70897
Merge branch 'dev' into feat/lint-services
Ramimashkouk Jul 1, 2024
2603e5f
Merge branch 'feat/lint-services' into feat/lint-service-front
Ramimashkouk Jul 1, 2024
153db53
chore: Add `.ttf` to acceptable suffices
Ramimashkouk Jul 1, 2024
12fa877
Merge branch 'dev' into feat/lint-service-front
Ramimashkouk Jul 2, 2024
b6a9a92
fix: fix url bug
mxerf Jul 2, 2024
5014e74
fix: fix unused validate_action var bug
mxerf Jul 2, 2024
d63cc25
refactor: Post snippet to lint using body request
Ramimashkouk Jul 2, 2024
dc84d6c
update: change lint request type
mxerf Jul 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions backend/df_designer/app/api/api_v1/endpoints/dff_services.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
from app.clients.dff_client import get_dff_conditions
from app.core.config import settings
from app.core.logger_config import get_logger
from app.schemas.code_snippet import CodeSnippet
from app.services.index import Index
from app.utils.ast_utils import get_imports_from_file

Expand All @@ -30,18 +31,18 @@ async def search_service(service_name: str, index: Index = Depends(get_index)) -


@router.post("/lint_snippet", status_code=200)
async def lint_snippet(snippet: str) -> dict[str, str]:
async def lint_snippet(snippet: CodeSnippet) -> dict[str, str]:
"""Lints a snippet with Pylint.

This endpoint Joins the snippet with all imports existing in the conditions.py file and then runs Pylint on it.
"""
snippet = snippet.replace(r"\n", "\n")
code_snippet = snippet.code.replace(r"\n", "\n")

imports = get_imports_from_file(settings.snippet2lint_path.parent / "conditions.py")
snippet = "\n\n".join([imports, snippet])
code_snippet = "\n\n".join([imports, code_snippet])

async with aiofiles.open(settings.snippet2lint_path, "wt", encoding="UTF-8") as file:
await file.write(snippet)
await file.write(code_snippet)

pylint_output = StringIO()
reporter = TextReporter(pylint_output)
Expand Down
2 changes: 1 addition & 1 deletion backend/df_designer/app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ async def route_static_file(path: str):
if not settings.start_page.exists():
return HTMLResponse(content="frontend is not built")
file_path = settings.static_files / path.split("/")[-1]
if file_path.suffix in (".js", ".css", ".html"):
if file_path.suffix in (".js", ".css", ".html", ".ttf"):
return FileResponse(file_path)
return FileResponse(settings.static_files / "index.html")

Expand Down
5 changes: 5 additions & 0 deletions backend/df_designer/app/schemas/code_snippet.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from pydantic import BaseModel


class CodeSnippet(BaseModel):
code: str
2 changes: 1 addition & 1 deletion frontend/src/api/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import axios from "axios";
import { DEV, VITE_BASE_API_URL } from "../env.consts";

// const baseURL = VITE_BASE_API_URL ?? "http://localhost:8000/api/v1"
const baseURL = DEV ? VITE_BASE_API_URL : window.location.href.replace(window.location.pathname, '/api/v1').replace('localhost', '0.0.0.0')
const baseURL = DEV ? VITE_BASE_API_URL : window.location.href.replace(window.location.pathname, '/api/v1')

export const $v1 = axios.create({
baseURL: baseURL
Expand Down
9 changes: 6 additions & 3 deletions frontend/src/api/services.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
import { $v1 } from '.';

import { $v1 } from "."

export const get_condition_methods = async () => {
return (await $v1.get("/services/get_conditions")).data
}
}

export const lint_service = async (text: string) => {
return (await $v1.post(`/services/lint_snippet`, { code: text })).data
}
189 changes: 178 additions & 11 deletions frontend/src/modals/ConditionModal/ConditionModal.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,15 @@ import {
ModalHeader,
ModalProps,
Tab,
Tabs
Tabs,
} from "@nextui-org/react"
import classNames from "classnames"
import { HelpCircle, TrashIcon } from "lucide-react"
import { useContext, useEffect, useMemo, useState } from "react"
import toast from "react-hot-toast"
import { useParams } from "react-router-dom"
import { useReactFlow } from "reactflow"
import { lint_service } from "../../api/services"
import ModalComponent from "../../components/ModalComponent"
import { flowContext } from "../../contexts/flowContext"
import { conditionType, conditionTypeType } from "../../types/ConditionTypes"
Expand All @@ -33,6 +35,16 @@ type ConditionModalProps = {

type ConditionModalTab = "Using LLM" | "Slot filling" | "Button" | "Python code" | "Custom"

type LintStatusType = {
status: "ok" | "error"
message: string
}

export type ValidateErrorType = {
status: boolean
reason: string
}

const ConditionModal = ({
data,
condition,
Expand All @@ -42,6 +54,8 @@ const ConditionModal = ({
size = "3xl",
}: ConditionModalProps) => {
const [selected, setSelected] = useState<conditionTypeType>(condition?.type ?? "python")
const [lintStatus, setLintStatus] = useState<LintStatusType | null>(null)
const [testConditionPending, setTestConditionPending] = useState(false)
const setSelectedHandler = (key: conditionTypeType) => {
setCurrentCondition({ ...currentCondition, type: key })
setSelected(key)
Expand All @@ -57,8 +71,87 @@ const ConditionModal = ({

const validateConditionName = (is_create: boolean) => {
const nodes = getNodes() as NodeType[]
if (!is_create) return !nodes.some((node: NodeType) => node.data.conditions?.some(c => (c.name === currentCondition.name && c.id !== currentCondition.id)))
return !nodes.some((node: NodeType) => node.data.conditions?.some(c => c.name === currentCondition.name))
if (!is_create) {
const is_name_valid = !nodes.some((node: NodeType) =>
node.data.conditions?.some(
(c) => c.name === currentCondition.name && c.id !== currentCondition.id
)
)
if (!is_name_valid) {
return {
status: false,
reason: "Name must be unique",
}
} else {
return {
status: true,
reason: "",
}
}
} else {
const is_name_valid = !nodes.some((node: NodeType) =>
node.data.conditions?.some((c) => c.name === currentCondition.name)
)
if (!is_name_valid) {
return {
status: false,
reason: "Name must be unique",
}
} else {
return {
status: true,
reason: "",
}
}
}
}

const validateConditionAction = () => {
const reasons: string[] = []
if (currentCondition.type === "python" && currentCondition.data.python?.action) {
if (
currentCondition.data.python?.action.includes("return") &&
currentCondition.data.python?.action.includes("def") &&
currentCondition.data.python?.action.includes("(ctx: Context, pipeline: Pipeline) -> bool:")
) {
return {
status: true,
reason: "",
}
} else {
if (!currentCondition.data.python?.action.includes("return")) {
reasons.push("Missing return statement")
}
if (!currentCondition.data.python?.action.includes("def")) {
reasons.push("Missing def statement")
}
if (
!currentCondition.data.python?.action.includes(
"(ctx: Context, pipeline: Pipeline) -> bool:"
)
) {
reasons.push("Missing condition statement")
}
return {
status: false,
reason: reasons.join("\n "),
}
}
} else if (currentCondition.type === "python" && !currentCondition.data.python?.action) {
return {
status: false,
reason: "Missing action",
}
} else if (currentCondition.type !== "python") {
return {
status: true,
reason: "",
}
}
return {
status: false,
reason: "Validation error",
}
}

const tabItems: {
Expand Down Expand Up @@ -119,11 +212,57 @@ const ConditionModal = ({
// console.log(currentCondition)
// }, [currentCondition])

const lintCondition = async () => {
setLintStatus(null)
if (currentCondition.type === "python") {
try {
const res = await lint_service(currentCondition.data.python?.action ?? "")
console.log(res)
setLintStatus(res)
return res
} catch (error) {
console.log(error)
}
} else {
return true
}
}

const testCondition = async () => {
setTestConditionPending(() => true)
if (currentCondition.type === "python") {
const lint = await lintCondition()
const validate_action = validateConditionAction()
console.log(lint)
if (lint && validate_action.status) {
setTestConditionPending(() => false)
return true
} else {
if (!validate_action.status) {
setLintStatus(() => ({
status: "error",
message: validate_action.reason,
}))
}
setTestConditionPending(() => false)
return false
}
} else {
setTestConditionPending(() => false)
return true
}
}

useEffect(() => {
setLintStatus(() => null)
}, [selected])

const saveCondition = () => {
const nodes = getNodes()
const node = getNode(data.id)
const currentFlow = flows.find((flow) => flow.name === flowId)
if (validateConditionName(is_create)) {
const validate_name: ValidateErrorType = validateConditionName(is_create)
if (validate_name.status) {
if (node && currentFlow) {
const new_node = {
...node,
Expand All @@ -138,16 +277,18 @@ const ConditionModal = ({
}
const new_nodes = nodes.map((node) => (node.id === data.id ? new_node : node))
setNodes(() => new_nodes)
currentFlow.data.nodes = nodes.map((node) => (node.id === data.id ? new_node : node))
updateFlow(currentFlow)
// currentFlow.data.nodes = nodes.map((node) => (node.id === data.id ? new_node : node))
// updateFlow(currentFlow)
quietSaveFlows()
}
onClose()
setCurrentCondition(
is_create || !condition ? generateNewConditionBase(data.name) : currentCondition
)
} else {
toast.error("Condition name already exists!")
if (!validate_name.status) {
toast.error(`Condition name is not valid: \n ${validate_name.reason}`)
}
}
}

Expand All @@ -165,8 +306,8 @@ const ConditionModal = ({
}
const new_nodes = nodes.map((node) => (node.id === data.id ? new_node : node))
setNodes(() => new_nodes)
currentFlow.data.nodes = nodes.map((node) => (node.id === data.id ? new_node : node))
updateFlow(currentFlow)
// currentFlow.data.nodes = nodes.map((node) => (node.id === data.id ? new_node : node))
// updateFlow(currentFlow)
quietSaveFlows()
}
onClose()
Expand Down Expand Up @@ -214,7 +355,26 @@ const ConditionModal = ({
onChange={(e) => setCurrentCondition({ ...currentCondition, name: e.target.value })}
/>
</div>
<div>{bodyItems[selected]}</div>
<div>
{bodyItems[selected]}
{selected === "python" && (
<div
className='grid transition-all duration-150 overflow-hidden'
style={{
gridTemplateRows: lintStatus ? "1fr" : "0fr",
}}>
<div className='min-h-0 transition-all duration-150'>
<p
className={classNames(
"text-xs p-2 mt-2 rounded-lg font-mono",
lintStatus?.status == "error" ? "bg-red-200" : "bg-green-200"
)}>
{lintStatus?.status == "ok" ? "Condition test passed!" : lintStatus?.message}
</p>
</div>
</div>
)}
</div>
</ModalBody>
<ModalFooter className='flex justify-between items-center'>
<div className='flex items-center justify-start gap-2'>
Expand All @@ -230,7 +390,14 @@ const ConditionModal = ({
<TrashIcon />
</Button>
</div>
<div>
<div className='flex items-end gap-2'>
<Button
data-testid='test-condition-button'
onClick={testCondition}
isLoading={testConditionPending}
className=''>
Test condition
</Button>
<Button
data-testid='save-condition-button'
onClick={saveCondition}
Expand Down
2 changes: 1 addition & 1 deletion frontend/src/modals/ConditionModal/editorOptions.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ class FirstLineWidget extends WidgetType {
}

toDOM() {
const html = `<span style="color: #ff8800;">def</span> <span style="color: #0400ff;">${this.name}</span>(<span style="color: #00cc99;">ctx</span>: <span style="color: #00cc99;">Context</span>, <span style="color: #00cc99;">pipeline</span>: <span style="color: #00cc99;">Pipeline</span>)-&gt; <span style="color: #00cc99;">bool</span>:`;
const html = `<span style="color: #999;">def</span> <span style="color: #999;">${this.name}</span>(<span style="color: #999;">ctx</span>: <span style="color: #999;">Context</span>, <span style="color: #999;">pipeline</span>: <span style="color: #999;">Pipeline</span>)-&gt; <span style="color: #999;">bool</span>:`;
const wrap = document.createElement("span");
wrap.innerHTML = html;
return wrap;
Expand Down
Loading