-
Notifications
You must be signed in to change notification settings - Fork 68
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'master' into griffin/feedback-replace-ts
- Loading branch information
Showing
122 changed files
with
1,155 additions
and
73 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+36.1 KB
(110%)
docs/docs/guides/integrations/imgs/dspy/dspy_weave_model_serve.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+32.5 KB
(110%)
docs/docs/guides/integrations/imgs/dspy/dspy_weave_model_v1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+37.2 KB
(110%)
docs/docs/guides/integrations/imgs/dspy/dspy_weave_model_v2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+42.5 KB
(110%)
docs/docs/guides/integrations/imgs/dspy/dspy_with_weave_op.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+46.1 KB
(110%)
docs/docs/guides/integrations/imgs/dspy/dspy_without_weave_op.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+38.8 KB
(110%)
docs/docs/guides/integrations/imgs/groq/groq_weave_dasboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+40.3 KB
(110%)
docs/docs/guides/integrations/imgs/groq/groq_weave_model.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+30.1 KB
(110%)
docs/docs/guides/integrations/imgs/groq/groq_weave_model_version.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+40.1 KB
(110%)
docs/docs/guides/integrations/imgs/groq/groq_weave_tracing.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+2.74 MB
(1500%)
docs/docs/guides/integrations/imgs/instructor/instructor_lm_trace.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+27.1 KB
(110%)
docs/docs/guides/integrations/imgs/instructor/instructor_op_trace.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+25.3 KB
(110%)
docs/docs/guides/integrations/imgs/instructor/instructor_serve.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified
BIN
+14.4 KB
(110%)
docs/docs/guides/integrations/imgs/instructor/instructor_weave_model.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Binary file modified
BIN
+40.4 KB
(110%)
docs/docs/guides/integrations/imgs/llamaindex_evaluation.png
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Binary file modified
BIN
+33.2 KB
(110%)
docs/docs/guides/integrations/imgs/notdiamond/evaluations.png
Oops, something went wrong.
Binary file modified
BIN
+21.9 KB
(110%)
docs/docs/guides/integrations/imgs/notdiamond/router-preferences.png
Oops, something went wrong.
Binary file modified
BIN
+25 KB
(110%)
docs/docs/guides/integrations/imgs/notdiamond/weave-trace.png
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
from weave.flow.annotation_spec import AnnotationSpec | ||
from weave.trace_server.trace_server_interface import ObjQueryReq | ||
|
||
|
||
def test_human_feedback_basic(client): | ||
# create a human feedback spec | ||
|
||
col1 = AnnotationSpec( | ||
name="Numerical field #1", | ||
description="A numerical field with a range of -1 to 1", | ||
json_schema={ | ||
"type": "number", | ||
"min": -1, | ||
"max": 1, | ||
}, | ||
unique_among_creators=True, | ||
op_scope=None, | ||
) | ||
ref1 = client.save(col1, "my numerical spec") | ||
assert ref1 | ||
|
||
col2 = AnnotationSpec( | ||
name="Text field #1", | ||
json_schema={"type": "string", "max_length": 100}, | ||
op_scope=["weave:///entity/project/op/name:digest"], | ||
) | ||
ref2 = client.save(col2, "my text spec") | ||
assert ref2 | ||
|
||
# query it by object type | ||
objects = client.server.objs_query( | ||
ObjQueryReq.model_validate( | ||
{ | ||
"project_id": client._project_id(), | ||
"filter": {"base_object_classes": ["AnnotationSpec"]}, | ||
} | ||
) | ||
) | ||
|
||
assert len(objects.objs) == 2 | ||
assert objects.objs[0].val["name"] == "Numerical field #1" | ||
assert objects.objs[1].val["name"] == "Text field #1" | ||
assert ( | ||
objects.objs[0].val["description"] | ||
== "A numerical field with a range of -1 to 1" | ||
) | ||
assert not objects.objs[1].val["description"] | ||
assert not objects.objs[0].val["op_scope"] | ||
assert objects.objs[1].val["op_scope"] == ["weave:///entity/project/op/name:digest"] | ||
assert objects.objs[0].val["json_schema"] == { | ||
"type": "number", | ||
"min": -1, | ||
"max": 1, | ||
} | ||
assert objects.objs[1].val["json_schema"] == { | ||
"type": "string", | ||
"max_length": 100, | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
from PIL import Image | ||
|
||
from weave.trace.custom_objs import decode_custom_obj, encode_custom_obj | ||
|
||
|
||
def test_decode_custom_obj_known_type(client): | ||
img = Image.new("RGB", (100, 100)) | ||
encoded = encode_custom_obj(img) | ||
|
||
# Even though something is wrong with the deserializer op, we can still decode | ||
decoded = decode_custom_obj( | ||
encoded["weave_type"], encoded["files"], "weave:///totally/invalid/uri" | ||
) | ||
|
||
assert isinstance(decoded, Image.Image) | ||
assert decoded.tobytes() == img.tobytes() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
47 changes: 47 additions & 0 deletions
47
...ents/PagePanelComponents/Home/Browse3/pages/PlaygroundPage/PlaygroundChat/LLMDropdown.tsx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
import {Box} from '@mui/material'; | ||
import {Select} from '@wandb/weave/components/Form/Select'; | ||
import React from 'react'; | ||
|
||
import {LLM_MAX_TOKENS} from '../llmMaxTokens'; | ||
|
||
interface LLMDropdownProps { | ||
value: string; | ||
onChange: (value: string, maxTokens: number) => void; | ||
} | ||
|
||
export const LLMDropdown: React.FC<LLMDropdownProps> = ({value, onChange}) => { | ||
const options = Object.keys(LLM_MAX_TOKENS).map(llm => ({ | ||
value: llm, | ||
label: llm, | ||
})); | ||
|
||
return ( | ||
<Box | ||
sx={{ | ||
width: '150px', | ||
'& #react-select-2-listbox': { | ||
width: '300px', | ||
maxHeight: '500px', | ||
}, | ||
'& #react-select-2-listbox > div': { | ||
maxHeight: '500px', | ||
}, | ||
}}> | ||
<Select | ||
value={options.find(opt => opt.value === value)} | ||
onChange={option => { | ||
if (option) { | ||
const maxTokens = | ||
LLM_MAX_TOKENS[ | ||
(option as {value: string}).value as keyof typeof LLM_MAX_TOKENS | ||
]?.max_tokens || 0; | ||
onChange((option as {value: string}).value, maxTokens); | ||
} | ||
}} | ||
options={options} | ||
size="medium" | ||
isSearchable | ||
/> | ||
</Box> | ||
); | ||
}; |
113 changes: 113 additions & 0 deletions
113
...s/PagePanelComponents/Home/Browse3/pages/PlaygroundPage/PlaygroundChat/PlaygroundChat.tsx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,113 @@ | ||
import {Box, CircularProgress, Divider} from '@mui/material'; | ||
import {MOON_200} from '@wandb/weave/common/css/color.styles'; | ||
import React, {SetStateAction, useState} from 'react'; | ||
|
||
import {PlaygroundState, PlaygroundStateKey} from '../types'; | ||
import {PlaygroundChatTopBar} from './PlaygroundChatTopBar'; | ||
|
||
export type PlaygroundChatProps = { | ||
entity: string; | ||
project: string; | ||
setPlaygroundStates: (states: PlaygroundState[]) => void; | ||
playgroundStates: PlaygroundState[]; | ||
setPlaygroundStateField: ( | ||
index: number, | ||
field: PlaygroundStateKey, | ||
value: SetStateAction<PlaygroundState[PlaygroundStateKey]> | ||
) => void; | ||
setSettingsTab: (callIndex: number | null) => void; | ||
settingsTab: number | null; | ||
}; | ||
|
||
export const PlaygroundChat = ({ | ||
entity, | ||
project, | ||
setPlaygroundStates, | ||
playgroundStates, | ||
setPlaygroundStateField, | ||
setSettingsTab, | ||
settingsTab, | ||
}: PlaygroundChatProps) => { | ||
// eslint-disable-next-line @typescript-eslint/no-unused-vars | ||
const [isLoading, setIsLoading] = useState(false); | ||
const chatPercentWidth = 100 / playgroundStates.length; | ||
|
||
return ( | ||
<Box | ||
sx={{ | ||
width: '100%', | ||
height: '100%', | ||
display: 'flex', | ||
flexDirection: 'column', | ||
alignItems: 'center', | ||
}}> | ||
<Box | ||
sx={{ | ||
width: '100%', | ||
height: '100%', | ||
maxHeight: 'calc(100% - 130px)', | ||
display: 'flex', | ||
position: 'relative', | ||
}}> | ||
{isLoading && ( | ||
<Box | ||
sx={{ | ||
position: 'absolute', | ||
top: 0, | ||
left: 0, | ||
right: 0, | ||
bottom: 0, | ||
backgroundColor: 'rgba(255, 255, 255, 0.7)', | ||
zIndex: 100, | ||
display: 'flex', | ||
alignItems: 'center', | ||
justifyContent: 'center', | ||
}}> | ||
<CircularProgress /> | ||
</Box> | ||
)} | ||
{playgroundStates.map((state, idx) => ( | ||
<React.Fragment key={idx}> | ||
{idx > 0 && ( | ||
<Divider | ||
orientation="vertical" | ||
flexItem | ||
sx={{ | ||
height: '100%', | ||
borderRight: `1px solid ${MOON_200}`, | ||
}} | ||
/> | ||
)} | ||
<Box | ||
sx={{ | ||
width: '100%', | ||
height: '100%', | ||
display: 'flex', | ||
flexDirection: 'column', | ||
}}> | ||
<Box | ||
sx={{ | ||
position: 'absolute', | ||
top: '8px', | ||
left: `calc(${idx * chatPercentWidth}% + 8px)`, | ||
width: `calc(${chatPercentWidth}% - 16px)`, | ||
zIndex: 10, | ||
}}> | ||
<PlaygroundChatTopBar | ||
idx={idx} | ||
settingsTab={settingsTab} | ||
setSettingsTab={setSettingsTab} | ||
setPlaygroundStateField={setPlaygroundStateField} | ||
setPlaygroundStates={setPlaygroundStates} | ||
playgroundStates={playgroundStates} | ||
entity={entity} | ||
project={project} | ||
/> | ||
</Box> | ||
</Box> | ||
</React.Fragment> | ||
))} | ||
</Box> | ||
</Box> | ||
); | ||
}; |
Oops, something went wrong.