-
Notifications
You must be signed in to change notification settings - Fork 105
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adds comet-ml plugin example #1702
base: master
Are you sure you want to change the base?
Adds comet-ml plugin example #1702
Conversation
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
"comet-ml==3.43.2", | ||
"lightning==2.3.0", | ||
"flytekitplugins-comet-ml", | ||
"unionai==0.1.43", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't need the unionai
dependency, right?
Should we add this in Union's doc instead of Open Source's doc?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was a mistake. unionai
is not needed.
|
||
REGISTRY = "localhost:30000" | ||
image = ImageSpec( | ||
name="unionai", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same as below, we should use ghcr.io/flyteorg
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For ImageSpec
, most users will not have push access to ghcr.io/flyteorg
, so they will need to change the registry to run it.
Using localhost:30000
means I am optimizing for the sandbox use case. An alternative is to use os.getenv("REGISTRY", "localhost:30000")
, and let it be an environment variable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No problem, I like your idea
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
||
REGISTRY = "localhost:30000" | ||
image = ImageSpec( | ||
name="unionai", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No problem, I like your idea
# dynamic-log-links: | ||
# - comet-ml-execution-id: | ||
# displayName: Comet | ||
# templateUris: {{ .taskConfig.host }}/{{ .taskConfig.workspace }}/{{ .taskConfig.project_name }}/{{ .executionName }}{{ .nodeId }}{{ .taskRetryAttempt }}{{ .taskConfig.link_suffix }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should add double quote here.
# templateUris: {{ .taskConfig.host }}/{{ .taskConfig.workspace }}/{{ .taskConfig.project_name }}/{{ .executionName }}{{ .nodeId }}{{ .taskRetryAttempt }}{{ .taskConfig.link_suffix }} | ||
# - comet-ml-custom-id: | ||
# displayName: Comet | ||
# templateUris: {{ .taskConfig.host }}/{{ .taskConfig.workspace }}/{{ .taskConfig.project_name }}/{{ .taskConfig.experiment_key }} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same thing
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left a few suggestions, otherwise LGTM
examples/comet_ml_plugin/README.md
Outdated
.. tags:: Integration, Data, Metrics, Intermediate | ||
``` | ||
|
||
Comet’s machine learning platform integrates with your existing infrastructure and tools so you can manage, visualize, and optimize models—from training runs to production monitoring. This plugin integrates Flyte with Comet by configuring links between the two platforms. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Comet’s machine learning platform integrates with your existing infrastructure and tools so you can manage, visualize, and optimize models—from training runs to production monitoring. This plugin integrates Flyte with Comet by configuring links between the two platforms. | |
Comet’s machine learning platform integrates with your existing infrastructure and tools so you can manage, visualize, and optimize models from training runs to production monitoring. This plugin integrates Flyte with Comet by configuring links between the two platforms. |
# | ||
# # Comet Example | ||
# Comet’s machine learning platform integrates with your existing infrastructure and | ||
# tools so you can manage, visualize, and optimize models—from training runs to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# tools so you can manage, visualize, and optimize models—from training runs to | |
# tools so you can manage, visualize, and optimize models from training runs to |
|
||
# %% [markdown] | ||
# First, we specify the project and workspace that we will use with Comet's platform | ||
# Please update `PROJECT_NAME` and `WORKSPACE` to the value associated with your account. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# Please update `PROJECT_NAME` and `WORKSPACE` to the value associated with your account. | |
# Please update `PROJECT_NAME` and `WORKSPACE` to the values associated with your account. |
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
@neverett The comet-ml check is passing now: https://github.com/flyteorg/flytesnacks/actions/runs/10182511543/job/28165251450?pr=1702 There are a bunch other tests that are failing tho. |
@thomasjpfan can you try merging |
…-ml-init Signed-off-by: Thomas J. Fan <[email protected]>
…-ml-init Signed-off-by: Thomas J. Fan <[email protected]>
Signed-off-by: Thomas J. Fan <[email protected]>
…-ml-init Signed-off-by: Thomas J. Fan <[email protected]>
Running the example given in this PR was not working for me with ImageSpec. I had to add the following command to create a home dir & assign permissions for the flytekit user -> image = ImageSpec(
name="comet-ml",
packages=[
"torch==2.3.1",
"comet-ml==3.43.2",
"lightning==2.3.0",
"flytekitplugins-comet-ml",
"torchvision",
],
commands=["mkdir -p /home/flytekit", "chown -R flytekit /home/flytekit"],
registry=REGISTRY,
) Anyone experiencing the same? |
Signed-off-by: Thomas J. Fan <[email protected]>
For this example to work, it needs |
This PR adds a comet-ml example.
To fully run the example, you need to use this
ImageSpec
, because the plugin is in a PR:flyteorg/flytekit#2550