feat(restapi): add metrics capabilities to RESTAPI #672
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jtsextonMITRE
requested review from
keithmanville and
jkglasbrenner
and removed request for
keithmanville
keithmanville
requested changes
Nov 7, 2024
| run_id: UUID = self._job_id_mlflowrun_service.get(job_id=job_id, **kwargs)[ | ||
| "mlflow_run_id" | ||
| ] | ||
| client = MlflowClient() |
There was a problem hiding this comment.
If it can't connect to the mlflow server it attempts to re-connect several times, so this request can hang for a bit. Not sure if this is the best behavior? Maybe we should set number of retries lower?
|
@jtsextonMITRE I made a couple commits to this branch:
|
9 tasks
This commit adds functionality for managing metrics via the REST API. The metrics are stored in
MLFlow, so this feature uses the MLFlow client in the service layer. Providing metrics management
via the Dioptra REST API is part of a larger effort to remove the need for the user to interact
with MLFlow directly.
It adds the following endpoints for managing metrics:
- GET `/experiments/{id}/metrics` - Gets all of the latest metrics for every job in the experiment
- GET `/jobs/{id}/metrics` - Gets a Job resource's latest metrics
- POST `jobs/{id}/metrics` - Sets a metric for a Job
- GET `/jobs/{id}/metrics/{name}/snapshots` - Gets a Job metric history
This commit also updates the Python client for the new functionality.
keithmanville
approved these changes
Dec 20, 2024
keithmanville
force-pushed
the
jtsexton-restapi-mlflow-metrics
branch
from
4a85084 to
6e74940
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #671.
Additionally:
Fixes some incorrect function descriptions.
Fixes some mypy linting issues in related files.
Adds a restapi test for the /jobs/{id}/mlflowRun endpoint using a fixture added to help the metrics test.