update quick start notebook #51
Conversation
use the new 5 min agent demo notebook with synthetic data generation API for the quick start demo
quick_start_demo/5-min-Mosaic-AI-Agents-Demo-with-Synthetic-Eval.py
Outdated
Show resolved
Hide resolved
quick_start_demo/5-min-Mosaic-AI-Agents-Demo-with-Synthetic-Eval.py
Outdated
Show resolved
Hide resolved
| # MAGIC | ||
| # MAGIC 2. Click Run all | ||
| # MAGIC | ||
| # MAGIC  |
There was a problem hiding this comment.
@peter lu has new gifs for all the gifs throughout the notebook - can we use those?
| # MAGIC import json | ||
| # MAGIC | ||
| # MAGIC DEFAULT_CONFIG = { | ||
| # MAGIC 'endpoint_name': "agents-demo-gpt4o", |
There was a problem hiding this comment.
let's change this to FMAPI llama 70b
There was a problem hiding this comment.
we need to re-think about the LLM endpoint choices in this notebook.
I have changed the default config in fc_agent.py and also the dev_config of the vibe check section. However, the baseline config still has "endpoint_name": "agents-demo-gpt4o""agents-demo-gpt4o".
More importantly, it makes me realize that the notebook requires the user have 2 external model endpoints (agents-demo-gpt4o and agents-demo-gpt4o-mini) to succeed the "Run All". That should be mentioned in the requirements and 0/setup. I think it is better to remove the external model dependencies, however, all the screenshots show gpt models.
| # MAGIC | ||
| # MAGIC This UIs, coupled with the speed of evaluation, help you efficiently test your hypotheses to improve quality, letting you reach the production quality bar in less time. | ||
| # MAGIC | ||
| # MAGIC  |
There was a problem hiding this comment.
need to update the gif if dev config uses llama-70b
|
|
||
| # Now we call the helper function to run evaluation. | ||
| # The configuration keys must match those defined in `fc_agent.py` | ||
| model_info_gpt4o, eval_results = log_and_evaluate_agent( |
There was a problem hiding this comment.
requires a deployed gpt-4o endpoint with name agents-demo-gpt4o in the workspace. do we want to use workspace foundation models like llama 3.1 405b or DBRX
will need to update variable name, endpoint name, and run name if the endpoint config changes
|
|
||
| # COMMAND ---------- | ||
|
|
||
| baseline_config = { |
There was a problem hiding this comment.
same comments here, need to decide whether or not to remove external models in the notebook
Related Issues/PRs
none
What changes are proposed in this pull request?
use the new 5 min agent demo notebook with synthetic data generation API for the quick start demo
How is this PR tested?
successful run here