🗺 Experiments Enhancements

[mikeldking]

Add a minimum set of pairwise evaluators that judges which answer is best from two sets of output

Fixes

• [BUG] [experiments] long eval names cut off in experiment run details slideover on experiment compare page #5792

API

• [datasets] DELETE /datasets/:dataset_id #3155
• [datasets] investigate feasibility of using fastapi #3423
• [datasets] PATCH / datasets/:dataset_id #3156
• [datasets] GET /datasets/:id #3044
• [datasets][gql] search param on datasets for typeaheads etc #3229
• [ui][datasets] full text view for datasets #3677
• [experiments] progress bar for evaluators does not complete #3678
• [datasets] flatten the task_output key in the REST API for experiment run output #3795
• [ENHANCEMENT][experiments] return fully hydrated RanExperiment from get_experiment client method #5137

Repetitions

• [experiments][ui] repetitions in the table view

Evaluators

• [experiments] pairwise evaluator #3738

Experiments

• [experiments][gql]baseline support in comparison gql #5682
• [experiments][ui] baseline selector #3737
• [experiments][ui] display example in a slide-over on the compare page. #3739
• [BUG] timeout error for other coroutines if task is synchronous function that takes too long to run #3844
• [experiments] filter on experiment runs #5681
• [ENHANCEMENT] Code Eval pass/fail designation #4256
• [ENHANCEMENT] Custom Columns in Experiments and Graphs Over Time #4254
• [ENHANCEMENT] Traces for an experiment run should include evaluations in the feedback column #4190
• [ENHANCEMENT] Add Milestones to Experiment Graphs #4133
• [ENHANCEMENT] Ability to specify phoenix host for phoenix.experiments.run_experiment() #4228

Datasets

• [datasets] containment search on examples #3729
• [datasets][ui] search datasets by name
• [datasets][ui] search bar for dataset examples
• [experiments] client.get_experiment (for re-running evals) #3766
• [datasets][ui] edit dataset UI #3938
• [ENHANCEMENT] enable curating datasets based on previous experiment runs #5016
• [datasets] enable curating splits from an experiment run #5704

Nice-to-Haves

• [ENHANCEMENT] allow concurrency without coroutines #3847
• [ENHANCEMENT] run faster than 10 iterations per second #3848
• [ENHANCEMENT][UI] for each example in dataset add the ability to see past versions #5710