From 8e235c9479b9b4e1be405519b6bcf1d9cd0c7d26 Mon Sep 17 00:00:00 2001 From: Jerry Liu Date: Fri, 8 Mar 2024 19:56:16 -0800 Subject: [PATCH] add claude3 benchmarks (#11685) * cr * lint --------- Co-authored-by: Andrei Fajardo --- docs/module_guides/models/llms.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/module_guides/models/llms.md b/docs/module_guides/models/llms.md index 366584f8e0cbac..2fc4c5f7ad68d3 100644 --- a/docs/module_guides/models/llms.md +++ b/docs/module_guides/models/llms.md @@ -94,6 +94,8 @@ If you have ways to improve the setup for existing notebooks, contributions to c | [gpt-3.5-turbo](https://colab.research.google.com/drive/1vvdcf7VYNQA67NOxBHCyQvgb2Pu7iY_5?usp=sharing) (openai) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | | [gpt-3.5-turbo-instruct](https://colab.research.google.com/drive/1Ne-VmMNYGOKUeECvkjurdKqMDpfqJQHE?usp=sharing) (openai) | ✅ | ✅ | ✅ | ✅ | ✅ | ⚠️ | Tool usage in data-agents seems flakey. | | [gpt-4](https://colab.research.google.com/drive/1QUNyCVt8q5G32XHNztGw4YJ2EmEkeUe8?usp=sharing) (openai) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | | +| [claude-3 opus](https://colab.research.google.com/drive/1xeFgAmSLpY_9w7bcGPvIcE8UuFSI3xjF?usp=sharing) | ✅ | ⚠️ | ✅ | ✅ | ✅ | ✅ | | +| [claude-3 sonnet](https://colab.research.google.com/drive/1xeFgAmSLpY_9w7bcGPvIcE8UuFSI3xjF?usp=sharing) | ✅ | ✅ | ✅ | ✅ | ✅ | ⚠️ | Prone to hallucinating tool inputs. | | [claude-2](https://colab.research.google.com/drive/1IuHRN67MYOaLx2_AgJ9gWVtlK7bIvS1f?usp=sharing) (anthropic) | ✅ | ✅ | ✅ | ✅ | ✅ | ⚠️ | Prone to hallucinating tool inputs. | | [claude-instant-1.2](https://colab.research.google.com/drive/1ahq-2kXwCVCA_3xyC5UMWHyfAcjoG8Gp?usp=sharing) (anthropic) | ✅ | ✅ | ✅ | ✅ | ✅ | ⚠️ | Prone to hallucinating tool inputs. |