From 99f025dad77d9a7ccc3368cdea98c8b214f27e60 Mon Sep 17 00:00:00 2001 From: Vladimir Prelovac Date: Sat, 10 Aug 2024 15:06:40 -0700 Subject: [PATCH] Update llm-benchmark.md --- docs/kagi/ai/llm-benchmark.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/kagi/ai/llm-benchmark.md b/docs/kagi/ai/llm-benchmark.md index 656d2674..388a216f 100644 --- a/docs/kagi/ai/llm-benchmark.md +++ b/docs/kagi/ai/llm-benchmark.md @@ -27,8 +27,8 @@ Last updated **July 29, 2024**. | OpenAI **gpt-3.5-turbo** | 22.00 | 1569 | 0.01552 | 0.51 | 45.03 | | Reka **reka-edge** | 20.00 | 5377 | 0.00798 | 2.02 | 46.87 | | Reka **reka-flash** | 16.00 | 5738 | 0.01668 | 3.28 | 28.75 | +| GoogleGenAI **gemini-1.5-pro-exp-0801** | 14.00 | 4942 | 0.26325 | 1.82 | 28.19 | | GoogleGenAI **gemini-1.5-flash** | 14.00 | 5287 | 0.02777 | 3.02 | 21.16 | -| GoogleGenAI **gemini-1.5-pro** | 12.00 | 5284 | 0.27762 | 3.32 | 16.49 | The table includes metrics such as overall mode quality (measured as percent of correct responses), total tokens output (some models are less verbose by default, affecting both cost and speed), total cost to run the test, median response latency and average speed in tokens per second at the time of testing.