Skip to content

Commit

Permalink
Update llm-benchmark.md
Browse files Browse the repository at this point in the history
  • Loading branch information
vprelovac authored Aug 10, 2024
1 parent 1624919 commit 99f025d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/kagi/ai/llm-benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@ Last updated **July 29, 2024**.
| OpenAI **gpt-3.5-turbo** | 22.00 | 1569 | 0.01552 | 0.51 | 45.03 |
| Reka **reka-edge** | 20.00 | 5377 | 0.00798 | 2.02 | 46.87 |
| Reka **reka-flash** | 16.00 | 5738 | 0.01668 | 3.28 | 28.75 |
| GoogleGenAI **gemini-1.5-pro-exp-0801** | 14.00 | 4942 | 0.26325 | 1.82 | 28.19 |
| GoogleGenAI **gemini-1.5-flash** | 14.00 | 5287 | 0.02777 | 3.02 | 21.16 |
| GoogleGenAI **gemini-1.5-pro** | 12.00 | 5284 | 0.27762 | 3.32 | 16.49 |


The table includes metrics such as overall mode quality (measured as percent of correct responses), total tokens output (some models are less verbose by default, affecting both cost and speed), total cost to run the test, median response latency and average speed in tokens per second at the time of testing.
Expand Down

0 comments on commit 99f025d

Please sign in to comment.