From 7b699b09b92b941f1b199a70e253c2c5df73f719 Mon Sep 17 00:00:00 2001 From: Holden X Date: Sat, 16 Dec 2023 16:05:58 +0800 Subject: [PATCH] Update demo video in README.md --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index b0da92f..b3db150 100644 --- a/README.md +++ b/README.md @@ -3,9 +3,9 @@ *Demo* 🔥 -https://github.com/hodlen/PowerInfer/assets/34213478/b782ccc8-0a2a-42b6-a6aa-07b2224a66f7 +https://github.com/SJTU-IPADS/PowerInfer/assets/34213478/d26ae05b-d0cf-40b6-8788-bda3fe447e28 -The demo is running with a single 24G 4090 GPU, the model is Falcon (ReLU)-40B, and the precision is FP16. +PowerInfer v.s. llama.cpp on a single RTX 4090(24G) running Falcon(ReLU)-40B-FP16 with a 11x speedup! --- ## Abstract