Skip to content

Commit

Permalink
📝 adds hint to performance
Browse files Browse the repository at this point in the history
  • Loading branch information
chriamue committed Jan 11, 2024
1 parent e25c18a commit e90f976
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,13 @@ The following table shows the performance metrics of the model on different syst
| phi-2 | AMD 5600G (8 Core VM) 16GB | 5.3 tokens/s |
| phi-2 | Apple M2 (10 Core) 16GB | 24.0 tokens/s |

### Hint

The performance of the model is highly dependent on the memory bandwidth of the system.
While getting 20.6 tokens/s for the Phi-2 Model on a AMD 7900X3D with 64GB of DDR5-4800 memory,
the performance could be increased to
21.8 tokens/s by overclocking the memory to DDR5-5600.

## Todo

- [x] implement api for https://huggingface.github.io/text-generation-inference/#/
Expand Down

0 comments on commit e90f976

Please sign in to comment.