Skip to content

Commit

Permalink
Update setfit-onnx-optimum.ipynb
Browse files Browse the repository at this point in the history
  • Loading branch information
danielkorat committed Feb 22, 2024
1 parent a34289b commit a11e000
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion notebooks/setfit-onnx-optimum.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -935,7 +935,7 @@
"id": "gvdggvIbvowO"
},
"source": [
"By applying ONNX, we were able to improve the latency from 13.43ms per sample to 2.19ms per sample, for a speedup of 5.41x!\n",
"By applying ONNX, we were able to improve the latency from 8.31ms per sample to 1.54ms per sample, for a speedup of 5.41x!\n",
"\n",
"For further improvements, we recommend increasing the inference batch size, as this may also heavily improve the throughput. For example, setting the batch size to 128 reduces the latency further down to 0.3ms, and down to 0.2ms at a batch size of 2048."
]
Expand Down

0 comments on commit a11e000

Please sign in to comment.