Releases: Synopsis/whisper_ane
Releases · Synopsis/whisper_ane
v0.1.0-rc1
This release has the small
, base
and tiny
encoders, both stock and optimised variants, at precisions fp16 and fp32.
All benchmarks have been run on a 16 inch M1 Max MBP 2021, 64GB RAM, MacOS 13.0
Key observations from the XCode benchmarks:
- All FP16 exports (base and optimised arch) run on ANE 100%
- All FP32 exports (base and optimised arch) run on GPU 100%
- In FP16, the optimised arch is significantly faster
- In FP32, the base arch is significantly faster
Caveats:
- It's unclear how exactly XCode runs their benchmark. We do not know how the performance scales at larger batch sizes.
- Though the FP32 base model looks the fastest per XCode, we may see a different result when running the optimised ANE model on larger batch sizes as it is 100% ANE accelerated