Fast conformer model is ~2.4X faster compare to Conformer model. Will I get ~2.4x speed up when I convert it to Riva Model?

[mehadi92]

I have created a conformer model and fastconformer model during training time I have got ~2X speed up for the fastconformer model. But When I convert the model to Riva and take inference I get similar inference time for conformed and fastconformer models.

Do you have any clue about it?

Thanks

[mehadi92]

After upgrading the onnx runtime version I get the performance improvement

[MinhNghia100402]

can you tell me how to convert Fast conformer model to riva, i am trying to convert it but it doesn't work, it only works with CTC conformer

[mehadi92]

@MinhNghia100402 can you share the Riva pipeline configuration that you use for it?

[MinhNghia100402]

I am using nemo model (https://huggingface.co/reazon-research/reazonspeech-nemo-v2) I want to convert this model to riva to use riva-server. I have converted and used fine-tuning model trained by ctc-conformer. The purpose I want to convert to riva model is to be able to use word boosting

[mehadi92]

@MinhNghia100402 it seems it's a RNN-T loss based model and as far as I know RNN-T loss based model does not support in Riva yet

[MinhNghia100402]

@mehadi92 Thank you!