Whisper is recognising Malayalam and outputing Tamil script (Bug)

[skillhacker-code]

Whisper is recognizing Malayalam Language but transcribing as tamil language

if anyone can look into it ,it would be awesome .below is a link to a malayalam video

[https://www.youtube.com/watch?v=uuMiwU5w_kE&pp=ygUZc2FudGhvc2ggZ2VvcmdlIGt1bGFuZ2FyYQ%3D%3D](Malayalam video)

malayalam look like - സന്തോഷവും സമാധാനവും
Tamil look like - அமைதி மற்றும் மகிழ்ச்சி

It would be great if whisper team can solve this issue .

[Jiltseb]

@skillhacker-code Whisper is trained only on negligible (0.5 hours) Malayalam audio which causes the issue with transcription, as also shown in the benchmarking results in the whisper paper. It is possible to achieve good results by fine-tuning the whisper model on Malayalam training data.

[skillhacker-code]

It would have been great if Malayalam was already trained . I tried translation whisperai is able to translate from malayalam audio to English . it is not perfect but it is working .

I read the docs and found out that Malayalam language is not included because it has error rate of more than 50% .

Thank you Jiltseb for taking time to replying .

[nuhman]

It would have been great if Malayalam was already trained . I tried translation whisperai is able to translate from malayalam audio to English . it is not perfect but it is working .

I read the docs and found out that Malayalam language is not included because it has error rate of more than 50% .

Thank you Jiltseb for taking time to replying .

Did you find a workaround?