Using only the realtime model.

[bilalshafim]

I had an issue where the main model does not ever transcribe. The text returned is always 'realtime;, never 'fullSentence' but that is not important to my use case.
For my use case (long sessions without the need for really accurate results) , I can work with the real time model only.

Optionally, if I can limit the real time to process only the last 2-3 minutes of audio, I can remove the lag in real time transcription that starts to come up after the live session is longer than 5 minutes (Transcription returned is in bigger phrases or sentences rather than a couple words at the start of the session. This is because the real time model transcribes from the beginning of the recording up to the most recent one.

This starts at 00:00.00 each time. At this time, I cannot run tests or access previous logs:

RealTimeSTT: faster_whisper - DEBUG - Processing segment at 02:39.200
RealTimeSTT: faster_whisper - DEBUG - Processing segment at 03:08.000
RealTimeSTT: faster_whisper - DEBUG - Processing segment at 03:32.560
RealTimeSTT: faster_whisper - DEBUG - Processing segment at 04:02.160
RealTimeSTT: faster_whisper - DEBUG - Processing segment at 04:26.800
RealTimeSTT: faster_whisper - DEBUG - Processing segment at 04:53.840
RealTimeSTT: faster_whisper - DEBUG - Processing segment at 05:10.480

Does anyone have any idea how I can modify the current implementation to support this use case or will I have to fork faster-whisper and modify that?

[homelab-00]

The maintainer can definitely give you more accurate information, but let me ask you a couple of things first. First of all, which audio_recorder.py are you using? Because if you look at recent issues, you can see that a lot of other people had problems with transcription and they were using the latest release. You should use the audio_recorder.py from the latest commit, just download the whole zip of the project and use the audio_recorder.py that's in the latest commit, not in the latest release because it hasn't been updated yet.

Secondly, how are you using the model? Which script are you using to transcript? Did you write the script yourself or are you using one of the scripts in the test folder? From my understanding of how the project works, it's built only around live transcription. Now, I'm using realtimestt_test.py as an example. It has all the recorder configs and stuff like that. You can build it yourself, of course, and the README has information about what each flag means and how to use it, what are the defaults and all of that stuff.

So in realtimestt_test.py there's a recorder_config and in that, two models are defined: there's a main model, which just says model, and there's a real time model. They're both for live transcription, but they're for different things. The real-time model is for live preview and the main model is for the actual final transcription. The real-time model is just to display a preview while you speak, and the main model handles the final transcription. For accurate transcription, use the largest model your system can handle for the main model and a smaller one for real-time previews. You can ignore the real-time model if you don't need the preview.

A very clear example of what I'm describing can be seen in the video below. You can see that as soon as the script starts processing audio, the sentences first appear in a bright yellow. After a little bit, they change to either a deeper yellow or cyan. So the bright yellow transcription is coming from the real-time model, which is just a preview. And the deep yellow or cyan transcription is coming from the main model.
See the video in this comment

If you want to make longer recordings where instead of the script transcribing as you're speaking and printing as you're speaking, instead it starts recording and then when it ends recording it then transcribes the whole thing you can use the script realtimestt_test_hotkeys_v2 which does have that ability or you can write it yourself.

---

Edit: For even longer recordings, it might be better to save the recording in a temp audio file and then feed that to whisper (or faster-whisper or distil-whisper or ...) directly. All these models work out of the box with static transcriptions and the challenge it to make them work properly with live transcription.