Speech to text service for Leolani. This repository is a component of the Leolani framework. For usage of the component within the framework see the instructions there.
This package contains multiple implementations to convert text from spoken language for any written text.
This repository uses Python >= 3.9
Be sure to run in a virtual python environment (e.g. conda, venv, mkvirtualenv, etc.)
-
In the root directory of this repo run
pip install -e .
There are various implementations included in cltl.asr. Depending on which
implementation is used different dependencies are required for this package.
To include those during installation specify the required extra dependencies
with pip, e.g. like:
pip install cltl.asr[whisper]
For the available options refer to setup.py.
This implementation utilizes whisper.cpp, which is a C++ implementation of whisper that can be run natively and provides improved performance over the Python implemenation of Whisper.
In order to use this implementation a whisper.cpp server needs to be setup. To setup the server follow the instructions in the whisper.cpp README and server specific README.
For the server setup on OS X on an Apple Silicon device refer to this setup script.
For using this repository as a package in a different project and on a different virtual environment, you may
-
install a published version from PyPI:
pip install cltl.asr
-
or, for the latest snapshot, run:
pip install git+git://github.com/leolani/cltl-asr.git@main
Then you can import it in a python script as:
import numpy as np
import soundfile as sf
from importlib_resources import path
from cltl.asr.speechbrain_asr import SpeechbrainASR
asr = SpeechbrainASR("speechbrain/asr-transformer-transformerlm-librispeech")
with path("resources", "test.wav") as wav:
speech_array, sampling_rate = sf.read(wav, dtype=np.int16)
transcript = asr.speech_to_text(speech_array, sampling_rate)Please take a look at the example scripts provided to get an idea on how to run and use this package. Each example has a comment at the top of the script describing the behaviour of the script.
For these example scripts, you need
-
To change your current directory to ./examples/
-
Run some examples (e.g. python test_speechbrain_asr.py)
Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature') - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
- Fix logging
- Fix config of language, save/play audio file, audio directory
- Check if we can switch voices via different APIs
- Check implementation middle layer
Distributed under the MIT License. See LICENSE
for more information.