[DOW-101] LiveKit integration #591
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ajar98
changed the base branch from
ajay/dow-105-interruptions-refactor
to
ajay/dow-105-refactor-interrupts
ajar98
force-pushed
the
ajay/dow-101-livekit-build-the-integration
branch
from
ed07985
to
9c8bba3
Compare
ajar98
changed the title
vocode-petern
approved these changes
Jul 3, 2024
ajar98
added a commit
that referenced
this pull request
Jul 5, 2024
* [DOW-118] set up code linting and tests (#589) * adds github workflow * run black * run isort * adds precommit * adds vscode settings * adds pre-commit guidelines (#590) * creates docker image, updates telephony app deps (#601) * [DOW-105] refactor interruptions into the output device (#586) * [DOW-105] refactor interruptions into the output device (#562) * initial refactor works * remove notion of UtteranceAudioChunk and put all of the state in the callback * move per_chunk_allowance_seconds into output device * onboard onto vonage * rename to abstract output device and onboard other output devices * initial work to onboard twilio output device * twilio conversation works * some cleanup with better comments * unset poetry.lock * move abstract play method into ratelimitoutputdevice + dispatch to thread in fileoutputdevice * rename back to AsyncWorker * comments * work through a bit of mypy * asyncio.gather is g2g: * create interrupt lock * remove todo * remove last todo * remove log for interrupts * fmt * fix mypy * fix mypy * isort * creates first test and adds scaffolding * adds two other send_speech_to_output tests * make send_speech_to_output more efficient * adds tests for rate limit interruptions output device * makes some variables private and also makes the chunk id coming back from the mark match the incoming audio chunk * adds twilio output device tests * make typing better for output devices * fix mypy * resolve PR comments * resolve PR comments * [DOW-101] LiveKit integration (#591) * checkpoint * livekit v0 * in progress changes * integrate with worker * fix import * update deps and remove unneeded files * integrate it properly into app * fix interrupts * make transcript publish work * a confounding fix * isort * constants, some cleanup --------- Co-authored-by: Kian <[email protected]> * upgrade to latest cartesia 1.0.3 (#587) * upgrade to latest cartesia 1.0.3 * fixed linting conflict * finish streaming * make cartesia optional --------- Co-authored-by: Ajay Raj <[email protected]> * poetry version prerelease (#602) * feat: Add ability to configure OpenAI base URL in ChatGPTAgentConfig (#577) * feat: Add ability to configure OpenAI base URL in ChatGPTAgentConfig - Added `base_url` parameter to `ChatGPTAgentConfig` to allow customization of the OpenAI API base URL. - Updated `instantiate_openai_client` function to use the `base_url` parameter from the configuration. - Modified `ChatGPTAgent` to utilize the updated `instantiate_openai_client` function. - Added tests to verify the new `base_url` functionality in `tests/streaming/agent/test_base_agent.py`. This enhancement allows users to specify a custom OpenAI API base URL, providing greater flexibility in agent configuration. * adding capability to use the openai compatible endpoint with token estimation for llama * lint fix * changing openai base_url parameter for overall less code changes * missed logging update * Update vocode/streaming/agent/chat_gpt_agent.py * Update tests/streaming/agent/test_base_agent.py * fix test --------- Co-authored-by: Ajay Raj <[email protected]> * Support passthrough of AsyncHTTPTransport (#603) Support passthrough of AsyncHTTPTransport object * add script used to make PR * adds test target for vocodehq-public * Remove catch-all exception logger for asyncio tasks (#605) * remove error log from exception for asyncio tasks * remove log error on chatgpt query --------- Co-authored-by: Kian <[email protected]> Co-authored-by: rjheeta <[email protected]> Co-authored-by: Clay Elmore <[email protected]> Co-authored-by: vocode-petern <[email protected]> Co-authored-by: Adnaan Sachidanandan <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
integrates LiveKit into vocode on top of their Python SDK by:
LiveKitOutputDevice- the cool thing here is that when their client library receives audio frames, it waits until the frame has been played to return. This means that we don't have to do anything special to handle interrupts!LiveKitConversationto start the conversation and set up the callbacks to wire everything up properly[VOCODE REVIEWER] Areas of interest for other vocode dependencies: this PR refactors
is_activeslightly to make it controlled by anasyncio.Event- this is so that clients can wait on a conversation to end. This is the only bit that really needs a review here!