Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include cartesia's voice controls on docs + update synthesizer #674

Merged
merged 8 commits into from
Aug 7, 2024

Conversation

sauhardjain
Copy link
Contributor

@sauhardjain sauhardjain commented Aug 5, 2024

  • Fix end of sentence issues
    • Send ctx.no_more_inputs() after a 1s timeout on the last ctx.send()
    • Pad out the last chunk to make it uniform
  • Fix the emotion and speed settings
  • Add docs for the Cartesia synthesizer

Copy link
Contributor

@ajar98 ajar98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let me know when lint passes as well!

vocode/streaming/models/synthesizer.py Outdated Show resolved Hide resolved
vocode/streaming/synthesizer/cartesia_synthesizer.py Outdated Show resolved Hide resolved
@sauhardjain sauhardjain requested a review from ajar98 August 6, 2024 20:30
@sauhardjain sauhardjain marked this pull request as draft August 6, 2024 22:05
@sauhardjain sauhardjain marked this pull request as ready for review August 6, 2024 22:56
Copy link
Contributor Author

@sauhardjain sauhardjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also getting lint errors from vocode/streaming/vector_db/pinecone.py

@ajar98 ajar98 merged commit c1148e3 into vocodedev:main Aug 7, 2024
4 checks passed
@cyrilS-dev
Copy link

This pull request is causing a problem; I have no audio output from my synthesizer, even though my configuration remains unchanged :

synthesizer=CartesiaSynthesizer(
            synthesizer_config=CartesiaSynthesizerConfig.from_output_device(
                output_device=output_device,
                model_id = "sonic-multilingual",
                voice_id = "a249eaff-1e96-4d2c-b23b-12efa4f66f41"
            )
        )

@sauhardjain
Copy link
Contributor Author

@cyrilS-dev happy to get this resolved! What cartesia version are you using and do you have any logs associated with the issue here?

@cyrilS-dev
Copy link

cyrilS-dev commented Aug 13, 2024

thanks for your reply @sauhardjain

pip show cartesia returns Version: 1.0.11

I don't see any errors in the logs, and everything appears normal, but there is no audio output :

2024-08-13 14:19:11.602 | DEBUG    | vocode.streaming.streaming_conversation:process:481 - Synthesizing speech for message
2024-08-13 14:19:11.604 | WARNING  | vocode.streaming.synthesizer.audio_cache:safe_create:23 - Redis ping failed on startup, disabling audio cache
2024-08-13 14:19:11.748 | DEBUG    | vocode.streaming.streaming_conversation:send_speech_to_output:929 - Start sending speech Bonjour comment allez vous? to output
2024-08-13 14:19:11.749 | DEBUG    | vocode.streaming.streaming_conversation:process:429 - Sending end of turn
2024-08-13 14:19:11.880 | INFO     | vocode.streaming.livekit.livekit_conversation:_on_track_subscribed:39 - track subscribed
2024-08-13 14:19:12.497 | DEBUG    | vocode.streaming.streaming_conversation:send_speech_to_output:963 - Finished sending chunks to the output device

As additional information, i'm using
output_device = LiveKitOutputDevice()

Sample rate is 48000 and audio encoding is LINEAR16

@sauhardjain
Copy link
Contributor Author

@cyrilS-dev my speaker output also uses sample rate 48000 and audio encoding LINEAR16 so not sure what's happening here! let's connect on the vocode discord to debug! my user is sauahrd

cyrilS-dev added a commit to cyrilS-dev/vocode-core that referenced this pull request Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants