Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Incorrect accent in Elevenlabs voice output when non-English language is selected #649

Open
1 task done
rwb-truelime opened this issue Sep 27, 2024 · 4 comments
Open
1 task done
Labels
type: bug Something isn't working

Comments

@rwb-truelime
Copy link

Description

When selecting a non-English language (specifically Dutch) for voice output using Elevenlabs, the generated speech still has a strong English accent instead of the expected Dutch accent. This issue persists even when a Dutch voice is explicitly selected.

Steps to reproduce:

  1. Open big_AGI
  2. Navigate to the language selection settings
  3. Choose Dutch as the language
  4. Select the Dutch voice (voice_id: YUdpWWny7k5yb4QCeweX) for Elevenlabs output
  5. Generate voice output for Dutch text

Expected behavior:
The voice output should have a proper Dutch accent matching the selected Dutch voice.

Actual behavior:
The voice output has a strong English accent, despite the Dutch language and voice selection.

Additional information:

  • Elevenlabs voice ID used: YUdpWWny7k5yb4QCeweX
  • This issue seems to occur consistently when selecting non-English languages

Please investigate why the accent is not changing according to the selected language and voice, and implement a fix to ensure the correct accent is applied for non-English language selections.

Device and browser

Device: Dell XPS 15 9520
OS: Microsoft Windows 11 Enterprise, Version 10.0.22631 Build 22631
Browser: Microsoft Edge 129.0.2792.52 (Official build) (64-bit)

Additional details:

Processor: 12th Gen Intel(R) Core(TM) i7-12700H, 2300 MHz, 14 core(s), 20 logical processor(s)
RAM: 32.0 GB
System Type: x64-based PC
BIOS Version: Dell Inc. 1.25.0, 11-7-2024
Language: Dutch (based on the system information being in Dutch)

Screenshots and more

No response

Willingness to Contribute

  • 🙋‍♂️ Yes, I would like to contribute a fix.
@rwb-truelime rwb-truelime added the type: bug Something isn't working label Sep 27, 2024
@darthalex2014
Copy link

I'm having the same issue. I even switched to the eleven_turbo_v2_5 model (it supports Russian, unlike eleven_multilingual_v1), but it still speaks with an English accent.

@enricoros
Copy link
Owner

Thanks @rwb-truelime -- does it do the same thing if the language is changed inside Preferences?
image

It's possible we are forcing this language in the app. @darthalex2014 does eleven_turbo_v2_5 detect the language automatically? If so then we don't need the UI selector for the language anymore, and we'd rather just call the latest model.

@rwb-truelime
Copy link
Author

Thanks @rwb-truelime -- does it do the same thing if the language is changed inside Preferences? image

It's possible we are forcing this language in the app. @darthalex2014 does eleven_turbo_v2_5 detect the language automatically? If so then we don't need the UI selector for the language anymore, and we'd rather just call the latest model.

Yes, that is the place I've set the language to Dutch. Thanks for looking into this!

@zoollcar
Copy link

I test in Chinese, Use the following code to switch to eleven_turbo_v2_5, I think it can detect the language automatically

src\modules\elevenlabs\elevenlabs.server.ts

    const body: ElevenlabsWire.TTSRequest = {
      text: text,
      ...({ model_id: 'eleven_turbo_v2_5' }),
    };

It sounds much better than eleven_multilingual_v1 or eleven_multilingual_v2, It still with an English accent but acceptable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants