feat: Add ErrorFrame emission to TTS/STT services for pipeline error detection #2881

Angad-2002 · 2025-10-17T20:29:58Z

PR for #2876

Changes Made -

Services Updated:

CartesiaTTSService - Connection, message sending, and general exception handling
CartesiaSTTService - Connection failures and error message handling
ElevenLabsTTSService - Connection, disconnect, message sending, and general exception handling
DeepgramFluxSTTService - Connection and disconnect failures
DeepgramTTSService - Runtime exception handling (updated to use push_error())
AssemblyAISTTService - Connection and disconnect failures
RimeTTSService - Connection, disconnect, message sending, and general exception handling
RimeHttpTTSService - Runtime exception handling
AzureTTSService - Runtime exception handling

Key Features:

ErrorFrame objects emitted with fatal=False for graceful degradation
Consistent error handling patterns across all services
Comprehensive test coverage with pytest test suite
All pre-commit hooks passed (ruff formatting)

Testing

Added pytest test suite (tests/test_tts_stt_error_handling.py) that verifies:

ErrorFrame emission during connection failures
ErrorFrame emission during runtime failures
Proper error message content and fatality settings
Integration with on_pipeline_error event handler

Angad-2002 · 2025-10-18T05:55:34Z

@markbackman could you look into this PR?

markbackman · 2025-10-18T12:51:04Z

src/pipecat/services/assemblyai/stt.py

        except Exception as e:
            logger.error(f"Failed to connect to AssemblyAI: {e}")
            self._connected = False
+            await self.push_error(ErrorFrame(error=e, fatal=False))


Can we use this as a pattern?

await self.push_error(ErrorFrame(error=f"{self} error: {e}, fatal=True"))

In this specific instance, is a connection error fatal?

Yeah, re checking it I believe it is fatal = True, I am updating on the patterns to be followed

@markbackman should we also standardize the logger.error messages as logger.error(f"{self} exception: {e}")
or leave them as they are?

in some cases its not the format and in some places its there

markbackman · 2025-10-18T12:51:53Z

src/pipecat/services/azure/tts.py


        except Exception as e:
            logger.error(f"{self} exception: {e}")
+            await self.push_error(ErrorFrame(error=e, fatal=False))


Should we also add an ErrorFrame to the Exception block starting in line 357?

markbackman · 2025-10-18T12:53:51Z

tests/test_tts_stt_error_handling.py

@@ -0,0 +1,252 @@
+"""


Thanks for offering this, but we need to create a pattern for this type of testing. For now, I'd say to just remove this and we can consider the best way to proceed.

noted, I would be removing this test

markbackman

This generally looks good. I would recommend revisiting all STT and TTS classes to make sure that all Exceptions have ErrorFrames. Also, let's stick to the pattern of:

await self.push_error(ErrorFrame(error=f"{self} error: {e}", fatal=True/False))

Thanks so much for contributing this 🙏. I was actually just looking to do this on Friday, now that we have an on_pipeline_error event which benefits from these ErrorFrames being push. 🙌

Can you also do the following?

Rebase on main
Make sure code is linted (scripts/fix-ruff.sh or install pre-commit hook: uv run pre-commit install)
Add a changelog entry

Angad-2002 · 2025-10-18T14:24:46Z

Noted. I'll be making these changes

Angad-2002 · 2025-10-22T07:43:09Z

@markbackman Can you check the updates made?

Changelog Entry
Rebased on main
Code linted

Angad-2002 · 2025-10-22T10:59:57Z

@markbackman I updated to add all the TTS and STT services

aconchillo · 2025-10-22T17:40:08Z

src/pipecat/services/assemblyai/stt.py

-            logger.error(f"Failed to connect to AssemblyAI: {e}")
+            logger.error(f"{self} exception: {e}")
            self._connected = False
+            await self.push_error(ErrorFrame(error=f"{self} error: {e}", fatal=True))


This should probably not be fatal. A switcher processor could just switch to another STT service. In general, I believe we shouldn't really be using fatal=True unless for uncaught exceptions, which we handle somewhere else.

Thx for the comment so should we remove the fatal flag, through all the stt tts services again? and where to use True

aconchillo · 2025-10-22T17:41:31Z

src/pipecat/services/assemblyai/stt.py

                except Exception as e:
-                    logger.error(f"Error processing WebSocket message: {e}")
+                    logger.error(f"{self} exception: {e}")
+                    await self.push_error(ErrorFrame(error=f"{self} error: {e}", fatal=True))


Same.Don't make it fatal. I believe we should just remove fatal=X form this PR and use the default (which is False). Fatal means the whole bot should end.

So bascially you suggest we remove all exception frames from all stt and tts files

aconchillo · 2025-10-22T17:43:51Z

src/pipecat/services/cartesia/stt.py

+        elif isinstance(frame, UserStoppedSpeakingFrame):
+            # Send finalize command to flush the transcription session
+            if self._connection and self._connection.state is State.OPEN:
+                await self._connection.send("finalize")


Why are we adding these 3 methods? These seem to conflict with other methods already defined (e.g. _disconnect).

I noticed that disconnect part, I would be removing this

aconchillo · 2025-10-22T17:47:43Z

@Angad-2002 This is great, thank you for taking the time and do this. I've added some comments.

Angad-2002 · 2025-10-22T17:55:54Z

Thx for the comments so what changes should be made overall?

aconchillo · 2025-10-22T18:11:59Z

Thx for the comments so what changes should be made overall?

Remove fatal=X from everywhere. Apologies if it was not clear 🙇

aconchillo · 2025-10-22T18:12:38Z

Thx for the comments so what changes should be made overall?

Remove fatal=X from everywhere. Apologies if it was not clear 🙇

Also, there were some functions added. Not sure what the purpose of those were.

Angad-2002 · 2025-10-23T05:53:27Z

I apologize for the duplication, actually I was rebasing on main, and merge conflicts led to duplicates

…detection - Add ErrorFrame emission to all major TTS/STT services during initialization and runtime failures - Services updated: Cartesia, ElevenLabs, Deepgram, AssemblyAI, Rime, Azure - ErrorFrame objects emitted with fatal=False for graceful degradation - Enables on_pipeline_error event handler to detect service failures programmatically - Add comprehensive pytest test suite to verify ErrorFrame emission - Fixes issue where services failed gracefully but didn't emit ErrorFrame objects This allows developers to implement real-time error monitoring and alerting using the on_pipeline_error event handler introduced in v0.0.90.

- Improves error handling consistency across all services

Angad-2002 · 2025-10-23T06:23:56Z

@aconchillo I have rebased on main again and pushed new commit for flag removal and duplicate removal

Angad-2002 · 2025-10-24T05:05:28Z

@aconchillo @markbackman pls check the changes

markbackman reviewed Oct 18, 2025

View reviewed changes

markbackman approved these changes Oct 18, 2025

View reviewed changes

Angad-2002 force-pushed the feature/tts-stt-error-frames branch from fdf129c to bd654fa Compare October 22, 2025 07:09

aconchillo reviewed Oct 22, 2025

View reviewed changes

markbackman self-requested a review October 22, 2025 22:53

Angad-2002 added 7 commits October 23, 2025 11:41

Update STT and TTS services to use consistent error handling pattern

0a8ba26

- Improves error handling consistency across all services

Add changelog entry for STT/TTS error handling improvements

cda7dd7

Linting issues Resolved

ef45efe

Azure STT ErrorFrames added with consistent patterns

a1b3cad

Cartesia STT and Deepgram STT; additional fixes made

6c6b397

Removed Fatal Flags across services, removed duplication

68ea479

Angad-2002 force-pushed the feature/tts-stt-error-frames branch from 0a7bba7 to 68ea479 Compare October 23, 2025 06:14

Angad-2002 requested a review from aconchillo October 23, 2025 18:53

feat: Add ErrorFrame emission to TTS/STT services for pipeline error detection #2881

Are you sure you want to change the base?

feat: Add ErrorFrame emission to TTS/STT services for pipeline error detection #2881

Conversation

Angad-2002 commented Oct 17, 2025

Key Features:

Testing

Uh oh!

Angad-2002 commented Oct 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

markbackman left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Angad-2002 commented Oct 18, 2025

Uh oh!

Angad-2002 commented Oct 22, 2025

Uh oh!

Angad-2002 commented Oct 22, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aconchillo commented Oct 22, 2025

Uh oh!

Angad-2002 commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aconchillo commented Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aconchillo commented Oct 22, 2025

Uh oh!

Angad-2002 commented Oct 23, 2025

Uh oh!

Angad-2002 commented Oct 23, 2025

Uh oh!

Angad-2002 commented Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

markbackman left a comment •

edited

Loading

Angad-2002 commented Oct 22, 2025 •

edited

Loading

aconchillo commented Oct 22, 2025 •

edited

Loading

Angad-2002 commented Oct 24, 2025 •

edited

Loading