feat(node): Instrument stream responses for openai #17110

RulaKhaled · 2025-07-21T12:21:53Z

This adds support for OpenAI streaming responses in the Node.js SDK

What's new

Streaming Chat Completions: Support for stream: true in chat.completions.create()
Streaming Responses API: Support for streaming in the new OpenAI Responses API


// Streaming chat completions - automatically instrumented
const stream = await openai.chat.completions.create({
  model: 'gpt-4',
  messages: [...],
  stream: true
});

// Streaming responses API - automatically instrumented  
const responseStream = await openai.responses.create({
  model: 'gpt-4',
  input: 'Hello',
  stream: true
});

github-actions · 2025-07-21T12:29:17Z

size-limit report 📦

Path	Size	% Change	Change
@sentry/browser	23.76 kB	-	-
@sentry/browser - with treeshaking flags	22.35 kB	-	-
@sentry/browser (incl. Tracing)	39.41 kB	-	-
@sentry/browser (incl. Tracing, Replay)	77.52 kB	-	-
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags	67.39 kB	-	-
@sentry/browser (incl. Tracing, Replay with Canvas)	82.23 kB	-	-
@sentry/browser (incl. Tracing, Replay, Feedback)	94.35 kB	-	-
@sentry/browser (incl. Feedback)	40.45 kB	-	-
@sentry/browser (incl. sendFeedback)	28.44 kB	-	-
@sentry/browser (incl. FeedbackAsync)	33.34 kB	-	-
@sentry/react	25.5 kB	-	-
@sentry/react (incl. Tracing)	41.38 kB	-	-
@sentry/vue	28.2 kB	-	-
@sentry/vue (incl. Tracing)	41.21 kB	-	-
@sentry/svelte	23.79 kB	-	-
CDN Bundle	25.28 kB	-	-
CDN Bundle (incl. Tracing)	39.28 kB	-	-
CDN Bundle (incl. Tracing, Replay)	75.39 kB	-	-
CDN Bundle (incl. Tracing, Replay, Feedback)	80.82 kB	-	-
CDN Bundle - uncompressed	73.86 kB	-	-
CDN Bundle (incl. Tracing) - uncompressed	116.29 kB	-	-
CDN Bundle (incl. Tracing, Replay) - uncompressed	230.53 kB	-	-
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed	243.35 kB	-	-
@sentry/nextjs (client)	43.44 kB	-	-
@sentry/sveltekit (client)	39.84 kB	-	-
@sentry/node-core	47.48 kB	-0.01%	-1 B 🔽
@sentry/node	144.84 kB	+0.5%	+712 B 🔺
@sentry/node - without tracing	91.58 kB	-	-
@sentry/aws-serverless	103.03 kB	-	-

View base workflow run

This reverts commit 1607304.

packages/core/src/utils/openai/streaming.ts

…enario-express-error.mjs

cursor · 2025-07-25T12:53:56Z

packages/core/src/utils/openai/index.ts

@@ -195,6 +143,9 @@ function addRequestAttributes(span: Span, params: Record<string, unknown>): void
  if ('input' in params) {
    span.setAttributes({ [GEN_AI_REQUEST_MESSAGES_ATTRIBUTE]: JSON.stringify(params.input) });
  }
+  if ('stream' in params) {
+    span.setAttributes({ [OPENAI_RESPONSE_STREAM_ATTRIBUTE]: Boolean(params.stream) });
+  }


Bug: Incorrect Attribute Assignment in Function

The addRequestAttributes function incorrectly sets the OPENAI_RESPONSE_STREAM_ATTRIBUTE (a response attribute) based on a request parameter. This function is intended for request attributes, and the stream request parameter is already correctly captured as GEN_AI_REQUEST_STREAM_ATTRIBUTE by extractRequestAttributes.

Locations (1)

packages/core/src/utils/openai/index.ts#L145-L148

Fix in Cursor • Fix in Web

cursor · 2025-07-25T12:53:56Z

dev-packages/node-integration-tests/suites/tracing/openai/test.ts

+          'openai.response.timestamp': '2023-03-01T06:31:50.000Z',
+          'openai.usage.completion_tokens': 0,
+          'openai.usage.prompt_tokens': 0,
+        },


Bug: Inconsistent Metadata Across PII Configurations

The test expectations for OpenAI streaming responses are inconsistent between sendDefaultPii: false and sendDefaultPii: true configurations. When sendDefaultPii: false, the gen_ai.response.finish_reasons is expected as ["in_progress"] and token usage fields (gen_ai.usage.*_tokens) are expected to be 0. However, when sendDefaultPii: true, finish_reasons is ["in_progress","completed"] and token usage is correctly captured. These metadata fields should be consistently collected regardless of PII settings, as the mock streaming implementation emits all relevant data in both scenarios.

Locations (1)

dev-packages/node-integration-tests/suites/tracing/openai/test.ts#L112-L122

Fix in Cursor • Fix in Web

cursor · 2025-07-25T12:53:56Z

packages/core/src/utils/openai/streaming.ts

+    return;
+  }
+
+  const { response } = event as { response: OpenAIResponseObject };


Bug: Event Handling Error: Missing Response Property

The processResponsesApiEvent function unsafely attempts to destructure a response property from the event object. Events of type response.output_text.delta (e.g., ResponseOutputTextDeltaEvent) do not contain a response property. While these events are typically handled by an early return, if recordOutputs is false or the delta property is missing, they will proceed to the destructuring, causing a runtime error. This also applies to other unknown event types that lack a response property. A check for the response property's existence is required before destructuring.

Locations (1)

packages/core/src/utils/openai/streaming.ts#L68-L74

Fix in Cursor • Fix in Web

cursor · 2025-07-25T12:53:56Z

packages/core/src/utils/openai/index.ts

+          } catch (error) {
+            captureException(error);
+            span.end();
+            throw error;


Bug: Streaming Error Reporting Inconsistency

OpenAI streaming calls that error before the stream begins incorrectly report status: 'ok'. This occurs because startSpanManual, used for streaming, does not automatically set the span status to an error on exceptions, unlike startSpan for non-streaming calls. This results in inconsistent error reporting and an incorrect test expectation.

Locations (2)

dev-packages/node-integration-tests/suites/tracing/openai/test.ts#L277-L278

packages/core/src/utils/openai/index.ts#L203-L207

Fix in Cursor • Fix in Web

AbhiPrasad

I'd like us to add some more jsdoc explanations to packages/core/src/utils/openai/streaming.ts

AbhiPrasad · 2025-07-25T12:53:04Z

packages/core/src/utils/openai/streaming.ts

+    return;
+  }
+  if (streamEvent instanceof Error) {
+    captureException(streamEvent);


m: when we capture an exception here, should we also set span status to errored?

also, I think we need to mark this as unhandled.

Potentially yeah. It's a series of events we're capturing in a single span, which makes things a bit vague. I guess if one fails it's fair to assume the span is in an error state, will update

AbhiPrasad · 2025-07-25T12:56:48Z

packages/core/src/utils/openai/streaming.ts

+  setTokenUsageAttributes,
+} from './utils';
+
+interface StreamingState {


m: It would be nice to have some doc strings about this interface and it's fields.

l: I'm not the biggest fan of having a big interface with a bunch of nullable fields. Can we type this a bit stronger? For example if chunk.usage we should always have promptTokens, completionTokens, and totalTokens defined. Having some helpers that construct state from input (builder pattern?) might also be a useful abstraction.

packages/core/src/utils/openai/streaming.ts

RulaKhaled added 2 commits July 21, 2025 09:53

feat: Stream responses for openai node

86f45af

manually end the span

b8bd91c

RulaKhaled added 5 commits July 21, 2025 16:16

Delete packages/node/test/integrations/tracing/openai/openai.test.ts

0a8ea4f

more refactoring

bb6b91d

Add tests

d1ebaaa

Merge branch 'develop' into stream-responses-openai

04baa51

just bump bundle size a bit

e41d2ad

RulaKhaled changed the title ~~[WIP] Stream responses OpenAI~~ feat(node): Instrument stream responses for openai Jul 23, 2025

Revert "try timeout sol"

4fba247

This reverts commit 1607304.

RulaKhaled requested a review from andreiborza July 24, 2025 12:43

RulaKhaled marked this pull request as ready for review July 24, 2025 14:24

cursor bot reviewed Jul 24, 2025

View reviewed changes

packages/core/src/utils/openai/streaming.ts Show resolved Hide resolved

mydea requested a review from AbhiPrasad July 25, 2025 09:24

RulaKhaled added 4 commits July 25, 2025 14:41

quick refactor

305fd52

Delete dev-packages/node-integration-tests/suites/tracing/vercelai/sc…

b4a43e2

…enario-express-error.mjs

Update test.ts

ffcd2f7

Merge branch 'develop' into stream-responses-openai

2dffa0e

cursor bot reviewed Jul 25, 2025

View reviewed changes

AbhiPrasad reviewed Jul 25, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(node): Instrument stream responses for openai #17110

feat(node): Instrument stream responses for openai #17110

Uh oh!

RulaKhaled commented Jul 21, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jul 21, 2025 •

edited

Loading

Uh oh!

Uh oh!

cursor bot Jul 25, 2025

Uh oh!

cursor bot Jul 25, 2025

Uh oh!

cursor bot Jul 25, 2025

Uh oh!

cursor bot Jul 25, 2025

Uh oh!

AbhiPrasad left a comment

Uh oh!

AbhiPrasad Jul 25, 2025

Uh oh!

RulaKhaled Jul 25, 2025

Uh oh!

AbhiPrasad Jul 25, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

feat(node): Instrument stream responses for openai #17110

Are you sure you want to change the base?

feat(node): Instrument stream responses for openai #17110

Uh oh!

Conversation

RulaKhaled commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

size-limit report 📦

Uh oh!

Uh oh!

cursor bot Jul 25, 2025

Choose a reason for hiding this comment

Bug: Incorrect Attribute Assignment in Function

Uh oh!

cursor bot Jul 25, 2025

Choose a reason for hiding this comment

Bug: Inconsistent Metadata Across PII Configurations

Uh oh!

cursor bot Jul 25, 2025

Choose a reason for hiding this comment

Bug: Event Handling Error: Missing Response Property

Uh oh!

cursor bot Jul 25, 2025

Choose a reason for hiding this comment

Bug: Streaming Error Reporting Inconsistency

Uh oh!

AbhiPrasad left a comment

Choose a reason for hiding this comment

Uh oh!

AbhiPrasad Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

RulaKhaled Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

AbhiPrasad Jul 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

RulaKhaled commented Jul 21, 2025 •

edited

Loading

github-actions bot commented Jul 21, 2025 •

edited

Loading