metrics: TTFT in streaming mode

[tao12345666333]

What type of PR is this?

metrics: TTFT in streaming mode

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #128

Release Notes: Yes/No

[netlify]

✅ Deploy Preview for vllm-semantic-router ready!

Name	Link
🔨 Latest commit	61f0374
🔍 Latest deploy log	https://app.netlify.com/projects/vllm-semantic-router/deploys/68d961cd75fa970008624abc
😎 Deploy Preview	https://deploy-preview-203--vllm-semantic-router.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[tao12345666333]

/hold

[github-actions]

👥 vLLM Semantic Team Notification

The following members have been identified for the changed files in this PR and have been automatically assigned:

📁 src

Owners: @rootfs, @Xunzhuo, @wangchen615
Files changed:

• src/semantic-router/pkg/extproc/metrics_integration_test.go
• src/semantic-router/pkg/extproc/request_handler.go
• src/semantic-router/pkg/extproc/response_handler.go

📁 website

Owners: @Xunzhuo
Files changed:

• website/docs/api/router.md
• website/docs/overview/architecture/envoy-extproc.md

---

🎉 Thanks for your contributions!

This comment was automatically generated based on the OWNER files in the repository.

[tao12345666333]

/unhold

[Xunzhuo]

/hold cancel

[rootfs]

@Xunzhuo

[rootfs]

@tao12345666333 would you please come up with a PR to explain how to enable streaming?

[tao12345666333]

@rootfs I've added some notes in the document. https://vllm-semantic-router.com/docs/api/router#streaming-sse-notes

Do you want me to add a dedicated section describing the request for enabling streaming?

In my current implementation, no special configuration is required, uust keep the default settings in the current project; it automatically determines whether to use streaming based on the request headers.

[rootfs]

@tao12345666333 gotcha, thanks for the info!