New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add streaming API to orchestration #352

Open

tomfrenken wants to merge 59 commits into main from orchestration-streaming

Member

tomfrenken commented Dec 6, 2024 •

edited

Loading

Context

SAP/ai-sdk-js-backlog#161.

This PR adds a streaming API to our orchestration client.

Definition of Done

Code is tested (Unit, E2E)
Error handling created / updated & covered by the tests above
Documentation updated
(Optional) Aligned changes with the Java SDK
(Optional) Release notes updated

tomfrenken added 3 commits

December 4, 2024 14:00


          branch

beac1bd


          add most parts

d55ec77


          stream method

3e36337

tomfrenken commented

View reviewed changes

packages/orchestration/src/orchestration-chat-completion-stream.ts Outdated Show resolved Hide resolved

tomfrenken changed the title ~~feat: Add streaming API to orchestration~~ WIP: feat: Add streaming API to orchestration

ZhongpinWang reviewed

View reviewed changes

packages/foundation-models/src/azure-openai/azure-openai-embedding-client.ts Outdated Show resolved Hide resolved

tomfrenken and others added 3 commits

December 9, 2024 16:15


          remove path

d0f0277


          change types

28e9146


          fix: Changes from lint

27f22a2

ZhongpinWang reviewed

View reviewed changes

Contributor

ZhongpinWang left a comment

Already looks promising. For code reuse, maybe after more finds are gathered, we can look into it again to see if it is worth the effort.

packages/orchestration/src/orchestration-chat-completion-stream-chunk-response.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-chat-completion-stream-chunk-response.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-chat-completion-stream.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-chat-completion-stream.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-chat-completion-stream.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-chat-completion-stream-chunk-response.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-client.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-client.ts Outdated Show resolved Hide resolved

tomfrenken and others added 10 commits

December 9, 2024 17:17


          remove todo

7c57cf0


          Merge branch 'main' of https://github.com/SAP/ai-sdk-js into orchestr…

ec8369b

…ation-streaming


          merge

217efdc


          common execute

27f4032


          fix: Changes from lint

15fae8a


          update docs, types

188e0ba


          narrow type

431abce


          add sample code, tests, ...

3fdd473


          update code

f04582d


          fix streaming

8b5b61e

tomfrenken changed the title ~~WIP: feat: Add streaming API to orchestration~~ feat: Add streaming API to orchestration

tomfrenken added 4 commits

December 18, 2024 15:16


          add mock responses

c64d3bd


          update test

032f46f


          update test

728ff82


          add last test

4180d54

MatKuhr reviewed

View reviewed changes

sample-code/src/orchestration.ts Outdated Show resolved Hide resolved

sample-code/src/orchestration.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-client.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-chat-completion-stream-chunk-response.ts Outdated Show resolved Hide resolved

tomfrenken and others added 4 commits

December 19, 2024 13:41


          fix tests

94d9c35


          lint:fix

48b55fe


          Merge branch 'main' into orchestration-streaming

46c9378


          remove internal from core

f6735f9

tomfrenken requested review from ZhongpinWang, deekshas8 and MatKuhr

January 6, 2025 16:06

ZhongpinWang reviewed

View reviewed changes

Contributor

ZhongpinWang left a comment

One more round

packages/orchestration/src/orchestration-client.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-utils.ts Show resolved Hide resolved

packages/orchestration/src/orchestration-types.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-utils.ts Outdated Show resolved Hide resolved

MatKuhr reviewed

View reviewed changes

Member

MatKuhr left a comment

Didn't get through everything, but some ideas. Also, let's remember to add E2E tests

packages/orchestration/src/orchestration-client.ts Outdated Show resolved Hide resolved

packages/orchestration/README.md Outdated Show resolved Hide resolved

packages/core/src/stream/line-decoder.ts Show resolved Hide resolved

packages/orchestration/README.md Show resolved Hide resolved

packages/orchestration/src/orchestration-client.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-types.ts Outdated Show resolved Hide resolved

packages/orchestration/src/orchestration-client.ts Show resolved Hide resolved

packages/orchestration/src/orchestration-stream-chunk-response.ts Show resolved Hide resolved

tomfrenken and others added 12 commits

January 7, 2025 17:44


          adjust tests & logic

2b03915


          merge main

0407c78


          fix: Changes from lint

5dcbf06


          final logic fixes

cb35f42


          Merge main

15bd1cd


          Add ability to remove options

a552844


          merge main

f83c029


          last exception

ee5b88b


          add docs

d454941


          fix: Changes from lint

b73ad7d


          vale

1eb65d7


          allow-list llm

27a3350

ZhongpinWang reviewed

View reviewed changes

packages/orchestration/README.md Show resolved Hide resolved

packages/orchestration/README.md Outdated Show resolved Hide resolved

packages/orchestration/README.md Outdated Show resolved Hide resolved

packages/orchestration/README.md Outdated Show resolved Hide resolved

packages/orchestration/README.md Outdated Show resolved Hide resolved

tomfrenken added 2 commits

January 9, 2025 17:24


          add tests and docs


          change orchestration exports

tomfrenken dismissed stale reviews from KavithaSiva and deekshas8

January 10, 2025 11:19

Done


          add a billion tests

a5d65ef

tomfrenken requested a review from ZhongpinWang

January 10, 2025 13:32

tomfrenken and others added 2 commits

January 10, 2025 15:03


          merge billion conflicts

24885c9


          fix: Changes from lint

4d78c6a

Member Author

tomfrenken commented Jan 10, 2025

Side node while I add streaming to our sample code and e2e tests, we should really refactor our server.ts file, lots of duplication and it should be more modularized ...

ZhongpinWang approved these changes

View reviewed changes

packages/orchestration/README.md Show resolved Hide resolved

tests/e2e-tests/src/orchestration.test.ts Outdated Show resolved Hide resolved

ZhongpinWang requested changes

View reviewed changes

Contributor

ZhongpinWang left a comment

After e2e added


          add sample code and e2e tests

5663d4e

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet