Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create button to stop streaming of response #1884

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

ssjdan27
Copy link

@ssjdan27 ssjdan27 commented Aug 4, 2024

Purpose

These changes will allow the user to stop the streaming of a response if they so choose. This is helpful if the response does not seem to be useful or if the LLM is hallucinating. The approach I used to implement this may not be perfect, but I would love to get some constructive feedback on this @pamelafox.

Does this introduce a breaking change?

When developers merge from main and run the server, azd up, or azd deploy, will this produce an error?
If you're not sure, try it out on an old environment.

[ ] Yes
[X] No

Does this require changes to learn.microsoft.com docs?

This repository is referenced by this tutorial
which includes deployment, settings and usage instructions. If text or screenshot need to change in the tutorial,
check the box below and notify the tutorial author. A Microsoft employee can do this for you if you're an external contributor.

[ ] Yes
[X] No

Type of change

[ ] Bugfix
[X] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Documentation content changes
[ ] Other... Please describe:

Code quality checklist

See CONTRIBUTING.md for more details.

  • The current tests all pass (python -m pytest).
  • I added tests that prove my fix is effective or that my feature works
  • I ran python -m pytest --cov to verify 100% coverage of added lines
  • I ran python -m mypy to check for type errors
  • I either used the pre-commit hooks or ran ruff and black manually on my code.

@ssjdan27 ssjdan27 closed this Aug 4, 2024
@ssjdan27 ssjdan27 reopened this Aug 4, 2024
@ssjdan27
Copy link
Author

ssjdan27 commented Aug 4, 2024

@microsoft-github-policy-service agree

@pamelafox
Copy link
Collaborator

Thanks for the PR! I will take a look soon. Am also interested to see feedback from other folks in the thread for #1221

@pamelafox
Copy link
Collaborator

This PR requires an update now that the internationalization change is in. I made the necessary changes but could not push them to your PR since your PR is off a "main" branch. You can see the changes I made here:
https://github.com/Azure-Samples/azure-search-openai-demo/compare/main...pamelafox:azure-search-openai-demo:stopmerge?expand=1
@ssjdan27 Could you bring in those changes?

@ssjdan27
Copy link
Author

ssjdan27 commented Sep 2, 2024

I've successfully merged the changes you made, @pamelafox, into this PR. Please let me know if there's anything else I should address with these updates!

<div>
<UserChatMessage message={lastQuestionRef.current} />
<div className={styles.chatMessageGpt}>
<Answer
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it that we need to have this if/else with a whole Answer component configuration, versus sending in a parameter to Answer?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question. I believe I remember running into weird issues when dealing with the partial responses that would sometimes cause the last messages to disappear. This was put into place to circumvent that, but I am all ears if there is a cleaner implementation!

@advanced-flow
Copy link

Any news here @ssjdan27? I would be really interest in this enhancement 🙌

@ssjdan27
Copy link
Author

ssjdan27 commented Nov 6, 2024

@advanced-flow Hi there! I am still keen on refining this feature, but life has gotten a bit busy these past few months. If I had to put a time on it, I can probably fully revisit this sometime mid December, but I will aim for sooner if time allows!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants