Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve response time #1

Open
itsderek23 opened this issue Sep 7, 2023 · 1 comment
Open

Improve response time #1

itsderek23 opened this issue Sep 7, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@itsderek23
Copy link
Contributor

Currently the CLI has a response time of ~40 sec when answering an AWS-related question.

image

Look into options for improving the performance.

@itsderek23 itsderek23 added the enhancement New feature or request label Sep 7, 2023
@itsderek23
Copy link
Contributor Author

itsderek23 commented Oct 6, 2023

While I don't have updated benchmarks to share yet, the response time on many questions is quite a bit faster. The agent will now re-use functions that are human-validated vs. re-generating new functions all of the time. I've also reduced the number of messages required to answer a question.

For example, note this took ~5 seconds vs 40+ seconds:

time llm "average ec2 cpu utilization"
The average CPU utilization of your EC2 instances over the past hour is approximately 5.20%.
llm "average ec2 cpu utilization"  0.22s user 0.07s system 5% cpu 4.935 total

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant