Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization suggestion: Reuse previous memory/lore queries when generating variant responses #57

Open
kickahaota opened this issue Jun 18, 2023 · 1 comment

Comments

@kickahaota
Copy link

When the user clicks the blue double-arrow button to generate a new variant response, it looks like the memory and lore queries are repeated. That seems a bit inefficient and expensive. Can the previous memory/lore results be reused instead?

@josephrocca
Copy link
Owner

josephrocca commented Jul 6, 2023

Not sure that this makes sense in all/most cases - the user may be retrying because (whether they know or not) the queries were bad. The memory queries are very cheap anyway (compared to bot responses - EDIT: actually this may be wrong, can't remember how much context memory-query-generations use. they should be cheaper - but not sure by how much), so it'd only be worth it to save the user some time (but again, not sure it would make sense in all/most cases due to aforementioned reason)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants