Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Master list and simplified data are not being saved. #38

Open
BovineOverlord opened this issue Jul 14, 2024 · 5 comments
Open

Master list and simplified data are not being saved. #38

BovineOverlord opened this issue Jul 14, 2024 · 5 comments

Comments

@BovineOverlord
Copy link

9 questions were generated from the text. I do not know if there is a backup of them that were saved, however what I can confirm is that the master list and simplified data were not saved at the end. As you can see in the image below, they are 0kb in size.

no save
@e-p-armstrong
Copy link
Owner

e-p-armstrong commented Jul 15, 2024

What's probably happened here is that since there are relatively few questions and it looks like some of the API requests are timing out, all the conversations that were generated probably failed validation, leaving the resulting files as 0kb.

Can you maybe share your config, the console output, and maybe some of the output files?

@BovineOverlord
Copy link
Author

No need- I can confirm this is the case. This is due to using a very heavy model to do the processing. Is there a way to disable to timeout check or modify the time length it checks?

@e-p-armstrong
Copy link
Owner

I think that, with streaming, timeout is essentially disabled on Augmentoolkit's side of things. So it's probably happening on your provider's end?

@mjh624
Copy link

mjh624 commented Sep 12, 2024

I also observed that the master list and simplified data were not written. The log file flags many/all conversations as too short. But when I look at the conversations, they are actually quite long.

@e-p-armstrong
Copy link
Owner

@mjh624 Conversations being too short means that not all questions were asked. There were too few turns in the conversation to account for all questions and answers that were inputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants