Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming oracles for better memory usage #19

Open
ValarDragon opened this issue Aug 14, 2020 · 0 comments
Open

Streaming oracles for better memory usage #19

ValarDragon opened this issue Aug 14, 2020 · 0 comments
Labels
good first issue Good for newcomers Memory memory usage improvements

Comments

@ValarDragon
Copy link
Member

Memory usage is a bottleneck that we hit for large deployments.

The peak memory usage at the moment is during the LDT Reducer. The LDT reducer takes a random linear combination of all constraints over the large codeword domain (L in papers) from the protocol. The current API is that it evaluates all constraints in full, and then takes a random linear combination of them all. Instead, we should stream these oracles into the LDT reducer one by one. The LDT Reducer would maintain some 'current state' and keep updating this with every streamed oracle.

This should significantly lower the memory requirement, and if we eventually use a 'codeword pool' it would likely improve prover time.

@ValarDragon ValarDragon added the Memory memory usage improvements label Aug 14, 2020
@ValarDragon ValarDragon added the good first issue Good for newcomers label Aug 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers Memory memory usage improvements
Projects
None yet
Development

No branches or pull requests

1 participant