Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle sparse storage properly #17

Open
khernyo opened this issue Aug 8, 2019 · 2 comments
Open

Handle sparse storage properly #17

khernyo opened this issue Aug 8, 2019 · 2 comments

Comments

@khernyo
Copy link
Member

khernyo commented Aug 8, 2019

The current implementation doesn't seem to do sparse allocation like https://github.com/random-access-storage/random-access-memory.

In short: RandomAccessMemory::buffers should be of type Vec<Option<Vec<u8>>> instead of the current type Vec<Vec<u8>>.

I can see two problems with the current implementation:

  1. RandomAccessMemory::write() allocates all pages before page_num unnecessarily. See

    self.buffers.resize(page_num + 1, buf);

  2. RandomAccessMemory::del() seems to be broken. When removing a page it shifts all following pages thereby corrupting the stored data. Compare

    self.buffers.remove(i);
    to https://github.com/random-access-storage/random-access-memory/blob/da2d94570fc48674c22746b288bf65c3bf44015d/index.js#L93

@bltavares
Copy link
Member

bltavares commented Aug 21, 2019

Should this adopt another of project on datrs that already seems to handle sparse data using Vec<Option<Page>>?

https://github.com/datrs/memory-pager/blob/8f3f650af47a0044e56b48d6aeb1037a1a0555c5/src/lib.rs#L24

@khernyo
Copy link
Member Author

khernyo commented Aug 22, 2019

Seems like a good idea. I'm not overly familiar with that project, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants