Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use overlayfs for storage instead of btrfs #85

Closed
tych0 opened this issue Apr 20, 2020 · 4 comments
Closed

use overlayfs for storage instead of btrfs #85

tych0 opened this issue Apr 20, 2020 · 4 comments

Comments

@tych0
Copy link
Collaborator

tych0 commented Apr 20, 2020

We should use overlayfs (with a writable upperdir= for storing modifications) instead of btrfs. That is, we should build a temporary filesystem up atomfs style, with each layer in its own directory. This will speed things up in several ways:

  1. we don't need do do go-mtree any more (although this will take some work in umoci to get right, since it computes fs diffs via mtree). the idea here is that we just wouldn't have to compute deltas any more, since each layer would be in its own directory
  2. we don't take two round trips through the vfs layer with the btffs backend
@smoser
Copy link
Contributor

smoser commented Apr 20, 2020

well, how would you do that with non-root?

@tych0
Copy link
Collaborator Author

tych0 commented Apr 20, 2020

On Ubuntu you can do it, but with the upstream kernel you can't. I think the speed savings would probably be worth it for non-patched kernels, though.

@cyphar
Copy link

cyphar commented Jun 15, 2020

You could generate a fake set of mtree deltas, though it might be simpler to just copy the core bits of the diff layer generation yourself to avoid going through mtree data structures at all.

My long-term goal is to solve this problem with hooks (see openSUSE/umoci#325, though I have discussed this for a while) such that all of the core steps of unpacking and repacking are hookable (so that you can bring-your-own storage and diff generation setup), and that feature would be available from the umoci API. But I do appreciate that you might view that as a layer violation of some kind -- you'd need to implement the storage side of things as an "umoci hook". But for Go users we could implement it as a function callback so you don't need to implement it as a binary.

@tych0
Copy link
Collaborator Author

tych0 commented Sep 14, 2020

I believe this is all implemented now.

@tych0 tych0 closed this as completed Sep 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants