Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce user quotas for temporary storage #209

Open
slifty opened this issue Aug 3, 2023 · 0 comments
Open

Enforce user quotas for temporary storage #209

slifty opened this issue Aug 3, 2023 · 0 comments

Comments

@slifty
Copy link
Contributor

slifty commented Aug 3, 2023

Users only have a certain amount of space in Permanent, but the current SFTP implementation does not enforce that storage until the point of upload to S3 (which doesn't happen until the file has been fully uploaded to the sftp service).

This means someone with 4GB of storage on their account might attempt to upload a 100GB file, but would not be informed of the failure until after all 100GB had been "transferred" (as opposed to the moment they hit 4.000001 GB in a given upload).

In addition to being a lesser user experience, this is also a vulnerability since it means that anybody could upload ANY sized file and that file would take up temporary storage space (which has a cost, of course, in addition to being a limited resource currently as described in #145).

We should have the SFTP service track how much data is "in flight" for a given user, and enforce restrictions. This becomes a much more complex problem in a world where there are multiple instances of the SFTP service (since the services would need to somehow be aware of one another's activities, and we currently are using in-memory data stores instead of a centralized datastore such as redis).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant