Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optional single-threaded "slow USB stick" mode? #151

Open
nekohayo opened this issue Nov 4, 2024 · 2 comments
Open

Optional single-threaded "slow USB stick" mode? #151

nekohayo opened this issue Nov 4, 2024 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@nekohayo
Copy link

nekohayo commented Nov 4, 2024

Some (most!) USB sticks I encounter, especially the ones given out for free as promotional conference swag, are garbage in terms of performance.

From what I am seeing with the 3.0.1 version of Raider from flathub, running on top of Fedora 41… throwing a dozen folders (with lots of small files and a couple of big files) at it is/feels extremely slow (maybe because the controller is bad, or the flash is just slow, or filesystem fragmentation, or…). For example, one of the files that have been taking the longest to shred, which took about an hour (if not more) while other files were also being slowly shredded, was a simple 25 megabytes file; you'd think that it wouldn't take that long if it was just that file being shredded.

I have a suspicion (just based on intuition) that in such cases it might be slower because it's trying to process 3 files simultaneously at all times, instead of one file at a time, so the flash drive is experiencing competing rewrites in the least efficient way, resources-starving it.

Although I'm not a hardware/filesystem expert, I think there may be such cases (also some SD cards, MTP and remote filesystems if this apps supports it, etc.) where processing only one file at a time "per device/filesystem" would end up with a faster total processing time (and if it's within the same physical device, I don't really see how parallelizing the shredding would give better performance in most cases?). I'm throwing the idea out here for your consideration.

@ADBeveridge
Copy link
Owner

Yes you are correct; the multithreaded shredding was for SSDs/usb3 disks. USB2 has a smaller queue depth (not sure by how much) than USB3 so processing 1 file at a time would result in faster performance in this case. I'll look into writing a parallelization limiter based on the storage medium of the file(s). It make take a bit now that I'm in college, but I'll definitely be working on it.

@ADBeveridge ADBeveridge added the enhancement New feature or request label Nov 5, 2024
@ADBeveridge ADBeveridge self-assigned this Nov 5, 2024
@ADBeveridge
Copy link
Owner

FYI I might not find time for this till June when i'm on summer break.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants