Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimizations and refactoring #35

Merged
merged 12 commits into from
Oct 21, 2021
Merged

Optimizations and refactoring #35

merged 12 commits into from
Oct 21, 2021

Conversation

renbou
Copy link
Contributor

@renbou renbou commented Oct 17, 2021

Hi! Loved the idea of this tool, so I decided to contribute some improvements:

  • Some simple refactoring
  • Fixed bug where content-disposition name wouldn't actually be used for d.name (since d.name was set only once, then even if Info returned a valid name, it wouldn't be used in the resulting filepath)
  • Removed usage of redundant HEAD request to determine range support, now only GET is used since its more reliable and makes the code simpler
  • Optimized the concurrent file writing by a lot, now all goroutines write to a single file concurrently using WriteAt (which is documented to be concurrent), so no temporary files are used and no merge is needed, again simplifying the code

Not sure why, but the test workflows on go 1.14 windows are sometimes not passing (even though they are marked as ok, some unrelated errors are thrown by go vet, which makes the github CI workflow get marked as failed)

Didn't measure the optimizations on a server with good internet, but my local testing seems to save around 3+ seconds each time since we avoid file copying

renbou and others added 12 commits October 17, 2021 16:12
Previously the file download file name from content-disposition wouldn't be actually used,
since d.name is set once during the first call to Name(), which is before the update using
content-disposition. Now it works correctly (renamed d.name to d.path to signify that its the whole path)
Refactor
Remove redundant download when no range support is available, since we always
download during init anyways.
Concurrently write directly to single file using WriteAt instead of writing to multiple files and then copying
Complete optimization
@renbou
Copy link
Contributor Author

renbou commented Oct 17, 2021

The optimizations done here (concurrently writing to single file) also fix what has been mentioned in #32 (slow writing to disk), since now the fastest way possible is used

@renbou
Copy link
Contributor Author

renbou commented Oct 17, 2021

Also fixes all problems described in #29!

@renbou
Copy link
Contributor Author

renbou commented Oct 19, 2021

image
This is what I was talking about - the tests pass as OK, but the action fails. Maybe it's time to update the actions to run tests on 1.16 and 1.17?

@melbahja
Copy link
Owner

it's fails only on 1.14.x windows, i think the tests gonna pass on 1.16 and 1.17

@melbahja
Copy link
Owner

Thank you for your valuable pr

@melbahja melbahja merged commit e30ed52 into melbahja:master Oct 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants