-
Notifications
You must be signed in to change notification settings - Fork 3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
execute batch downloads in parallel worker threads
- limit downloads to 10 at a time instead of starting all at once - use more specific types for BatchDownloader#__call__ - add NEWS - calculate byte lengths with a HEAD request - add cli arg to limit download parallelism - initial implementation of pooled progress bar - pooled progress bar looks very nice - factor out receiving thread exceptions into a contextmanager - default batch parallelism to 10 - quiet all progress output from -q - don't write colored output with --no-color - make batch download parallelism 1 in html index test - write a lot more documentation for the new progress bar logic - use ProgressBarType enum everywhere
- Loading branch information
1 parent
858a515
commit e5d75de
Showing
14 changed files
with
818 additions
and
98 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Download concrete dists for metadata-only resolves in parallel using worker threads. Add ``--batch-download-parallelism`` CLI flag to limit parallelism. Use ``ProgressBarType`` enum class for ``--progress-bar`` choices. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.