Bug: Creating a Job with thousands of tiny input files will fail the syncInputJobAttachments step

[YutongLi291]

Describe the bug

When submitting any job to a CMF with worker agent installed, with thousands (10000 as tested) of small 1 byte input files, after the job starts, the syncInputJobAttachments step will fail most of the time, with an error message"Input Syncing failed due to successive low transfer rates"

Expected Behaviour

The job should successfully sync all of the input job attachments.

Current Behaviour

The Sync attachments step fails with "Input Syncing failed due to successive low transfer rates"

Reproduction Steps

Submit any job with 10000 1 byte files. Check the sync attachments step after the job has been submitted.

Environment

At minimum:

1. Operating system: Amazon Linux 2023 and Windows Server 2022
2. Output of python3 --version: Python 3.12.3
3. Output of deadline --version: deadline, version 0.48.8
4. If this is from a version installed by the Deadline Cloud Submitter installer, then what version of the submitter installer?
   No
   Please share other details about your environment that you think might be relevant to reproducing the bug.