Bug: Ctrl+C is not handled properly (or: how to pause the download before it is resumed?)

[rwiesbach]

Hi,

on Ctrl+C I and likely most users would expect a graceful close (like end all current transfers and cleanly exit), especially as you highlight the ability to resume an interrupted download.

If I do Ctrl+C during download in hibp-downloader it shows me Tracebacks, so unhandled excepctions.
It may end the current transfers properly/gracefully or not (the python.exe subprocesses are terminated, but I don't know whether this is done gracefully), but the main process is not terminated (until I do manually).

Given this unclean kind of termination I don't know whether I can be sure that after the resume all downloads of ranges are continued properly (or whether problems may occur like incomplete .gz files on disk that hibp-downlaoder may consider as downloaded just because the .gz exists even though it is incomplete)

Thanks

(Python 3.12.5 on Windows 10)

[ndejong]

Thanks @github-staff in nixing the malware pusher comment above

[ndejong]

@rwiesbach thanks for commenting on this.

I'd love to be able to fix this unclean shutdown issue, have taken a look at it a few times and can't really figure out how to best terminate the threads and queues cleanly - if you have suggestions please do make them.

With respect to it being clean when you restart - then yes it is clean even though the CTL-C shutdown looks messy - this is because the metadata file is only written after the sucessful download of the datafile - there maybe a race condition in-between writing the metadata file I guess.

[rwiesbach]

I'd love to be able to fix this unclean shutdown issue, have taken a look at it a few times and can't really figure out how to best terminate the threads and queues cleanly - if you have suggestions please do make them.

In code or as a concept? with no python expertise I'd use threads instead of processes and a global variable that is only written by the monitor but checked by the workers. every n iteration or every n seconds the workers check this variable and if it has changed they terminate cleanly (end the current download and exit). if all workers are down, the monitor (/main) can terminate, too.

[ndejong]

Yes, the concept is fine - it's putting just that into practice in Python code where things get sideways in a way I can't see how to resolve.

Open call to anyone that can cleanly sort this out - would be awesome