Ctrl-C while downloading HSC data in jupyter notebook produced unexpected stack trace

[drewoldag]

Attempting to download a small set of HSC data using smalhsccat_z_0_025.fits in a jupyter notebook so that I can experiment with HSC train/validate data.

[2024-12-02 11:09:19,885 fibad.download:INFO] Download command Start
[2024-12-02 11:09:19,886 fibad.download:INFO] Reading in fits catalog: /Users/drew/code/fibad/docs/notebooks/smallhsccat_z_0_025.fits
[2024-12-02 11:09:20,015 fibad.download:INFO] Downloading cutouts to /Users/drew/code/fibad/docs/notebooks/data
[2024-12-02 11:09:20,015 fibad.download:INFO] Making a list of cutouts...
[2024-12-02 11:10:48,341 fibad.download:INFO] Checking the list against currently downloaded cutouts...
[2024-12-02 11:10:48,343 fibad.download:INFO] Only 3703395 sky locations, which is less than the number of threads, so we will use only one thread.
[2024-12-02 11:10:48,343 fibad.download:INFO] Dividing 3703395 sky locations among 1 threads...
[2024-12-02 11:10:48,343 fibad.download:INFO] Started 1 request threads
[2024-12-02 11:10:48,344 fibad.download:INFO] Thread 0 starting download of 3703395 rects
[2024-12-02 11:13:21,264 fibad.download:INFO] Assembling download manifest
[2024-12-02 11:13:21,332 fibad.download:INFO] Writing out download manifest with 0 entries.
[2024-12-02 11:13:21,519 fibad.download:INFO] Finished writing download manifest

---

KeyboardInterrupt Traceback (most recent call last)
Cell In[2], line 1
----> 1 fibad_instance.download()

File ~/code/fibad/src/fibad/fibad.py:171, in Fibad.download(self, **kwargs)
168 from .download import Downloader
170 downloader = Downloader(config=self.config)
--> 171 return downloader.run(**kwargs)

File ~/code/fibad/src/fibad/download.py:155, in Downloader.run(self)
153 logger.info(f"Started {len(download_threads)} request threads")
154 [thread.start() for thread in download_threads]
--> 155 [thread.join() for thread in download_threads]
156 finally: # Ensure manifest is written even when we get a KeyboardInterrupt during download
157 self._write_manifest()

File ~/code/fibad/src/fibad/download.py:155, in (.0)
153 logger.info(f"Started {len(download_threads)} request threads")
154 [thread.start() for thread in download_threads]
--> 155 [thread.join() for thread in download_threads]
156 finally: # Ensure manifest is written even when we get a KeyboardInterrupt during download
157 self._write_manifest()

File ~/opt/miniconda3/envs/fibad/lib/python3.11/threading.py:1119, in Thread.join(self, timeout)
1116 raise RuntimeError("cannot join current thread")
1118 if timeout is None:
-> 1119 self._wait_for_tstate_lock()
1120 else:
1121 # the behavior of a negative timeout isn't documented, but
1122 # historically .join(timeout=x) for x<0 has acted as if timeout=0
1123 self._wait_for_tstate_lock(timeout=max(timeout, 0))

File ~/opt/miniconda3/envs/fibad/lib/python3.11/threading.py:1139, in Thread._wait_for_tstate_lock(self, block, timeout)
1136 return
1138 try:
-> 1139 if lock.acquire(block, timeout):
1140 lock.release()
1141 self._stop()

KeyboardInterrupt: