Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error when downloading TinyTaskonomy #65

Open
AvivSham opened this issue Nov 1, 2022 · 0 comments
Open

Error when downloading TinyTaskonomy #65

AvivSham opened this issue Nov 1, 2022 · 0 comments

Comments

@AvivSham
Copy link

AvivSham commented Nov 1, 2022

Hi All,
I followed the README file and run the following command after succecfully installing aria2 and omnidata-tools.
It seems like for some files the connection to the server is broken.
Here is the trace:

[DL:97MiB][#dcfcb1 0B/7.4GiB(0%)][#a82352 450MiB/821MiB(54%)][#d9c894 335MiB/2.4GiB(13%)][#c72378 265MiB/1.1GiB(21%)]
11/01 17:36:58 [NOTICE] Download complete: compressed//segment_unsup25d__taskonomy__almena.tar
 *** Download Progress Summary as of Tue Nov  1 17:38:32 2022 ***
=====================================================================================================================
[#dcfcb1 0B/7.4GiB(0%) CN:1 DL:0B]
FILE: compressed//rgb__taskonomy__alfred.tar
---------------------------------------------------------------------------------------------------------------------
[#a82352 697MiB/821MiB(84%) CN:16 DL:0B]
FILE: compressed//depth_euclidean__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#d9c894 583MiB/2.4GiB(23%) CN:16 DL:0B]
FILE: compressed//edge_texture__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#c72378 502MiB/1.1GiB(41%) CN:16 DL:0B]
FILE: compressed//keypoints2d__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#abb076 856MiB/1.8GiB(46%) CN:16 DL:0B]
FILE: compressed//keypoints3d__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#5042be 1.1GiB/1.8GiB(60%) CN:16 DL:0B]
FILE: compressed//normal__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#4ec56e 456MiB/1.7GiB(25%) CN:16 DL:0B]
FILE: compressed//principal_curvature__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#c90ad6 358MiB/2.8GiB(12%) CN:16 DL:0B]
FILE: compressed//rgb__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#26a78e 770MiB/1.0GiB(71%) CN:16 DL:0B]
FILE: compressed//reshading__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#34d64e 0B/9.4GiB(0%) CN:1 DL:0B]
FILE: compressed//rgb_large__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#e8a3d4 0B/60MiB(0%) CN:1 DL:0B]
FILE: compressed//segment_unsup2d__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#0efe21 0B/55MiB(0%) CN:1 DL:0B]
FILE: compressed//class_object__taskonomy__almota.tar
---------------------------------------------------------------------------------------------------------------------
[#bb4e30 0B/2.7GiB(0%) CN:1 DL:0B]
FILE: compressed//edge_texture__taskonomy__almota.tar
---------------------------------------------------------------------------------------------------------------------

[DL:0B][#dcfcb1 0B/7.4GiB(0%)][#a82352 697MiB/821MiB(84%)][#d9c894 583MiB/2.4GiB(23%)][#c72378 502MiB/1.1GiB(41%)][#a[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_keypoints3d.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_rgb_large.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_edge_texture.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_depth_euclidean.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almota_edge_texture.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_rgb.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_normal.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_keypoints2d.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almota_class_object.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_reshading.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/alfred_rgb.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_principal_curvature.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_segment_unsup2d.tar (stacktrace below)
multiprocess.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/opt/conda/lib/python3.8/http/client.py", line 1344, in getresponse
    response.begin()
  File "/opt/conda/lib/python3.8/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/opt/conda/lib/python3.8/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/opt/conda/lib/python3.8/socket.py", line 669, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/opt/conda/lib/python3.8/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/opt/conda/lib/python3.8/site-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 451, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 340, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=6800): Read timed out. (read timeout=60.0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.8/site-packages/multiprocess/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/opt/conda/lib/python3.8/site-packages/omnidata_tools/dataset/download.py", line 298, in process_model
    raise e
  File "/opt/conda/lib/python3.8/site-packages/omnidata_tools/dataset/download.py", line 283, in process_model
    tar_fpath = download_tar(
  File "/opt/conda/lib/python3.8/site-packages/omnidata_tools/dataset/download.py", line 161, in download_tar
    success = wait_on(aria2api, res)
  File "/opt/conda/lib/python3.8/site-packages/omnidata_tools/dataset/download.py", line 187, in wait_on
    while not (a2api.get_downloads([gid])[0].is_complete or a2api.get_downloads([gid])[0].has_failed):
  File "/opt/conda/lib/python3.8/site-packages/aria2p/api.py", line 298, in get_downloads
    downloads.append(Download(self, self.client.tell_status(gid)))
  File "/opt/conda/lib/python3.8/site-packages/aria2p/client.py", line 877, in tell_status
    return self.call(self.TELL_STATUS, [gid, keys])  # type: ignore
  File "/opt/conda/lib/python3.8/site-packages/aria2p/client.py", line 262, in call
    return self.res_or_raise(self.post(payload))
  File "/opt/conda/lib/python3.8/site-packages/aria2p/client.py", line 358, in post
    return requests.post(self.server, data=payload, timeout=self.timeout).json()
  File "/opt/conda/lib/python3.8/site-packages/requests/api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/conda/lib/python3.8/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/opt/conda/lib/python3.8/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='localhost', port=6800): Read timed out. (read timeout=60.0)
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/bin/omnitools.download", line 8, in <module>
    sys.exit(download())
  File "/opt/conda/lib/python3.8/site-packages/fastcore/script.py", line 119, in _f
    return tfunc(**merge(args, args_from_prog(func, xtra)))
  File "/opt/conda/lib/python3.8/site-packages/omnidata_tools/dataset/download.py", line 304, in download
    errors = list(tqdm.tqdm(p.imap(process_model, models), total=len(models)))
  File "/opt/conda/lib/python3.8/site-packages/tqdm/_tqdm.py", line 1060, in __iter__
    for obj in iterable:
  File "/opt/conda/lib/python3.8/site-packages/multiprocess/pool.py", line 868, in next
    raise value
requests.exceptions.ReadTimeout: None: None
  1%|▉                                                                          | 137/10600 [05:12<6:37:42,  2.28s/it]
root@a804f8c0dde0:/images#
11/01 17:38:41 [NOTICE] Shutdown sequence commencing... Press Ctrl-C again for emergency shutdown.
[DL:50KiB][#dcfcb1 0B/7.4GiB(0%)][#a82352 697MiB/821MiB(84%)][#d9c894 583MiB/2.4GiB(23%)][#c72378 502MiB/1.1GiB(41%)]
11/01 17:38:44 [NOTICE] Download GID#dcfcb1a904b54c49 not complete: compressed//rgb__taskonomy__alfred.tar

11/01 17:38:44 [NOTICE] Download GID#a82352ebd95c4aec not complete: compressed//depth_euclidean__taskonomy__almena.tar

11/01 17:38:44 [NOTICE] Download GID#d9c89480fd02a3cb not complete: compressed//edge_texture__taskonomy__almena.tar

11/01 17:38:44 [NOTICE] Download GID#c723789b4a0b6657 not complete: compressed//keypoints2d__taskonomy__almena.tar

11/01 17:38:44 [NOTICE] Download GID#abb076fee190d5d8 not complete: compressed//keypoints3d__taskonomy__almena.tar

11/01 17:38:44 [NOTICE] Download GID#5042be4145f66b55 not complete: compressed//normal__taskonomy__almena.tar

11/01 17:38:44 [NOTICE] Download GID#4ec56efd51f8b934 not complete: compressed//principal_curvature__taskonomy__almena.tar

11/01 17:38:44 [NOTICE] Download GID#c90ad6e2c8686ae1 not complete: compressed//rgb__taskonomy__almena.tar

11/01 17:38:44 [NOTICE] Download GID#26a78e95921374b3 not complete: compressed//reshading__taskonomy__almena.tar

Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
dcfcb1|INPR|       0B/s|compressed//rgb__taskonomy__alfred.tar
a82352|INPR|   2.7MiB/s|compressed//depth_euclidean__taskonomy__almena.tar
d9c894|INPR|   2.4MiB/s|compressed//edge_texture__taskonomy__almena.tar
c72378|INPR|   2.4MiB/s|compressed//keypoints2d__taskonomy__almena.tar
abb076|INPR|   2.9MiB/s|compressed//keypoints3d__taskonomy__almena.tar
5042be|INPR|   3.6MiB/s|compressed//normal__taskonomy__almena.tar
4ec56e|INPR|   2.5MiB/s|compressed//principal_curvature__taskonomy__almena.tar
c90ad6|INPR|   2.4MiB/s|compressed//rgb__taskonomy__almena.tar
26a78e|INPR|   2.8MiB/s|compressed//reshading__taskonomy__almena.tar

Status Legend:
(INPR):download in-progress.

aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page for details.

For example #dcfcb1 remains on 0MB while other files been downloaded. I must say I tried to download the dataset on multiple machines and still same error came up.

Please help.
@alexsax
@amir32002
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@AvivSham