Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Having trouble to download the data from huggingface #7

Open
ghost opened this issue Jun 22, 2024 · 0 comments
Open

Having trouble to download the data from huggingface #7

ghost opened this issue Jun 22, 2024 · 0 comments

Comments

@ghost
Copy link

ghost commented Jun 22, 2024

Hi !

I just tried to download Real-IAD.

But an error occurred constantly..

the error is..

  File "/home/jisu/anaconda3/lib/python3.9/site-packages/huggingface_hub/hf_file_system.py", line 638, in get_file
    http_get(
  File "/home/jisu/anaconda3/lib/python3.9/site-packages/huggingface_hub/file_download.py", line 570, in http_get
    raise EnvironmentError(
OSError: Consistency check failed: file should be of size 23215841838 but has size 10001900996 ((…)f5878d/realiad_raw/rolled_strip_base.zip).
We are sorry for the inconvenience. Please retry with `force_download=True`.
If the issue persists, please let us know by opening an issue on https://github.com/huggingface/huggingface_hub.

The code I used is 

from datasets import load_dataset
# Real-IAD/Real-IAD 데이터셋 로드 및 다운로드
dataset = load_dataset("Real-IAD/Real-IAD", cache_dir="/SSD2/Datasets/RealIAD")

I downloaded the data above amount.

Resolving data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 123/123 [00:00<00:00, 477.45it/s]
Downloading data:  77%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████▏                                | 93/121 [1:51:55<5:05:29, 654.63s/files]
Downloading data:  87%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎                  | 105/121 [7:30:11<1:08:36, 257.25s/files]

Any solution??

thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants