103 imgw hourly year 2024

[bczernecki]

Closing #103 #102

[codecov]

Codecov Report

Attention: Patch coverage is 94.64286% with 3 lines in your changes missing coverage. Please review.

Project coverage is 87.85%. Comparing base (9f7e5b9) to head (79108d4).
Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
R/meteo_imgw_hourly.R	88.23%	2 Missing ⚠️
R/meteo_imgw_daily.R	97.43%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #104      +/-   ##
==========================================
+ Coverage   87.83%   87.85%   +0.02%     
==========================================
  Files          29       29              
  Lines        1619     1639      +20     
==========================================
+ Hits         1422     1440      +18     
- Misses        197      199       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Nowosad]

@bczernecki, I've tried the code below, and one of the messages states that one of the files is skipped. I was able to download and unzip the file by hand and it contains some data. Is skipping the best approach here?

war_fil = meteo_imgw_hourly(
  rank = "climate",
  year = 2024,
  station = "WARSZAWA-FILTRY")

/tmp/Rtmp4aviDh/file6677103f94e2.zip
Detected problems in: https://danepubliczne.imgw.pl/data/dane_pomiarowo_obserwacyjne/dane_meteorologiczne/terminowe/klimat/2024/2024_07_k.zip - skipping

[bczernecki]

@Nowosad I tried to download this file in R and unzip it internally, but it fails. It also fails on MacOS if I download it manually, so couldn't find a robust solution, but I am open to any suggestions

[Nowosad]

Ok -- I see it from R (I can unzip it without any warnings/issues in Fedora):

unzip("2024_07_k.zip", exdir = "2024_07_k")
Warning message:
In unzip("2024_07_k.zip", exdir = "2024_07_k") : zip file is corrupt

I do not have any better universal ideas on what to do -- except maybe using warnings in such cases instead of messages?

[bczernecki]

@Nowosad - Yea.. I've contacted IMGW to let them know that the ZIP files are corrupted and do not work on all modern OS. The main reason is not consistent ZIP version usage. This is how it looks for Linux:

> system("zipinfo 2024_05_k.zip")
Archive:  2024_05_k.zip
Zip file size: 15992 bytes, number of entries: 1
-rw-a--     6.3 fat   145234 bx defN 24-Jul-10 14:03 k_d_05_2024.csv
1 file, 145234 bytes uncompressed, 15828 bytes compressed:  89.1%
> system("zipinfo 2024_07_k.zip")
Archive:  2024_07_k.zip
Zip file size: 19823 bytes, number of entries: 1
-rw-a--     2.0 fat   187213 tx defN 24-Nov-12 12:29 k_d_07_2024.csv
1 file, 187213 bytes uncompressed, 19659 bytes compressed:  89.5%

So sometimes they use ZIP 2.0 and sometimes 6.3. New files will be uploaded only in the modern ZIP standard, but the problematic ones won't be modified.

Maybe let's keep this ticket open for some time and then we can decide whether we want to add these files internally in the package or create package environment that will keep logging which files failed and then provide summary while finishing downloading 🤔