zipfile probably corrupt (segmentation violation)

[edsu]

Have you ever run across an error like this when doing an unzip.sh? I saw it happen for two different days:

(pillbox)ubuntu@ip-10-39-110-115:~/pillbox-data-process/scripts$ ./unzip.sh 2013-05-23
processing...
removed /tmp-original files
removed /tmp-unzipped files
removed /tmp-images files
original files unzipped to /tmp-original
error:  zipfile probably corrupt (segmentation violation)
rm: cannot remove ‘../tmp-images/HRX//.xml’: No such file or directory

[ghost]

Have never seen this error. I didn't run the 2014-05-23 data, which is still current. Processing it now. Do your checksums match? The script doesn't check.

dm_spl_release_human_rx_part1.zip: 32752658516586046eed17c38d9ca938
dm_spl_release_human_rx_part2.zip: bfb71e28776d2ff4105e33d50f4f1d44
dm_spl_release_human_otc_part1.zip: a16df391d4f8a964abd72b0c69f39f07
dm_spl_release_human_otc_part2.zip: 138d36ed51abeeb29fa1819bb6b33cd4
dm_spl_release_human_otc_part3.zip: 54a374cd8969a845b551e6deedcc7b5d
dm_spl_release_homeopathic.zip: 5bc1654ba9179df85fd06375eac11c0e
dm_spl_release_animal.zip: 02892c10b685bff488a18a0a14040641
dm_spl_release_remainder.zip: c6bd517581fbffd532cc4de9349b627a

Edit: I ran the 2014-05-23 data and did not encounter that error.

[smit1678]

@edsu I haven't seen that either but I'm guessing this was due to a bad zipfile. What version of ubuntu are you running on?

[edsu]

Ubuntu 13.10

I did try multiple days, and the same thing happened. Perhaps it was the same file in each run that caused the problem? It's hard to say without adding some more diagnostic information. Since @lostonroute66 was able to run 2014-05-23 I will try it again and see what happens.

[edsu]

If checksums are available it would be good to check them once a file has been downloaded. Also, the download/unzip bash scripts would probably be a bit easier to maintain if they were written in Python :-)

[ghost]

As far as I know the checksums are only available in the table on the DailyMed download page. I'll ask the DM team about this. It would be best for the script to be able to automatically verify the checksum.