Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add leading zeros to adler32 checksum to ensure 8 chars length #9809

Merged
merged 2 commits into from
Jul 8, 2020

Conversation

amaltaro
Copy link
Contributor

@amaltaro amaltaro commented Jul 8, 2020

Fixes #9763

Status

not-tested

Description

Apparently there are cases that adler32 checksum does not result in 8 chars length word. This fix will pad it with zeros and ensure it's always a 8 chars string.

Note: for files already

Is it backward compatible (if not, which system it affects?)

yes

Related PRs

none

External dependencies / deployment changes

none

@cmsdmwmbot
Copy link

Jenkins results:

  • Unit tests: failed
    • 1 tests added
  • Pylint check: succeeded
    • 6 comments to review
  • Pycodestyle check: succeeded
    • 2 comments to review
  • Python3 compatibility checks: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/10249/artifact/artifacts/PullRequestReport.html

fix unit tests
@cmsdmwmbot
Copy link

Jenkins results:

  • Unit tests: succeeded
    • 1 tests added
  • Pylint check: succeeded
    • 6 comments to review
  • Pycodestyle check: succeeded
    • 2 comments to review
  • Python3 compatibility checks: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/10252/artifact/artifacts/PullRequestReport.html

Copy link
Contributor

@todor-ivanov todor-ivanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@amaltaro
Copy link
Contributor Author

amaltaro commented Jul 8, 2020

Just a note, if the checksum and adler32 is calculated in chunks of 2 ** 15 instead of 2 ** 12, it gives us a speedup by a factor of 4. This for a file of 1.7GB, so bigger files would provide us with even a better speed up.
I'm just not sure whether this can give us unwanted side effects...

@amaltaro
Copy link
Contributor Author

amaltaro commented Jul 8, 2020

New issue created in case we want to give further thought to this better performance in calculating the checksums:
#9811

@amaltaro amaltaro merged commit f5d3987 into dmwm:master Jul 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

WMAgent fails to inject some files into rucio
3 participants