Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tools] Add a script to move checkpoint files #6143

Merged
merged 5 commits into from
Jun 24, 2024

Conversation

zhangchiqing
Copy link
Member

This PR adds a tool to move checkpoint files. It does both moving and renaming.

See the following example:

root@mainnettest-24-execution:/var/flow# bash move-checkpoint.sh test/checkpoint.001 test/root.checkpoint
Moved: /var/flow/test/checkpoint.001 -> /var/flow/test/root.checkpoint
Moved: /var/flow/test/checkpoint.001.000 -> /var/flow/test/root.checkpoint.000
Moved: /var/flow/test/checkpoint.001.001 -> /var/flow/test/root.checkpoint.001
Moved: /var/flow/test/checkpoint.001.002 -> /var/flow/test/root.checkpoint.002
Moved: /var/flow/test/checkpoint.001.003 -> /var/flow/test/root.checkpoint.003
Moved: /var/flow/test/checkpoint.001.004 -> /var/flow/test/root.checkpoint.004
Moved: /var/flow/test/checkpoint.001.005 -> /var/flow/test/root.checkpoint.005
Moved: /var/flow/test/checkpoint.001.006 -> /var/flow/test/root.checkpoint.006
Moved: /var/flow/test/checkpoint.001.007 -> /var/flow/test/root.checkpoint.007
Moved: /var/flow/test/checkpoint.001.008 -> /var/flow/test/root.checkpoint.008
Moved: /var/flow/test/checkpoint.001.009 -> /var/flow/test/root.checkpoint.009
Moved: /var/flow/test/checkpoint.001.010 -> /var/flow/test/root.checkpoint.010
Moved: /var/flow/test/checkpoint.001.011 -> /var/flow/test/root.checkpoint.011
Moved: /var/flow/test/checkpoint.001.012 -> /var/flow/test/root.checkpoint.012
Moved: /var/flow/test/checkpoint.001.013 -> /var/flow/test/root.checkpoint.013
Moved: /var/flow/test/checkpoint.001.014 -> /var/flow/test/root.checkpoint.014
Moved: /var/flow/test/checkpoint.001.015 -> /var/flow/test/root.checkpoint.015
Moved: /var/flow/test/checkpoint.001.016 -> /var/flow/test/root.checkpoint.016
Files moved successfully.

@codecov-commenter
Copy link

codecov-commenter commented Jun 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 41.63%. Comparing base (8041951) to head (cebdce6).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6143      +/-   ##
==========================================
- Coverage   41.63%   41.63%   -0.01%     
==========================================
  Files        1973     1973              
  Lines      139355   139355              
==========================================
- Hits        58026    58023       -3     
+ Misses      75313    75311       -2     
- Partials     6016     6021       +5     
Flag Coverage Δ
unittests 41.63% <ø> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

fxamacker

This comment was marked as resolved.

@zhangchiqing
Copy link
Member Author

Oh, good catch

tools/move-checkpoint.sh Outdated Show resolved Hide resolved
Copy link
Member

@fxamacker fxamacker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Just ideas (feel free to ignore):

  • maybe add a flag to use mv -i (for extra safety when used interactively)
  • maybe use sha256sum in the script to confirm all the files didn't get modified/corrupted

@zhangchiqing zhangchiqing added this pull request to the merge queue Jun 24, 2024
@zhangchiqing zhangchiqing removed this pull request from the merge queue due to a manual request Jun 24, 2024
@zhangchiqing
Copy link
Member Author

zhangchiqing commented Jun 24, 2024

maybe add a flag to use mv -i (for extra safety when used interactively)

Good point. I added a --run flag, to actually run the mv command, without this flag would be a preview of the files to be moved.

maybe use sha256sum in the script to confirm all the files didn't get modified/corrupted

I'd like to keep the script simple for moving files around, the checksum should be done by separate tools.

@zhangchiqing zhangchiqing added this pull request to the merge queue Jun 24, 2024
Merged via the queue into master with commit 3126f9a Jun 24, 2024
55 checks passed
@zhangchiqing zhangchiqing deleted the leo/move-checkpoint-script branch June 24, 2024 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants