Skip to content

Commit

Permalink
docs: Add note about deduplicators like jdupes
Browse files Browse the repository at this point in the history
  • Loading branch information
taoky committed Aug 25, 2024
1 parent d36e46d commit 8a8a2ff
Showing 1 changed file with 10 additions and 0 deletions.
10 changes: 10 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -219,6 +219,16 @@ let relative_filepath = relative_filepath.to_string_lossy();

You might see arguments like `--exclude debian/ --include debian/dists/${DEBIAN_CURRENT}`, with trailing slash exclusion in examples. This is just because we don't need to exclude directory listing of `debian` folder out.

### Deduplication

Tsumugu relies on local file size and mtime to check if file shall be downloaded. Some file-level deduplicators like [jdupes](https://codeberg.org/jbruchon/jdupes) would ignore file mtime when deduplicating with hard links. This could be an issue for some repos, as some files would be redownloaded again and again every time as it does not have a correct mtime locally.

Workarounds:

- Set `--compare-size-only`.
- Use filesystem-level/block-level deduplication like `zfs dedup`.
- Use another file-level deduplicator which considers mtime (though I don't know which would do this).

## Naming

The name "tsumugu", and current branch name "pudding", are derived from the manga *A Drift Girl and a Noble Moon*.
Expand Down

0 comments on commit 8a8a2ff

Please sign in to comment.