Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feat: Check if item version is already preserved before bagging (Issue #102) #103

Merged
merged 61 commits into from
Sep 16, 2024
Merged

Feat: Check if item version is already preserved before bagging (Issue #102) #103

merged 61 commits into from
Sep 16, 2024

Commits on Jun 5, 2024

  1. Check if bag exists in AP Trust and Wasabi before creating the bag

    adepojuh authored and adepojuh committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    4421acf View commit details
    Browse the repository at this point in the history

Commits on Jun 10, 2024

  1. Document all functions in figshare/Utils.py

    [email protected] authored and [email protected] committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    a76146e View commit details
    Browse the repository at this point in the history

  2. Check if bag exists in AP Trust and Wasabi before creating the bag

    @HafeezOJ
    HafeezOJ committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    89eaea5 View commit details
    Browse the repository at this point in the history

  3. Document all functions in figshare/Utils.py

    @HafeezOJ
    HafeezOJ committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    eb5685d View commit details
    Browse the repository at this point in the history

  4. Merge branch '102-feature-check-if-bag-exist-in-ap-trust-prior-to-bag… 

    …ging'
    @HafeezOJ
    HafeezOJ committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    6315c67 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Count and log already preserved collections

    @HafeezOJ
    HafeezOJ committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    283793e View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Include AP Trust details in .env.sample.ini

    @HafeezOJ
    HafeezOJ committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    25bcc88 View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2024

  1. Include aptrust_api configuration in How to run section of README.md

    @HafeezOJ
    HafeezOJ committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    1f24c30 View commit details
    Browse the repository at this point in the history

  2. Correct preservation_storage_location in README.md

    @HafeezOJ
    HafeezOJ committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    77595ca View commit details
    Browse the repository at this point in the history

  3. Lint

    @HafeezOJ
    HafeezOJ committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    719b8ab View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. Skip authors list in metadata sorting

    @HafeezOJ
    HafeezOJ committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    8431f4c View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. Remove logging for articles not found in Wasabi or AP Trust

    @HafeezOJ
    HafeezOJ committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    32cc1b1 View commit details
    Browse the repository at this point in the history

  2. Keep counts of skipped articles and versions in a class dict

    @HafeezOJ
    HafeezOJ committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    d35d11a View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. Fix counts of articles in log

    @HafeezOJ
    HafeezOJ committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    743fb9a View commit details
    Browse the repository at this point in the history

  2. Make a function to get preserved packages filenames and their sizes f… 

    …rom Wasabi
    @HafeezOJ
    HafeezOJ committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    db084a9 View commit details
    Browse the repository at this point in the history

  3. Make check_wasabi() return a tuple containing hash and size in Utils.py

    @HafeezOJ
    HafeezOJ committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    7c87657 View commit details
    Browse the repository at this point in the history

  4. Get wasabi_preserved_size from check_wasabi() in Article.py

    @HafeezOJ
    HafeezOJ committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    c86a79c View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. Document get_filenames_and_sizes_from_ls() in Utils.py

    @HafeezOJ
    HafeezOJ committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    d1e86b7 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. Create a function calculate size of UAL_RDM in Utils.py

    @HafeezOJ
    HafeezOJ committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    6270394 View commit details
    Browse the repository at this point in the history

  2. Create a function to calculate Figshare json file size in Utils.py

    @HafeezOJ
    HafeezOJ committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    320f8a4 View commit details
    Browse the repository at this point in the history

  3. Create a function to calculate payload size in Utils.py

    @HafeezOJ
    HafeezOJ committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    fc3c6b8 View commit details
    Browse the repository at this point in the history

  4. Correct variable names in calculate_payload_size and calculate_json_f… 

    …ile_size
    @HafeezOJ
    HafeezOJ committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    0a5833d View commit details
    Browse the repository at this point in the history

  5. Return AP Trust payload size in get_preserved_version_hash_and_size

    @HafeezOJ
    HafeezOJ committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    bbcb704 View commit details
    Browse the repository at this point in the history

  6. Remove unused imports from and variables in Utils.py

    @HafeezOJ
    HafeezOJ committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    23a582d View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. Log size mismatch if any for already preserved article

    @HafeezOJ
    HafeezOJ committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    c81055a View commit details
    Browse the repository at this point in the history

  2. Use tempfile in place of buffer folder creation

    @HafeezOJ
    HafeezOJ committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    5e8596f View commit details
    Browse the repository at this point in the history

  3. Fix sorting where there are no uniform fields in nested dicts

    @HafeezOJ
    HafeezOJ committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    bb558f9 View commit details
    Browse the repository at this point in the history

  4. Include recursive sort of nested lists

    @HafeezOJ
    HafeezOJ committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    c756c88 View commit details
    Browse the repository at this point in the history

  5. Initialize already_preserved_counts_dict items at class level

    @HafeezOJ
    HafeezOJ committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    0b13234 View commit details
    Browse the repository at this point in the history

  6. Avoid unsupported dict < dict in recursive sorting

    @HafeezOJ
    HafeezOJ committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    73550ea View commit details
    Browse the repository at this point in the history

  7. Add more to AP Trust config

    @HafeezOJ
    HafeezOJ committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    377042a View commit details
    Browse the repository at this point in the history

Commits on Aug 31, 2024

  1. Paginate calls to AP Trust

    @HafeezOJ
    HafeezOJ committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    415866d View commit details
    Browse the repository at this point in the history

  2. Remove duplicate log messages

    @HafeezOJ
    HafeezOJ committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    0e825a4 View commit details
    Browse the repository at this point in the history

  3. Document new functions in Utils.py

    @HafeezOJ
    HafeezOJ committed Aug 31, 2024
    Configuration menu
    Copy the full SHA
    3be308a View commit details
    Browse the repository at this point in the history

Commits on Sep 1, 2024

  1. Fix summary counts for collections

    @HafeezOJ
    HafeezOJ committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    dffc1c7 View commit details
    Browse the repository at this point in the history

  2. Use set to keep articles skipped in Article.py

    @HafeezOJ
    HafeezOJ committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    f03dbe4 View commit details
    Browse the repository at this point in the history

  3. Remove unused lines

    @HafeezOJ
    HafeezOJ committed Sep 1, 2024
    Configuration menu
    Copy the full SHA
    b84f6bb View commit details
    Browse the repository at this point in the history

Commits on Sep 2, 2024

  1. Document skipping of already preserved article

    @HafeezOJ
    HafeezOJ committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    457f9a3 View commit details
    Browse the repository at this point in the history

  2. Document skipping of already preserved collection

    @HafeezOJ
    HafeezOJ committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    20388fc View commit details
    Browse the repository at this point in the history

  3. Remove unused lines

    @HafeezOJ
    HafeezOJ committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    22826d5 View commit details
    Browse the repository at this point in the history

  4. Remove item_type from AP Trust config

    @HafeezOJ
    HafeezOJ committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    7e17595 View commit details
    Browse the repository at this point in the history

  5. Document additional config for AP Trust in README

    @HafeezOJ
    HafeezOJ committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    2b35118 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. Remove Pycharm-specific import format for multi-level folders

    @HafeezOJ
    HafeezOJ committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    d2191fa View commit details
    Browse the repository at this point in the history

  2. Lint

    @HafeezOJ
    HafeezOJ committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    24d3559 View commit details
    Browse the repository at this point in the history

  3. Lint

    @HafeezOJ
    HafeezOJ committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    b8e3e9e View commit details
    Browse the repository at this point in the history

Commits on Sep 4, 2024

  1. Correct AP Trust url in README

    @HafeezOJ
    HafeezOJ committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    2600e3b View commit details
    Browse the repository at this point in the history

  2. Remove PyCharm auto-import style

    @HafeezOJ
    HafeezOJ committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    4c2225e View commit details
    Browse the repository at this point in the history

  3. Fix strings that broken into newlines in the codes

    @HafeezOJ
    HafeezOJ committed Sep 4, 2024
    Configuration menu
    Copy the full SHA
    6047346 View commit details
    Browse the repository at this point in the history

Commits on Sep 5, 2024

  1. Align log count figures

    @HafeezOJ
    HafeezOJ committed Sep 5, 2024
    Configuration menu
    Copy the full SHA
    bc6c40f View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2024

  1. Reorder log messages

    @HafeezOJ
    HafeezOJ committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    24c00f3 View commit details
    Browse the repository at this point in the history

  2. Remove warning when collections are skipped and fetched/processed cou… 

    …nts match
    @HafeezOJ
    HafeezOJ committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    f10c8e7 View commit details
    Browse the repository at this point in the history

  3. Append / to AP Trust base url if not present

    @HafeezOJ
    HafeezOJ committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    5f4e4bc View commit details
    Browse the repository at this point in the history

  4. Lint

    @HafeezOJ
    HafeezOJ committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    a5d563b View commit details
    Browse the repository at this point in the history

  5. Lint

    @HafeezOJ
    HafeezOJ committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    30bfef6 View commit details
    Browse the repository at this point in the history

  6. Align log figures based on standard Linux terminal

    @HafeezOJ
    HafeezOJ committed Sep 9, 2024
    Configuration menu
    Copy the full SHA
    8c12a19 View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2024

  1. Clean up and prettify log outputs

    @zoidy
    zoidy committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    71e9709 View commit details
    Browse the repository at this point in the history

  2. lint

    @zoidy
    zoidy committed Sep 11, 2024
    Configuration menu
    Copy the full SHA
    f2637b0 View commit details
    Browse the repository at this point in the history

Commits on Sep 16, 2024

  1. Skip checking required space when all articles are skipped

    @HafeezOJ
    HafeezOJ committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    95eae49 View commit details
    Browse the repository at this point in the history

  2. Exclude size of curation folder for skipped articles during space check

    @HafeezOJ
    HafeezOJ committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    4062de9 View commit details
    Browse the repository at this point in the history

  3. Lint

    @HafeezOJ
    HafeezOJ committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    94080a4 View commit details
    Browse the repository at this point in the history

  4. Lint

    @HafeezOJ
    HafeezOJ committed Sep 16, 2024
    Configuration menu
    Copy the full SHA
    937fd45 View commit details
    Browse the repository at this point in the history