Standard release

Added

• MLDevEnv: README.md now contains a few examples of using mlenvtool directly

Changed

• The framework is made compatible with PyTorch 1.8 and PyTorch-Lightning 1.2.4.
• All the regression tests and examples are made sure to pass on version 0.2.1 of PyTorch MLDevEnv

Removed

• MLDevEnv is moved to PyTorch MLDevEnv to make it more re-usable.

Fixed

• face_image_classification_gender_hybrid_vision_transformer.ipynb Jupyter notebook example is correct but the output saved inside the notebook contains a stacktrace from out of memory error.
• validation metrics are now reported separately from test metrics in progress bar callback

Standard release

Added

• A new Jupyter notebook example for BERT finetuning on a SST2 subset
• Reusable AAutoClsHeadClassificationTaskWDAdamWStepLR and AAutoClsHeadClassificationTaskWDAdamWWarmup classes for shaping classification heads for a desired number of classes and using AdamW optimizer in a combination with StepLR scheduler from PyTorch LinearWarmup scheduler from HuggingFrace transformers library.
• A new overridable method _forward_batch in AClassificationTask class for extracting a model inputs from a batch and running a forward pass for the module (by default it just uses batch[0] as a target)
• A new overridable method _get_target_from_batch responsible for extracting the target from the batch (by default it just uses batch[1] as a target)
• More explanatory comments for Hybrid ViT jupyter notebook examples, the notebook is re-run with higher number of epochs
• init_dl.sh is renamed to run_all.sh and all child scripts for running Jupyter notebook, MLFlow and TensorFlow now accept one more command line paramter defining a working directory

Changed

• All vision transformer-based Jupyter notebook examples are switched to AAutoClsHeadClassificationTaskWDAdamWStepLR class
• Changes to MLDevEnv packages:
  • The following conda packages were replaced with pip versions for MLDevEnv
    • jupytext
    • jupyterlab-git
  • The following packages were updated:
    • numpy==1.20.1
    • papermill==2.3.2
    • joblib==1.0.1
    • pillow==7.2.0
    • spacy==2.3.5
    • transformers==4.3.2
    • tokenizers==0.10.1
    • ray[tune]==1.2.0
    • ray[serve]==1.2.0
    • wandb==0.10.19

Fixed

• test_dataset is created twice in pytorch_hyperlight.datasets.cifar.py

Bug fixing release

Added

• More explanatory comments for Hybrid ViT jupyter notebook examples

Fixed

• Hybrid ViT jupyter notebook examples print the absolute paths of best model checkpoints
• pytorch_hyperlight.datasets module's folder and its subfolders do not contain __init__.py file

Standard release

Added

• CIFARDataLoadersBuilder for cifar100 and cifar10 datasets. The builder shares a lot of source code with UTKFacesDataLoadersBuilder
• random_index_split function in utils.random_utils module. The function helps with splitting the monolithic datasets into train, val and test parts
• Two new jupyter notebook examples of comparing Hybrid ViT (R50 + ViT) and classical ViT on both CIFAR100 and UTKFaces datasets

Fixed

• Added missing dependencies for PyTorch-Hyperlight
  • albumentations # ==0.5.2
  • papermill # ==2.3.1
  • tqdm # ==4.55.1
  • numpy # ==1.20.0
  • pillow # ==7.2.0
• "visualize_attention_map.ipynb" Jupyter notebook from external ViT-pytorch github repo (used in some of the Jupyter notebook examples) is collected and run along with other PyTorch-Hyperlight examples when "pytest" command is issued

Standard release

Added

• ViT attention analysis, ViT positional embedding analysis and references are added to "face_image_classification_gender_vision_transformer.ipynb" Jupyter notebook
• The new PTH dependencies:
  • kaggle==1.5.10
• create_subplots function in pytorch_hyperlight.utils.plot_utils for easy creation of subplots
• pytorch_hyperlight.datasets.kaggle.utkfaces module containing the functions for creating dataloaderes for UTKFaces dataset from Kaggle
• pytorch_hyperlight.tasks.classification.AAutoClsHeadClassificationTaskAdamStepLR class for automating the creation of classification heads for different classification models including CNNs and Transformers
• happy-path usage Jupyter notebook example for TrialMetrics class
• pytorch_hyperlight.utils.jupyter_nb_utils module containing the hepler functions for running jupyter notebooks (especially notebook examples)

Changed

• "face_image_classification_gender_vit_vs_efficientnet.ipynb" Jupyter notebook is renamed to "face_image_classification_gender_vision_transformer.ipynb"
• "face_image_classification_gender_vision_transformer.ipynb" now imports create_datasets and create_dataloaders from pytorch_hyperlight.datasets.kaggle.utkfaces
• MLDev python packages version changes:
  • jupytext==1.10.1
• image_utils.show_image_tensors now uses create_subplots from utils_subplot
• pytorch_hyperlight.metrics.TrialMetrics.plot method (and show_report method) now uses the newly added create_subplots function from pytorch_hyperlight.utils.plot_utils
• examples/_run_all_examples.ipynb notebook and test_notebook_examples testcase a) are refactored to use the functions from jupyter_nb_utils module b) use "_notebook_workspace" as a working folder for jupyter notebooks
• all examples are placed into separate folders by categories

Bug fixing release

Added

• unit tests for load_image_as_resized_tensor that make sure the function works with images with different number of color channels

Fixed

• nst pipeline fails when content and style images contain different number of color channels

Standard release

Added

• new MLDev Python packages as dependencies
  • gitpython==3.1.12
  • wget==3.2
• unit tests for load_image_as_resized_tensor

Fixed

• ./check_if_rogue_jupyterlab_is_installed.sh doesn't fail if rogue version is found
• copy_urls_to_files creates Boto3 session even for https urls
• utils.image_utils.load_image_as_resized_tensor function fails to produce tensors of equal size in certain scenarious. The may lead to nst pipeline failure for certain image sizes (as NSTImageUrlOrPathDataLoader uses load_image_as_resized_tensor)

Bug fixing release

Added

• MLDev dependencies
  • visdom==0.1.8.9
  • dominate==2.6.0
  • ml_collections==0.1.0
  • kornia==0.4.1
  • opencv-python==4.5.1.48
  • visdom==0.1.8.9
  • dominate==2.6.0
• NST model builder in pytorch_hyperlight.tasks.nst module now checks that style_weight_list has the same length as style_image_list

Fixed

• plain_simple_nst.ipynb ignores all stypes except for the first one

Standard release

Added

• The following packaes were added to MLDev:

  • gym==0.18.0
  • pytorch-lightning-bolts==0.3.0
  • timm==0.3.4
  • ray[serve]==1.1.0
  • pipdeptree==2.0.0

• New copy_file_to_s3 and copy_fileobj_to_s3 functions in pytorch_hyperlight.utils.request_utils package

Changed

• Improved face_image_classification_gender_vit_vs_efficientnet.ipynb by using data augmentation based Albumentations library,
  added class imbalance plotting and displaying sample images, number of trials for each model is reduced down to 1
• Default number of checkpoints kept by PyTorch-Hyperlight in a single-trial model is reduced down to 2
• Many packages in MLDev has been updated:
  • scikit-learn==0.24.1
  • pandas==1.2.1
  • numpy==1.20.0
  • pip==21.0.1
  • ipython==7.20.0
  • matplotlib==3.3.4
  • tqdm==4.55.1
  • captum==0.3.1
  • xeus-python==0.10.2
  • tensorboard==2.4.1
  • papermill==2.3.1
  • scikit-image==0.18.1
  • notebook==6.2.0
  • jupyterlab==3.0.6
  • jupyterlab-lsp==3.3.0
  • jupyterlab_code_formatter==1.4.3
  • ipympl==0.6.3
  • pytorch-lightning==1.1.6
  • boto3==1.16.63
  • psutil==5.8.0
  • transformers==4.2.2
  • wandb==0.10.17
  • pytest==6.2.2
  • coverage==5.4
• The following packages were downgraded
  • pillow==7.2.0 # because gym package requires the older version

Deprecated

Removed

• The following packages were temporarily remoted from MLDev
  • datasets (as it requires tqdm<4.50)

Fixed

• Classifier head was not trained in face_image_classification_gender_vit_vs_efficientnet.ipynb example due to the incorrect classifier
  head paramter names
• Checkpoint file names in 'singe-trial' mode contain confusing prefix and a value of val_loss of the last epoch step.
  This is fixed by reverting back to the checkpoint naming scheme used by PyTorch Lightning by default.
• check_if_rogue_jupyterlab_is_installed.sh script in MLDev silently crashes when JupyterLab version doesn't match the expected version

Bug fixing release

Fixed

• "torch" PyPI package was referred to as pytorch in requirements.txt which results in failure to install pytorch_hyperlight via pip