One example to learn all the core concepts of Pytorch. This repo will also work as a training template for any experiment.
In this repo, I tried to implement training routines and inference routines. Along with this, I tried to add various reference links for some of the concepts. Following concepts are implemented in the repo.
- Custom weight initialization
- SAME and VALID padding for Conv2D and Pooling layer in Pytorch just like Tensorflow
- Custom Learning Rate Schedules
- Cosine Anneling Learning Rate
- Learning rate plotting in Tensorboard before starting training.
- Custom implementation of regularization loss
- Model Summary just like Keras
- Reproducibility of each experiments with setting seeds for pytorch and other modules
- TensorBoard Summary Support
- Automatic Mixed Precision Training
- Inference scripts for Pytorch and ONNX
- Custom Dataloader mechanism for dataset handling
- Using pretrained models such as e.g. resnet18
- Restore weights to architecture just like Tensorflow 1.x for pretrained models for finetuning purpose
- Save Checkpoint during training and resume training from that checkpoint
- Remove old checkpoint just like Tensorflow 1.x checkpoint saver mechanism.
- Experiment management: Saving of experiment files in a separate folder during each run.