My solution for Digit Recognizer Kaggle competition based on @cdeotte experiments.
Input images were augmented with random rotation up to 20 degrees and shift up to 3 pixels. Here I didn't use random horizontal flip because digit '2' could transform into '5', and learning could be messed up. Data augmentation usually improves DL model performance so I didn't compare current model with model on non-augmented data.
I took original LeNet-5 and compared adjustments in above-mentioned experiments. All comparisons were built up to 10 epochs. Then, final model was trained for 100 epochs (46 hours on Intel Core i5-4670@3400MHz CPU) and got accuracy score 99.6%. Solution was submitted to Kaggle and took 370th place with accuracy of 0.99557.
To skip implementation of visualizing, I used Weights & Biases tool.
Visualizations, network architechtures and execution logs available at Weights & Biases project.
Make sure to install dependencies from requirements.txt. After that run train.py from root directory which is equivalent to:
train.py --epochs=100 --lr=0.01 --momentum=0.5 --use-wandb=False --output-dir=. --predict_every=1