A little bit of background: I have been involved in Machine Learning since the past year or so. I've taken university courses and online courses(Andrew NG's, Udacity's Intro to Machine Learning and currently taking up Andrew NG's course on CNN) in this brief period of time. However, I've been into ML on and off(well, mostly off), and just have basic knowledge of scikit-learn and tensorflow. With this 100 Days of ML Code challenge I would like to strengthen my code writing skills that is get an insight into sci-kit learn library, tensorflow and also keras.
Today's Progress : Initialised the repository to keep track of my 100 Days of ML journey
Thoughts : Hope I can continue through the 100 days and do not stop in between :)
Today's Progress : Created a simple Linear Regression model using scikit-learn library, baby-steps! Also, scraped Kaggle for ML datasets.
Thoughts : Maybe implement my own version of Linear Regression model in the future
Link to work : Linear Regression Model
Today's Progress : Created a single layered Neural Network MNIST classifier using tensorflow. That's a combined dive into the world of tensorflow and neural networks!
Thoughts : The official tensorflow tutorial, MNIST for ML Beginners is really good as everything is explained in deep. I'm thinking of implementing a Deep Neural Network MNIST classifier and a CNN MNIST classifier and compare their accuracy.
Link to work : MNIST Classifier
Today's Progress : Created a Deep Neural Network MNIST Classifier. Learned a few things about tensorflow and Neural Networks in general while constructing this model, details inside the work folder.
Thoughts : Increasing the depth of a Neural Network and increasing the number of epochs increases the accuracy of the model but this tends to saturate at a particular limit. Further increase in both doesn't result in greater accuracies.
Link to work : MNIST Classifier Deep
Today's Progress : Studied about various optimization techniques, including Gradient Descent and Adam Optimizer.
Thoughts : Implement my own various of these optimizing techniques maybe.
Today's Progress : Attempted first assignment of Week 1 of Coursera's Convolutional Neural Networks course where I implemented forward propagation from scratch using numpy.
Thoughts : CNNs are fun! numpy is a powerful library!
Course Link : Convolutional Neural Networks
Today's Progress : Fixed my Deep MNIST Classifier and implemented SGD correctly. Getting higher accuracy with the new model. Also watched an introduction video for tensorflow js.
Thoughts : I think I've hit the accuracy ceiling with Deep Neaural Networks, time for CNN model.
Link to work : MNIST Classifier Deep
Today's Progress : Created Convolution Neural Network model for MNIST classifier. Learned a lot about CNNs in this process.
Thoughts : I will retrain the model with more epochs and hope for higher accuracy. Dropouts really help a lot.
Update : Reran the model with updated dropout probabilities and got an increase in accuracy.
Link to work : MNIST CNN Classifier
Today's Progress : Thought of creating a CNN classifier for classifying cats and dogs. PC ran out of memory loading the images.
Thoughts : Find an optimized way of loading images or get more RAM.
Today's Progress : Watched videos on how to load images into python and convert them into arrays. Also dived into tf.keras
Thoughts : Use tf.keras for quicker model initialization.
Today's Progress : Going a bit theoretical for a few days.
Today's Progress : Read few articles and watched videos on RNN. Grasped the basic idea behind RNNs but still need to figure out its inner workings.
Thoughts : Time to read papers/more articles related to RNN and its implementation.
Today's Progress : Reading MSCOCO Dataset Paper. Read few more articles on RNN and NLP. Preping for a huge project coming up ahead.
Today's Progress : Read Show and Tell: A Neural Image Caption Generator paper. Finally cleaned up the code and uploaded my Dogs vs Cats CNN Classifier on GitHub, which I implemented during my off days.
Thoughts : I will implement a Neural Image Caption Generator next. Before that I'll get my hands dirty on nltk and basic LSTM cell.
Link to work : Cats Dogs CNN Classifier
Today's Progress : Getting my hands dirty on nltk. Started work on a simple positive and negative sentence classifier.
Thoughts : To complete an advanced project you first gotta build your basics.
Today's Progress : Continuing building the pos neg classifier.
Thoughts : Pre-processing data is challenging!
Today's Progress : Finally completed my pos neg classifier. Also this is my first keras implementation.
Thoughts : Get more data when working on a Deep Neural Network!
Today's Progress : Watching Andrej Karpathy's lecture on Recurrent Neural Networks, Image Captioning and LSTM(Winter 2016). Learnt about vanishing and exploding gradient problem.
Link to work : CS231n Winter 2016: Lecture 10: Recurrent Neural Networks, Image Captioning, LSTM - Andrej Karpathy
Today's Progress : Initialised Neural Image Caption repository on Github.
Link to work : Neural Image Caption
Today's Progress : Pre processing Image and Caption data for Neural Image Caption.
Link to work : Neural Image Caption
Today's Progress : Finished implementing Neural Image Caption using keras. Basic training done on Flickr8k dataset and decent results were obtained.
Thoughts : Read more about LSTMs and RNNs. Find the best model which gives optimal results. Train on Flickr30k and if possible on MSCOCO dataset as well. Add comments and improve readability of the code.
Link to work : Neural Image Caption
Today's Progress : Experimenting with different encoder decoder models(CNN RNN) that would give optimal results. Trained basic model on Flickr30k for 10 hours, results unsatisfactory.
Today's Progress : Training-tweaking models, hoping for better results, result negative.
Today's Progress : Re-trained my NIC model on Flickr30k dataset for 2 epochs(2 hours/epoch) and the model learnt to spell out a bunch of 'a's. Disaster!
Thoughts : Train for more epochs.
Today's Progress : Trained my NIC model on Flickr8k for 25 epochs. Skimmed through 'Where to put the Image in Image Caption Generator' paper to find which model would be the best(paper was too long). Watched a video on autoencoders.
Today's Progress : Cleaned up Neural Image Caption's code and added comments for easier understanding. Read about sub sampling and negative sampling somewhere(it was too good but I lost the link!)
Update: Found the link. I was reading an article on Medium hence I was confused as to why there was nothing related to it in my browser's history.
Thoughts : Gonna take a break from this project and focus on something else.
Link to work : Neural Image Caption
Today's Progress : Break.
Today's Progress : Watched a couple of Siraj's video on Machine Learning and refreshed my memory.
Today's Progress : Started with auto-encoders. Attended Big Data seminar held at my college. Got an insight into the world of AI and Big Data in Healthcare.
Thoughts : Auto-encoders look cool!
Today's Progress : Created an MNIST denoiser in keras using auto encoders.
Thoughts : Read articles on auto encoders.
Link to work : MNIST autoencoder
Today's Progress : Watched a few videos on autoencoders. They aren't lossless hence do not have much of practical utility.
Thoughts : Implement a small autoencoders project in the future. But for that gotta find a practical use of them too!
Today's Progress : Setting up pytorch on my development environment. Most of the research models(by Google, NVIDIA, Microsoft, Facebook) are built using pytorch hence a bit of knowledge on pytorch is a must.
Today's Progress : Started learning about seq2seq models.
Thoughts : Next project is gonna be a translator.
Today's Progress : Read 'Sequence to Sequence Learning with Neural Networks' paper and jotted down the important points. Initialised Neural Machine Translation repository on Github.
Thoughts : This project seems a bit challenging.
Link to paper : Sequence to Sequence Learning with Neural Networks
Link to work : Neural Machine Translation
I'm one-third way through the challenge, yay!
Today's Progress : Reading texts related to seq2seq learning.
Today's Progress : College takes a toll on me. Read an article on Machine Translation and how to go about it.
Today's Progress : Import and playing with training data for Neural Machine Translation project.
Thoughts : I've no idea how I will implement this project but I love challenges :)
Today's Progress : Updating Neural Machine Learning jupyter notebook with proper documentation so that it will be easier for anyone to understand the model as well as the paper. Re-reading and summarising the paper.
Thoughts : Teaching is the best way to learn.
Today's Progress : Watched OpenAI-5 video by Siraj. Watched video on Quadruped Locomotion patterns by Two Minute Papers.
Thoughts : Quadruped Locomotion Videos have my mind blown. This is the first time I saw something like this.
Today's Progress : Preprocessing the training data for NMT project. Hit some roadblocks.
Today's Progress : Changed the NMT dataset I was using since the original dataset site was down. Preprocessing input.
Today's Progress : Continuing working on NMT.
Thoughts : Long weekend = low productivity
Today's Progress : Continuing working on NMT.
Today's Progress : Finding the perfect course structure to teach ML from the very basic.
Today's Progress : Continued building my Neural Machine Translation project. Fixing previous errors.
Today's Progress : Finished creating Dictionary and Encoding class for Neural Machine Translation.
Thoughts : Gonna continue with NMT after a few days.
Today's Progress : Created a basic tensorflow model and a bit Deep CNN model for CIFAR-10 dataset. Learnt about kNN and implemented a kNN model, using sklearn library as well as from scratch, on MNIST dataset.
Thoughts : I always thought kNN was a tough algorithm(FOTU), turns out its the easiest ML algorithm I've ever seen. Implement transfer learning on COFAR-10.
Today's Progress : Getting started with Reinforcement Learning. Watched Introduction to Move 37 video.
Today's Progress : Debugged my friends tensorflow CNN model.
Thoughts : Seems simple but it took hours.
Today's Progress : Started with Denoising Dirty Documents problem on Kaggle.
Thoughts : Hopefully I can complete this in 1-2 days.