PROJECT 2: Dhruv Karthik

[botforge]

Github link
Due to critical paper submissions, started this assignment on Thursday & took 3 late days :( I was really looking forward to doing a ton of extra credit on this assignment, it is unfortunate that it came during a very busy week of the semester. I hope on exploring implementations of NN's through the rest of the course.

Character Recognition: XOR converges if you initialize nicely. Interestingly, it does not converge if you feed it as an entire batch (all 4 samples), as your loss doesn't change. Implemented batched inputs and an MLP Builder that allows you to arbitrarily specify input dimensions and 'number of samples' per batch for batch gradient descent. Did not get the time to try the full character recognition portion of it. Would love to keep working on it.

Feedback: Not that far from a CIS 680 assignment or a CIS 520 assignment (which each required some linear algebra prereq and took one lecture). Good practice to implement NN's from scratch, you learn something new every time. This time, I learnt about the importance of initialization and 'batch construction'. With very large datasets, you can randomly sample inputs to form a batch, but with the XOR problem, you should be more careful.

Stream Compaction: Did the GPU vs CPU extra credit (ended non-working threads early). Surprisingly, my WEF scan did better than thrust on some array sizes.