Seasons Of Code

The Unreasonable Effectiveness of Recurrent Neural Networks    • Mohd. Safwan    • Arpit Aggarwal    • Satvik Mashkaria   

The Unreasonable Effectiveness of Recurrent Neural Networks

The Unreasonable Effectiveness of Recurrent Neural Networks


This project will involve learning many machine learning algorithms leading to RNNs. Mentees will implement a Neural Network and a Recurrent Neural Network framework from scratch

“Almost 4 years ago, Karpathy published a blog post(http://karpathy.github.io/2015/05/21/rnn-effectiveness/) that has since become quite well known in the community. Karpathy discusses some awesome results he achieved by training character level RNN on various text corpus. Anybody interested in this project is expected to go through the post thoroughly, even if you can’t understand most of it. We aim to follow Karpathy’s approach to understand and gain a deeper appreciation of RNNs while also exploring their versatility.

This project will involve learning many machine learning algorithms leading to RNNs. Mentees will implement a Neural Network and a Recurrent Neural Network framework from scratch. We will attempt to reproduce Karpathy’s results and go beyond to training on more data like Obama’s speeches, Trump’s tweets, the Bible, turtlesim code, cooking recipes, MIDI sequences, etc.

Prerequisites:
  • Enthusiam

For students who have participated in Summer of Science(Machine Learning track) before, this would be a great hands-on project!

Proposal expectations:

Write about your prior experience with things mentioned in the prerequisites and a list any prior machine learning projects completed. Do send across links to your project repos and demos, if any, along with the proposal. Although this is not mandatory but try to include a rough expected timeline for yourself.

The following points must be included in the proposal for the project:

  • Your motivation and understanding of the project
  • Background in ML/DL (include your previous projects)
  • How do you want to approach the problem, you thoughts/remarks.
  • Experience with Python and scientific libraries
Timeline for each week :
  • Finish Linear Algebra, Vector Calculus, and Statistics refresher
  • Install Ubuntu, set up a development environment
  • Learn/Brush-up Python, Torch, Jupyter, Numpy, Unix commands
  • Learn Linear Regression, Logistic Regression, Neural Networks
  • Read up on the use cases and building blocks of Deep Learning
  • Implement a recurrent neural network from scratch and train it on toy dataset.
  • Learn PyTorch, implement an RNN/LSTM network using PyTorch.
  • Start collecting data and training
  • Document all interesting observations