Seasons Of Code

PAC MAN    • Utkarsh Agarwal, Gagan Jain    

WnCC - Seasons of Code

Seasons of Code is a programme launched by WnCC along the lines of the Google Summer of Code. It provides one with an opprtunity to learn and participate in a variety of interesting projects under the mentorship of the very best in our institute.


List of Running Projects

PAC MAN

PAC MAN


If you remember the game in the image above, your childhood was awesome :p

In this project, we aim to build an intelligent agent “Pac-Man” that uses Reinforcement Learning to ace the game I’m sure all of us have played.

No. of mentees: 4

Description:

Reinforcement Learning (RL) is a field of Artificial Intelligence where an agent learns by interacting with its environment and receiving a reward/penalty for its actions. RL has recently started receiving a lot more attention, owing to the famous victory by an RL agent over the world champion in the game of “Go”. This project aims to get people familiar with the field and get started with a hands-on project that will require them to understand and implement some of the popular RL algorithms. We will start with building a simple GUI, and will then try out algorithms like SARSA and Q-learning for training agents in basic environments. We will then move on to Deep RL techniques for handling Pac-Man which involve the use of neural networks for training the agents.

Reading:

Nothing as such right now. If you’re interested, you might want to read the first chapter of the book “Introduction to Reinforcement Learning” by Sutton and Barto to get a feel of RL. The book is available freely online and a simple google search should do :)

Watching the video lectures of the Reinforcement Learning Specialisation on Coursera by The University of Alberta can be a good starting point for getting the hang of RL.

Links:

Sutton and Barto (http://incompleteideas.net/book/RLbook2018trimmed.pdf) Coursera (https://www.coursera.org/specializations/reinforcement-learning?) Enjoy watching this beautiful documentary about AlphaGo by DeepMind - (https://www.youtube.com/watch?v=WXuK6gekU1Y)

Plan:

The mentees will work collaboratively towards the project, with individually defined goals for everyone at a regular frequency.

Proposal:

We will be happy to mentor 4 students for this project. Familiarity with Python is essential. We would also use an ML framework such as PyTorch or Keras, but don’t worry if do not have experience working with them, you can spend some extra time on it along with the reading material during the initial phase of the project. Familiarity with probability theory will help you grasp the material better. You should mention your motivation for taking up this project and your past experience with programming in the proposal. Also, mention a rough approach for the project based on your understanding of the problem.

Tentative Project Timeline

Week Number Tasks to be Completed
Week 1 Basics of RL and some basic implementations. Python beginners should also start learning about the ML libraries. The goal for this week from the project side will be to have a simple GUI from openai gym ready and running for visualization.
Week 2-3 Understanding RL algorithms and implementing them on basic examples such as grid-world and mountain car. Training the agent based on these algorithms and observing the results.
Week 4-5 Understanding the technique of Deep Reinforcement Learning, implementing them, and comparing the results with other available implementations.
Week 6 Wrapping things up, documenting the work done and exploring possible extensions to the work done.

Checkpoints:

Checkpoint Number Progress
1 Getting started with the theory of RL. Getting the GUI ready
2 Understanding simple RL concepts and algorithms like MDP planning, SARSA and Q Learning
3 Implementing the basic applications of RL algorithms to get a hang of them
4 Understanding and implementing Deep RL algorithm for learning for Pac Man
5 Finishing up the implementation by fine-tuning the hyperparameters and removing bugs
6 Documenting everything