Seasons Of Code

Visual Perception for Self Driving Cars    • Aaron Jerry Ninan    • Thomas Jacob   

WnCC - Seasons of Code

Seasons of Code is a programme launched by WnCC along the lines of the Google Summer of Code. It provides one with an opprtunity to learn and participate in a variety of interesting projects under the mentorship of the very best in our institute.


List of Running Projects

Visual Perception for Self Driving Cars

Visual Perception for Self Driving Cars


Computer Vision. You might’ve heard of this extremely versatile domain and some of you might even be familiar with it. With applications ranging from self-driving cars to say, automated translation, this is a very handy and not to mention, cool thing to know and in this project, we aim to do just that.

No. of mentees: 7

Pre-requisites: Python programming, and some basic ML experience might be helpful

We would implement various vision-based algorithms using machine learning to drive a car autonomously using a single front camera.

We would explore both end to end models and Pipeline based models. The Pipeline could contain various smaller models like depth estimation, 3d object detection, lane detection, semantic segmentation, visual odometry etc. Many papers are available online. We can train and test our models on any freely available simulators. We can even self-drive in games like GTA V ;)

If time permits, we could also explore Deep Reinforcement Learning algorithms, where the agent (AI) learns to drive through mistakes and reward.

Include the following in your proposal:-

  • Your understanding of the project
  • Motivation
  • Background and experience
  • Weekly timeline based on your efficiency and background

Some References:-

  • End to End model by NVIDIA:- https://arxiv.org/pdf/2010.08776.pdf
  • Similar Project on GTA V:- https://github.com/Will-J-Gale/GTA5-Self-Driving-Car
  • Deep Learning Course:- https://www.coursera.org/specializations/deep-learning
  • Pytorch:- https://pytorch.org/tutorials/

Tentative Project Timeline

Week Number Tasks to be Completed
Week 1-2 Get Exposed to Deep Learning along with coding frameworks like Tensorflow or Pytorch
Week 3 Learn about some advanced CNN architectures and setup of the simulation environment
Week 4 Implement the end to end model and train it on your collected Dataset
Week 5 Explore models for pipeline approach(Object detection , Depth estimation ,Lane detection)
Week 6 Conclude by comparing your results and document your learnings.