Seasons Of Code

Generating a human pose dataset using PC games    • Rohit Jena   

Generating a human pose dataset using PC games

Generating a human pose dataset using PC games


This project will focus on getting human pose estimates in games to generate a dataset using no manual annotations or labelling.

Deep networks are very data hungry in this age. Annotating lots of data is very tedious, expensive, and inefficient.

However, a lot of ground truth data can be easily generated by using the rendering of video games to extract specific information like semantic segmentation, depth maps, etc. The project will focus on getting human pose estimates in games to generate a “in-the-wild” dataset using no manual annotations or labelling.

This will be done by injecting specialized code into the DirectX rendering API. We’ll further test the effectiveness of the dataset on real images to see if such a dataset can provide benefits in training.

Timeline

Week Task
Week1 Understand the main paper, and what pose estimation is
Week2 Download a free game and start exploring the DirectX API
Week 3, 4 Extract the pose information from pre-renders
Week 5, 6 Cleaning up the dataset, and testing a small DNN to predict pose
Week 7, 8 Try on one more game and start testing on real datasets