Gå til hovedinnhold
Hopp til hovedinnhold

Hao Nhien Vu

Hao Nhien Vu
Master ICT
Fakultet for teknologi og realfag
15. desember 2020

My experience as a student research assistant on estimating human poses

This autumn, I have been working with Associate Professor Lei Jiao on a project concerning anonymization in videos for the local law enforcement. However, Lei tasked me with a subtask; I was to estimate human poses using deep learning. A project like this has been enriching as it provides excellent experience in how machine learning is applied in the real world.

A human pose estimation system is defined as a system that can localize human joints based on images or videos. These joints are also known as keypoints and enclose body parts such as hips, knees, elbows, etc. It is essentially a set of coordinates to these various keypoints that can be connected to describe the person's pose. 


My responsibility in this project is to create a working human pose estimation model and apply it to videos to estimate human poses. Microsoft released a large-scale object detection, segmentation, and captioning dataset in 2014, known as COCO. The COCO dataset is one of the most popular image datasets out there, and it provides keypoint detection, which makes it ideal for a human pose estimation problem.


For this project, I decided to use a Simple Pose model built on top of a residual neural network, or ResNet for short. Simple Pose's final layer is a convolution layer predicting one heatmap for each keypoint. Once the model is created, we can set it up for training. When the training is complete, the only thing left is to iterate through a video, frame by frame, and feed each frame to the network. The network will load each frame, preprocess the image, detect people in the image, postprocess the detected people, estimate the pose for each person, and finally, plot the result.


I am very grateful for the opportunity to work on such an interesting and exciting project as this one. It has given me an overall better understanding of machine learning and the various neural networks in play. As a student, it is highly educational to be a part of a research project as it challenges oneself in ways different from projects given as part of a course. Thanks to Lei Jiao for having faith in me and my abilities throughout this project. Finally, I want to recommend other students to consider applying for a student research assistant position. It is a great learning experience and a great introduction to real-world problems.