Projects

My project experience is across the fields of computer vision, designing aerial and indoor robotics software systems, planning for robotics, embedded systems, electronic design automation (EDA), mobile app development and web development. It has been acquired through working in industry, academic research, startups and through participating in numerous competitions.

Feb 2019 - June 2020

2017_brl

A Discriminative-Deliberative Framework for 6-Dof Pose Estimation

Location: Search Based Planning Lab, RI, CMU

Learning based discriminative 6-Dof pose estimation methods require large datasets with annotated ground truth poses. Deliberative search based perception methods like PERCH 2.0 make assumptions that limit their scalibility in 6-Dof pose estimation scenarios. In this work, we combine the two together, building on their strengths and addressing the shortcomings of each. Our MaskRCNN and PERCH 2.0 based framework achieves better accuracy than state-of-the-art purely discriminative methods and requires training only for instance segmentation.

Paper : Master’s Thesis, IROS 2020 / Videos : Clip 1 / Code : GitHub

Skills : C++, CUDA, Python, ROS, PCL, OpenCV, OpenGL, MPI, PyTorch, Unity

Sep 2019 - June 2020

2017_brl

PERCH 2.0 : Large Scale Deliberative Perception on GPU

Location: Search Based Planning Lab, RI, CMU

Perception Via Search (PERCH) is a class of algorithms that search for the best explanation of the observed scene in a space of possible rendered scenes, thus accounting for occlusion. However these algorithms are slow, owing to their deliberative nature. In this work we redesign these algorithms in order to exploit their inherent parallel nature better on a GPU. The redesigned algorithm, PERCH 2.0, is upto a ~100X faster that its predecessor PERCH.

Paper : Master’s Thesis, IROS 2020 / Videos : Clip 1 / Code : GitHub / Skills : CUDA, OpenMP, C++, Python, MPI

March 2019 - May 2019

2017_brl

Visual Learning for Jenga Tower Stability Prediction

Location: RI, CMU

Deep neural networks are commonly used for object detection and classification. In this work, we explore if we can enable networks to learn physical intuition for predicting stability of a Jenga tower on removal of individual tower blocks. A Mask-RCNN + Inception-V4 based pipeline was created for this purpose.

Links : Presentation / Code : GitHub / Skills : Python, PyTorch, TensorFlow, MuJoCo

April 2014 - May 2015

2017_brl

Perception for Estimating Weed Density and Controlling Herbicide Use

Location: IIT Kharagpur, Kharagpur, India

Optimal application of herbicide for weed eradication is both cost effective and good for the environment. In this project, I designed a 6 camera and embedded controller based system that varies herbicide output based on weed density estimation from color images, cutting chemical usage by 79.5% & achieving weeding efficiency of 90.26%

Paper : Current Science 2018 / Media :

Skills : C++, OpenCV, Arduino

Feb 2017 - Jun 2017

2017_brl

Computer Vision Based Retail Checkout System

Location: Bangalore, India

SaturnCart is AI that accompanies the user as he shops, verifying his cart-items in real-time, surfacing offers based on cart items, and providing valuable recommendations. In this project, we designed a prototype consisting of a mobile app (which is placed on the shopping cart), linked to a computer vision system on the cloud and a weight based embedded system (under the shopping cart).

Videos : Clip 1

Skills : OpenCV, Arduino, TensorFlow, React Native

May 2014 - July 2014

2017_brl

Visual Odometry in Smoke Occluded Environments

Location: AIR Lab, RI, CMU

Presence of smoke in the environment leads to loss of color contrast, precision and saturation and hence visual odometry algorithms are unable to find distinguishing features in the image. In this project, I developed an integrated real-time image dehazing and visual odometry approach for autonomous aerial navigation on a smoke degraded shipboard which was tested on a real smoke filled shipboard dataset.

Links : Report / Videos : Clip 1

Skills : C++, OpenCV, ROS