Projects
Ph.D. THESIS:
Topic: Deep Facial Expression Modeling and Motion Retargeting from 2D Images[thesis]
Advisors: Prof. Linda Shapiro, Graphics and Imaging Laboratory, UW CSE and Dr. Alex Colburn, Apple Inc.
Abstract: Facial expression modeling and motion retargeting is an important problem in both computer graphics and computer vision, which involves estimating the 3D motion of a human face from a 2D image and transferring it to a 3D character. Traditional methods fit a 3D morphable model (3DMM) to the face, which requires an additional face detection step, does not ensure perceptual validity of the retargeted expression, and has limited modeling capacity (hence fails to generalize well to in-the-wild data). In this thesis, I present five approaches to overcome these limitations: (1) a supervised end-to-end network to jointly predict the bounding box locations and 3DMM parameters for multiple faces in a 2D image, (2) a self-supervised end-to-end framework to jointly learn a personalized face model per user and per-frame facial motion parameters from a large corpus of in-the-wild videos of user expressions, (3) joint audio and video features for improved facial motion modeling, (4) a semi-supervised multi-stage deep learning system that leverages a database of hand-animated character expressions to predict a character's rig parameters from a user's facial expressions, and (5) an unsupervised cycle-consistent generative adversarial network that directly predicts the character's 3D geometry with retargeted expression from the input image. Experimental results have shown that these approaches outperform state-of-the-art methods in terms of retargeting accuracy. Applications of my approaches include avatar animation for visual storytelling or virtual conversation, motion capture films, social AR/VR experience and so on.
M.TECH. THESIS:
Topic: Region-Based Retrieval of Remote Sensing Images using Graph-Theoretic Approaches [thesis]
Advisor: Prof. Subhasis Chaudhuri, Vision and Image Processing Laboratory, IITB EE
Abstract: This thesis presents an unsupervised and a semi-supervised graph-theoretic approaches. The proposed approaches are characterized by modeling each image by a graph and retrieving the images most similar to the query image by evaluating graph-based similarities. The two approaches involve a trade-off between the amount of user intervention and the accuracy of retrieval. Experiments carried out on an archive of aerial images show that these approaches significantly improve the retrieval performance compared to the state-of-the-art RS image retrieval methods.
B.E. THESIS:
Topic: Low Cost Low Bandwidth Virtual Education Platform Design for Underserved People with special emphasis to students having vision and hearing impairments [pdf]
Advisor: Prof. Iti Saha Misra, OPNET Laboratory, JU ETCE
An initiative of SIGHT (Special Interest Group on Humanitarian Technology), IEEE
Abstract: Developed a virtual classroom application using Visual C# (in Microsoft. Net environment) that used TCP/IP and VoIP for effective data transmission. Included the provision of live teacher-student interaction using whiteboard and different kinds of files (Powerpoint, Word, Excel files etc.)
Ph.D. COURSE PROJECTS:
Comparitive study of model-free reinforcement learning methods for continuous control [pdf] (Course: Machine Learning)
Advisor: Prof. Sham Kakade, UW CSE
Compared Trust Region Policy Optimization and Deep Deterministic Policy Gradient on simulated MuJoCo tasks on a common basis
Local Collision Avoidance using Laser Sensors for a Nano-Drone (Course: Robotics)
Advisor: Tapomayukh Bhattacharjee, UW CSE
M.TECH. COURSE PROJECTS:
Seminar: Content-Based Image Retrieval using Machine Learning Techniques [pdf]
Abstract: Described how supervised and unsupervised machine learning techniques are used in content-based image retrieval
Real-time tracking of non-rigid objects using Mean Shift Algorithm [pdf] (Course: Computer Vision)
Advisor: Prof. Ajit V. Rajwade, IITB CSE
Implemented a system in MATLAB to track non-rigid objects with high accuracy in a video.
Depth map and 3-D shape estimation from stereo and motion estimation from optical flow [pdf] (Course: Computer Vision)
Advisor: Prof. Subhasis Chaudhuri, Vision and Image Processing Laboratory, IITB EE
Image reconstruction from compressive measurements using OMP algorithm [pdf] (Course: Computer Vision)
Advisor: Prof. Ajit V. Rajwade, IITB CSE
Video stabilization by rectifying 2-D affine motion [pdf] (Course: Computer Vision)
Advisor: Prof. Ajit V. Rajwade, IITB CSE
Use of Gabor Filters for Image Segmentation [pdf] (Course: Image Processing)
Advisor: Prof. S. N. Merchant, SPANN Laboratory, IITB EE
Applied Gabor filters for edge detection and texture-based segmentation of images.
Face recognition using discrete wavelet transform and machine learning techniques [pdf] (Course: Wavelets)
Advisor: Prof. V. M. Gadre, IITB EE
Designed a user interface to recognize face images with varying poses, expressions and lighting conditions.
B.E. COURSE PROJECTS:
Architecture Design for scan-testability and security of crypto chips [pdf]
Advisor: Prof. M. K. Naskar, Advanced Digital and Embedded Systems Laboratory, Jadavpur University
Abstract: Proposed and implemented a new cost-effective design on Spartan-3E FPGA using SREEP and data encryption algorithms like DES, AES etc. to improve the security of chip data from side channel attacks and hacking without hampering scan-observability and scan-controllability.
Survey on effects of electromagnetic radiation from mobile towers, Wi-fi hotspots etc. at various places in Kolkata
Advisor: Prof. Iti Saha Misra, QUALNET Laboratory, RCC Building, Jadavpur University
Abstract: Published a report headlined "Caught in radiation trap" on mobile tower radiation in The Times of India on 7th July, 2013. Got practical experience in this field of wireless communication and microwave engineering.
Code optimization in compiler design [pdf] (System Software and Computer Architecture)
Advisor: Prof. Amit Konar, Artificial Intelligence Laboratory, Jadavpur University
Abstract: Coding and optimization of an English grammar parser.