Projects

Ph.D. THESIS:

Topic: Deep Facial Expression Modeling and Motion Retargeting from 2D Images[thesis]

  • Advisors: Prof. Linda Shapiro, Graphics and Imaging Laboratory, UW CSE and Dr. Alex Colburn, Apple Inc.

  • Abstract: Facial expression modeling and motion retargeting is an important problem in both computer graphics and computer vision, which involves estimating the 3D motion of a human face from a 2D image and transferring it to a 3D character. Traditional methods fit a 3D morphable model (3DMM) to the face, which requires an additional face detection step, does not ensure perceptual validity of the retargeted expression, and has limited modeling capacity (hence fails to generalize well to in-the-wild data). In this thesis, I present five approaches to overcome these limitations: (1) a supervised end-to-end network to jointly predict the bounding box locations and 3DMM parameters for multiple faces in a 2D image, (2) a self-supervised end-to-end framework to jointly learn a personalized face model per user and per-frame facial motion parameters from a large corpus of in-the-wild videos of user expressions, (3) joint audio and video features for improved facial motion modeling, (4) a semi-supervised multi-stage deep learning system that leverages a database of hand-animated character expressions to predict a character's rig parameters from a user's facial expressions, and (5) an unsupervised cycle-consistent generative adversarial network that directly predicts the character's 3D geometry with retargeted expression from the input image. Experimental results have shown that these approaches outperform state-of-the-art methods in terms of retargeting accuracy. Applications of my approaches include avatar animation for visual storytelling or virtual conversation, motion capture films, social AR/VR experience and so on.


M.TECH. THESIS:

Topic: Region-Based Retrieval of Remote Sensing Images using Graph-Theoretic Approaches [thesis]

  • Advisor: Prof. Subhasis Chaudhuri, Vision and Image Processing Laboratory, IITB EE

  • Abstract: This thesis presents an unsupervised and a semi-supervised graph-theoretic approaches. The proposed approaches are characterized by modeling each image by a graph and retrieving the images most similar to the query image by evaluating graph-based similarities. The two approaches involve a trade-off between the amount of user intervention and the accuracy of retrieval. Experiments carried out on an archive of aerial images show that these approaches significantly improve the retrieval performance compared to the state-of-the-art RS image retrieval methods.


B.E. THESIS:

Topic: Low Cost Low Bandwidth Virtual Education Platform Design for Underserved People with special emphasis to students having vision and hearing impairments [pdf]

  • Advisor: Prof. Iti Saha Misra, OPNET Laboratory, JU ETCE

  • An initiative of SIGHT (Special Interest Group on Humanitarian Technology), IEEE

  • Abstract: Developed a virtual classroom application using Visual C# (in Microsoft. Net environment) that used TCP/IP and VoIP for effective data transmission. Included the provision of live teacher-student interaction using whiteboard and different kinds of files (Powerpoint, Word, Excel files etc.)


Ph.D. COURSE PROJECTS:

  1. Comparitive study of model-free reinforcement learning methods for continuous control [pdf] (Course: Machine Learning)

    • Advisor: Prof. Sham Kakade, UW CSE

    • Compared Trust Region Policy Optimization and Deep Deterministic Policy Gradient on simulated MuJoCo tasks on a common basis

  2. Local Collision Avoidance using Laser Sensors for a Nano-Drone (Course: Robotics)


M.TECH. COURSE PROJECTS:

  1. Seminar: Content-Based Image Retrieval using Machine Learning Techniques [pdf]

    • Abstract: Described how supervised and unsupervised machine learning techniques are used in content-based image retrieval

  2. Real-time tracking of non-rigid objects using Mean Shift Algorithm [pdf] (Course: Computer Vision)

    • Advisor: Prof. Ajit V. Rajwade, IITB CSE

    • Implemented a system in MATLAB to track non-rigid objects with high accuracy in a video.

  3. Depth map and 3-D shape estimation from stereo and motion estimation from optical flow [pdf] (Course: Computer Vision)

    • Advisor: Prof. Subhasis Chaudhuri, Vision and Image Processing Laboratory, IITB EE

  4. Image reconstruction from compressive measurements using OMP algorithm [pdf] (Course: Computer Vision)

    • Advisor: Prof. Ajit V. Rajwade, IITB CSE

  1. Video stabilization by rectifying 2-D affine motion [pdf] (Course: Computer Vision)

    • Advisor: Prof. Ajit V. Rajwade, IITB CSE

  1. Use of Gabor Filters for Image Segmentation [pdf] (Course: Image Processing)

    • Advisor: Prof. S. N. Merchant, SPANN Laboratory, IITB EE

    • Applied Gabor filters for edge detection and texture-based segmentation of images.

  2. Face recognition using discrete wavelet transform and machine learning techniques [pdf] (Course: Wavelets)

    • Advisor: Prof. V. M. Gadre, IITB EE

    • Designed a user interface to recognize face images with varying poses, expressions and lighting conditions.


B.E. COURSE PROJECTS:

  1. Architecture Design for scan-testability and security of crypto chips [pdf]

  • Advisor: Prof. M. K. Naskar, Advanced Digital and Embedded Systems Laboratory, Jadavpur University

  • Abstract: Proposed and implemented a new cost-effective design on Spartan-3E FPGA using SREEP and data encryption algorithms like DES, AES etc. to improve the security of chip data from side channel attacks and hacking without hampering scan-observability and scan-controllability.

  1. Survey on effects of electromagnetic radiation from mobile towers, Wi-fi hotspots etc. at various places in Kolkata

  1. Code optimization in compiler design [pdf] (System Software and Computer Architecture)

  • Advisor: Prof. Amit Konar, Artificial Intelligence Laboratory, Jadavpur University

  • Abstract: Coding and optimization of an English grammar parser.