Nitin Agarwal

I am a Research Scientist at Common Sense Machines (CSM), where I work on 3D computer vision, computer graphics and machine learning.

I completed my PhD in Computer Science at UC Irvine where I was advised by Gopi Meenaskshisundaram and closely collaborated with Niloy Mitra. Prior to that, I recevied my Masters from University of Washington and Bachelors from BITS Pilani.

                   

News

  • [November, 2020] - Joined CSM as a Research Scientist. [NEW!]
  • [October, 2020] - Our paper on meshing the output points of a point network accepted to 3DV 2020. [NEW!]
  • [August, 2020] - Our paper on style compatibility of 3D furnitures accepted to Pacific Graphics 2020. [NEW!]
  • [March, 2020] - Gave a 3hr talk (slides available) on 3D Deep Learning @ UCI.
  • [August, 2019] - I received the travel award to attend BMVC 2019.
  • [July, 2019] - Our paper on Learning Embedding of 3D models with Quadric Loss accepted for Oral Presentation (acceptance rate - 4.6%) at BMVC 2019.
  • show more

Research

I enjoy solving problems at the intersection of computer vision, graphics and 3D deep learning. In the past, I have also worked on low-level vision, medical image processing and 3D visualization.

Datasets
coming soon

High-Quality 3D Models (Coming Soon)

High quality mesh dataset comprising of 3D models which contain artistic embellishments and decorations.
mouseBrain

3D Annotated Mouse Brain Model

A 3D annotated mouse brain model constructed from the 2D Allen Reference Atlas. Our 3D model has correct topology and geometry and can be virtually sliced in arbritary directions and intervals serving as a template for automatic registration.

v1 - with 20 major anatomical regions
v2 - with 166 major anatomical regions (Coming Soon)

3D Deep Learning
3DV_2020

GAMesh: Guided and Augmented Meshing for Deep Point Networks
Nitin Agarwal, Gopi Meenakshisundaram
International Conference on 3D Vision (3DV), 2020
project page / poster / bibtex

We propose a meshing algorithm to generate a surface with correct topology for the output points of a point network. GAMesh can be used both in post-processing to mesh the output points or to train the point network to directly optimize the vertex positions of the final 3D mesh.
PG_talk

Image-Driven Furniture Style for Interactive 3D Scene Modeling
Tomer Weiss, Ilkay Yildiz, Nitin Agarwal, Esra Ataer-Cansizoglu, Jae-Woo Choi
Pacific Graphics, 2020
paper / bibtex / video

We infer the style of 3D furniture models by training a variation of siamese network on scene images. Our style estimation not only embodies geometry, but also other elements reflected in scene images inlcuding color, texture, material, illumination and the use of space.
3DL_talk

Tutorial on 3D Deep Learning
Nitin Agarwal
Course on Advanced Computer Graphics, Winter 2020
slides

I gave a 3hr talk on 3D deep learning in the graduate class (CS-211B) taught by Prof. Gopi Meenaskshisundaram at UCI, where I provided an overview of the area along with a lot of resources useful for anyone starting research in this direction.
bmvc_2019

Learning Embedding of 3D models with Quadric Loss
Nitin Agarwal, Sung-Eui Yoon, Gopi Meenakshisundaram
British Machine Vision Conference (BMVC), 2019  (Oral Presentation)
project page / slides / poster / bibtex

We propose a new point-to-surface based loss function named Quadric Loss, which penalizes displacements of points in the normal direction thereby preserving sharp features and edges in the output reconstruction. Its differentiable and can be easily incorporated into any point/mesh based network.
Multi-View Geometry & 3D Reconstruction
wacv_2018

Towards Automated Transcription of Label Text from Pinned Insect Collections
Nitin Agarwal, Nicola Ferrier, Mark Hereld
Winter Conference on Application of Computer Vision (WACV), 2018
project page / slides / poster / bibtex

We develop an image processing pipeline which demonstrates that, with some computation, it is feasible to produce label images that are OCR-ready from images of "intact” pinned insect specimens.
eScience

Designing a high-throughput pipeline for digitizing pinned insects
Mark Hereld, Nicola Ferrier, Nitin Agarwal, Petra Sierwald
BigDig Workshop (eScience), 2017  (Oral Presentation)
project page / slides / bibtex

We design a camera rig setup for multi-view reconstruction of small pinned insects (found in museum collections) with their labels using Lytro cameras.
Low Level Vision, Computer Graphics & 3D Visualization
mouseBrain

Geometry Processing of Conventionally Produced Mouse Brain Slice Images
Nitin Agarwal, Xiangmin Xu, Gopi Meenakshisundaram
Journal of Neuroscience Methods, 2018
project page / bibtex

We develop techniques and algorithms for automatic registration and 3D reconstruction of conventionally produced mouse brain slices in a standardized atlas space.
miccai

Automatic Detection of Histological Artifacts in Mouse Brain Slice Images
Nitin Agarwal, Xiangmin Xu, Gopi Meenakshisundaram
Workshop on Medical Computer Vision: Algorithms for Big Data (MICCAI), 2016  (Oral Presentation)
project page / slides / poster / bibtex

Histological artifacts are extremely common in conventional histological procedures. In this work we present a new method to automatically detect and ignore such artifacts for achieveing accurate registration.
icvgip

Robust Registration of Mouse Brain Slice with Severe Histological Artifacts
Nitin Agarwal, Xiangmin Xu, Gopi Meenakshisundaram
Indian Conference on Computer Vision, Graphics & Image Processing (ICVGIP), 2016  
project page / poster / bibtex

We propose a method for non-linear registration of mouse brain histology slices (with various histological artifacts) to a standardized atlas space.
Projects before PhD (click to expand)
Academic Services
review

Conferences & Journals Reviewed

Vision & Graphics: Graphical Models (2020), Pacific Graphics (2019), TVCG, I3D (2015), Visual Computer.

Medical Image Processing: PLOS ONE, Bioinformatics, Neuroscience Methods, Methods in Ecology & Evolution.

Personal



In my spare time I love to do photography. Do check out my images.


Thanks Jon!