Share This Author
PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning
- Santhosh K. Ramakrishnan, Devendra Singh Chaplot, Ziad Al-Halah, J. Malik, K. Grauman
- Computer ScienceComputer Vision and Pattern Recognition
- 25 January 2022
A network that predicts two complementary potential functions conditioned on a semantic map and uses them to decide where to look for an unseen object is proposed, a modular approach that disentangles the skills of 'where to look?' for an object and 'how to navigate to $(x,\ y)$?’.
SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency
- Devendra Singh Chaplot, Murtaza Dalal, Saurabh Gupta, J. Malik, R. Salakhutdinov
- Computer ScienceNeural Information Processing Systems
- 2 December 2021
It is demonstrated that the SEAL framework can be used to close the action-perception loop: it improves object detection and instance segmentation performance of a pretrained perception model by just moving around in training environments and the improved perception model can be use to improve Object Goal Navigation.
Retrospectives on the Embodied AI Workshop
This analysis focuses on 13 challenges presented at the Embodied AI Workshop at CVPR, grouped into three themes: visual navigation, rearrangement, and embodied vision-and-language.
Recognizing Scenes from Novel Viewpoints
This work proposes a model which takes as input a few RGB images of a new scene and recognizes the scene from novel viewpoints by segmenting it into semantic categories, and demonstrates its ability to jointly capture semantics and geometry of novel scenes with diverse layouts, object types and shapes.
Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instances
- Jacob Krantz, Stefan Lee, J. Malik, Dhruv Batra, Devendra Singh Chaplot
- Computer ScienceArXiv
- 29 November 2022
The Instance-speciﬁc ImageNav task (InstanceImageNav) is presented, a solution to the problem of embodied visual navigation given an image-goal where an agent is initialized in an unfamiliar environment and tasked with navigating to a location ‘described’ by an image.
Multi-skill Mobile Manipulation for Object Rearrangement
- Jia-Yuan Gu, Devendra Singh Chaplot, Hao Su, J. Malik
- Psychology, Computer ScienceArXiv
- 6 September 2022
This work proposes that the manipulation skills should include mobility to have accessibility in interacting with the target object from multiple locations and at the same time the navigation skill could have multiple end points which lead to successful manipulation, and operationalizes these ideas by implementing mobile manipulation skills rather than stationary ones.