Share This Author
Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environments
- Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, Stefan Lee
- Computer ScienceEuropean Conference on Computer Vision
- 6 April 2020
A language-guided navigation task set in a continuous 3D environment where agents must execute low-level actions to follow natural language navigation directions is developed, suggesting that performance in prior `navigation-graph' settings may be inflated by the strong implicit assumptions.
Waypoint Models for Instruction-guided Navigation in Continuous Environments
- Jacob Krantz, Aaron Gokaslan, Dhruv Batra, Stefan Lee, Oleksandr Maksymets
- Computer ScienceIEEE International Conference on Computer Vision
- 1 October 2021
A class of language-conditioned waypoint prediction networks is developed to examine the role of action spaces in language-guided visual navigation and finds more expressive models result in simpler, faster to execute trajectories, but lower-level actions can achieve better navigation metrics by approximating shortest paths better.
Where Are You? Localization from Embodied Dialog
- Meera Hahn, Jacob Krantz, Peter Anderson
- Computer ScienceConference on Empirical Methods in Natural…
- 1 November 2020
We present Where Are You? (WAY), a dataset of ~6k dialogs in which two humans -- an Observer and a Locator -- complete a cooperative localization task. The Observer is spawned at random in a 3D…
Language-Agnostic Syllabification with Neural Sequence Labeling
- Jacob Krantz, Max W. Dulin, P. Palma
- Computer Science18th IEEE International Conference On Machine…
- 29 September 2019
This work presents a novel approach to the syllabification problem which leverages modern neural network techniques and shows that the network is competitive with state of the art systems in syllabifying English, Dutch, Italian, French, Manipuri, and Basque datasets.
Syllabification by phone categorization
- Jacob Krantz, Max W. Dulin, P. Palma, M. VanDam
- Computer ScienceAnnual Conference on Genetic and Evolutionary…
- 6 July 2018
A hybrid genetic algorithm constructs a categorization of phones optimized for syllabification on top of a hidden Markov model sequence classifier to find syllable boundaries.
Abstractive Summarization Using Attentive Neural Techniques
This work modify and optimize a translation model with self-attention for generating abstractive sentence summaries, and proposes a new approach based on the intuition that an abstractive model requires an Abstractive evaluation.
Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environments
This work explores the gap in performance between the standard VLN setting built on topological environments where navigation is abstracted away and the VLLN-CE setting where agents must navigate continuous 3D environments using low-level actions, and demonstrates the potential for this direction.
Iterative Vision-and-Language Navigation
It is found that extending the implicit memory of high-performing transformer VLN agents is not sufficient for IVLN, but agents that build maps can benefit from environment persistence, motivating a renewed focus on map-building agents in VLn.
Retrospectives on the Embodied AI Workshop
This analysis focuses on 13 challenges presented at the Embodied AI Workshop at CVPR, grouped into three themes: visual navigation, rearrangement, and embodied vision-and-language.