Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping

@article{Rosinol2020KimeraAO,
  title={Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping},
  author={Antoni Rosinol and Marcus Abate and Yun Chang and Luca Carlone},
  journal={2020 IEEE International Conference on Robotics and Automation (ICRA)},
  year={2020},
  pages={1689-1696}
}
We provide an open-source C++ library for real-time metric-semantic visual-inertial Simultaneous Localization And Mapping (SLAM). The library goes beyond existing visual and visual-inertial SLAM libraries (e.g., ORB-SLAM, VINS-Mono, OKVIS, ROVIO) by enabling mesh reconstruction and semantic labeling in 3D. Kimera is designed with modularity in mind and has four key components: a visual-inertial odometry (VIO) module for fast and accurate state estimation, a robust pose graph optimizer for… Expand
Kimera-Multi: a System for Distributed Multi-Robot Metric-Semantic Simultaneous Localization and Mapping
TLDR
Kimera-Multi is able to build accurate 3D metric-semantic meshes, is robust to incorrect loop closures while requiring less computation than state-of-the-art distributed SLAM backends, and is efficient, both in terms of computation at each robot as well as communication bandwidth. Expand
Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems
TLDR
Kimera-Multi is presented, the first multi-robot system that is robust and capable of identifying and rejecting incorrect inter and intra-ro Bot loop closures resulting from perceptual aliasing, and is fully distributed and only relies on local (peerto-peer) communication to achieve distributed localization and mapping. Expand
Superquadric Object Representation for Optimization-based Semantic SLAM
TLDR
This work proposes a pipeline to leverage semantic mask measurements to fit SQ parameters to multi-view camera observations using a multi-stage initialization and optimization procedure, and demonstrates the system’s ability to retrieve randomly generated SQ parameters from multi-View mask observations in preliminary simulation experiments. Expand
RGB-D Object SLAM Using Quadrics for Indoor Environments
TLDR
This paper proposes an object-level semantic SLAM algorithm based on RGB-D data, which uses a quadric surface as an object model to compactly represent the object’s position, orientation, and shape, and innovatively applies a nonparametric pose graph to solve data associations in the back end. Expand
CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations
TLDR
A dense mapping framework to complement sparse visual SLAM systems which takes as input the camera poses, keyframes and sparse points produced by the SLAM system and predicts a dense depth image for every keyframe and can be used not only for local mapping but also globally consistent dense 3D reconstruction through TSDF fusion. Expand
Co-Planar Parametrization for Stereo-SLAM and Visual-Inertial Odometry
TLDR
A novel SLAM framework for stereo and visual inertial odometry estimation and an efficient and robust parametrization of co-planar points and lines which leverages specific geometric constraints to improve camera pose optimization in terms of both efficiency and accuracy. Expand
Visual and Visual-Inertial SLAM: State of the Art, Classification, and Experimental Benchmarking
TLDR
A review of popular SLAM approaches with a focus on vSLAM/viSLAM, both at fundamental and experimental levels and a new classification of a dozen main state-of-the-art methods. Expand
PL-VINS: Real-Time Monocular Visual-Inertial SLAM with Point and Line
TLDR
PL-VINS is presented, a real-time optimization-based monocular VINS method with point and line, developed based on state-of-the-art point-based VINS-Mono based on hidden parameter tuning and length rejection strategy. Expand
Representations and Benchmarking of Modern Visual SLAM Systems
TLDR
The structure of the representations and optimisation problems involved in Spatial AI are discussed, and new synthetic datasets are proposed that include accurate ground truth information about the scene composition as well as individual object shapes and poses. Expand
A Comparison of Modern General-Purpose Visual SLAM Approaches
TLDR
This analysis is motivated to find general purpose, feature complete, and multi-domain VSLAM options to support a broad class of robot applications for integration into the new and improved ROS 2 Nav2 System as suitable alternatives to traditional 2D lidar solutions. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 86 REFERENCES
ORB-SLAM: A Versatile and Accurate Monocular SLAM System
TLDR
A survival of the fittest strategy that selects the points and keyframes of the reconstruction leads to excellent robustness and generates a compact and trackable map that only grows if the scene content changes, allowing lifelong operation. Expand
Maplab: An Open Framework for Research in Visual-Inertial Mapping and Localization
Robust and accurate visual-inertial estimation is crucial to many of today's challenges in robotics. Being able to localize against a prior map and obtain accurate and drift-free pose estimates canExpand
VSO: Visual Semantic Odometry
TLDR
A novel visual semantic odometry (VSO) framework to enable medium-term continuous tracking of points using semantics and demonstrates a significant improvement over state-of-the-art baselines in the context of autonomous driving simply by integrating the authors' semantic constraints. Expand
SLAM++: Simultaneous Localisation and Mapping at the Level of Objects
TLDR
The object graph enables predictions for accurate ICP-based camera to model tracking at each live frame, and efficient active search for new objects in currently undescribed image regions, as well as the generation of an object level scene description with the potential to enable interaction. Expand
Fusion++: Volumetric Object-Level SLAM
TLDR
An online object-level SLAM system which builds a persistent and accurate 3D graph map of arbitrary reconstructed objects is proposed, and performance evaluation shows the approach is highly memory efficient and runs online at 4-8Hz despite not being optimised at the software level. Expand
VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator
TLDR
This paper presents VINS-Mono: a robust and versatile monocular visual-inertial state estimator that is applicable for different applications that require high accuracy in localization and performs an onboard closed-loop autonomous flight on the microaerial-vehicle platform. Expand
Visual-Inertial-Semantic Scene Representation for 3D Object Detection
TLDR
A system to detect objects in three-dimensional space using video and inertial sensors (accelerometer and gyrometer), ubiquitous in modern mobile platforms from phones to drones, and provides a representation of objects in the scene that is persistent. Expand
Keyframe-Based Visual-Inertial SLAM using Nonlinear Optimization
TLDR
A novel approach to tightly integrate visual measurements with readings from an Inertial Measurement Unit (IMU) in SLAM using the powerful concept of ‘keyframes’ to maintain a bounded-sized optimization window, ensuring real-time operation. Expand
Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery
TLDR
This work presents an approach to incrementally build volumetric object-centric maps during online scanning with a localized RGB-D camera and demonstrates that the proposed approach for building instance-level semantic maps is competitive with state-of-the-art methods, while additionally able to discover objects of unseen categories. Expand
SemanticFusion: Dense 3D semantic mapping with convolutional neural networks
TLDR
This work combines Convolutional Neural Networks (CNNs) and a state-of-the-art dense Simultaneous Localization and Mapping (SLAM) system, ElasticFusion, which provides long-term dense correspondences between frames of indoor RGB-D video even during loopy scanning trajectories, and produces a useful semantic 3D map. Expand
...
1
2
3
4
5
...