Kimera-Semantics
Real-time 3D semantic reconstruction library for robotics, building dense metric-semantic maps from 2D sensor data.
What is Kimera-Semantics?
Kimera-Semantics is an open-source library that performs real-time 3D semantic reconstruction from 2D sensor data, such as RGB-D or stereo cameras. It builds dense, voxel-based maps where each element is labeled with a semantic class (like 'wall' or 'floor'), enabling robots to understand their environment beyond just geometry. It solves the problem of creating rich, semantically annotated 3D maps for autonomous navigation and scene understanding.
Robotics researchers and engineers working on autonomous systems, semantic SLAM, 3D scene understanding, and robotic perception who need real-time, dense semantic mapping capabilities.
Developers choose Kimera-Semantics for its real-time performance, tight integration with the ROS ecosystem, and lean design that builds on the robust Voxblox library. It provides a practical, open-source solution for adding semantic layers to geometric maps without the overhead of custom mesh management.
Overview
Real-Time 3D Semantic Reconstruction from 2D data
Use Cases
Best For
- Building real-time metric-semantic maps for autonomous drones or ground robots
- Research in semantic SLAM and dense 3D scene understanding
- Robotic applications requiring environment semantics for task planning
- Integrating semantic reconstruction into existing ROS-based perception pipelines
- Projects needing voxel-based semantic mapping with efficient ESDF management
- Educational or experimental platforms for robotic perception and mapping
Not Ideal For
- Projects requiring instance segmentation or object-level semantic understanding, as it focuses on dense scene segmentation without instance awareness
- Non-ROS applications or systems not using the Robot Operating System, due to its deep integration and reliance on ROS toolchains
- Scenarios with strict real-time constraints on embedded or low-power hardware, as it requires significant compute resources for dense reconstruction
- Teams needing out-of-the-box 2D semantic mapping without 3D reconstruction, since it specializes in voxel-based 3D semantics
Pros & Cons
Pros
The 'fast' integrator type achieves updates in approximately 0.1 seconds, an order of magnitude faster than the 'merged' method, enabling near-incremental processing for dynamic environments.
Assigns semantic labels like floor or wall to every voxel in the 3D reconstruction, providing rich scene understanding beyond pure geometry for autonomous systems.
Comes with pre-configured launch files and rviz setups, making deployment straightforward in ROS-based robotic pipelines, as shown in the simulation and Euroc dataset examples.
Leverages Voxblox for efficient ESDF management, avoiding custom meshing code and building on a proven library, which keeps the code lean and focused on semantics.
Cons
Installation and operation are tightly coupled with ROS, requiring catkin, wstool, and system dependencies, which adds complexity for non-ROS users or alternative frameworks.
Unlike Voxblox++, it lacks instance-aware segmentation capabilities, limiting its use for applications that require object detection or manipulation based on individual instances.
Installation involves multiple steps with wstool for dependencies and catkin builds, which can be error-prone and daunting for those new to ROS ecosystems.
The README does not detail support for custom semantic classes; it relies on input from semantic segmentation models, potentially restricting flexibility for novel use cases.
Frequently Asked Questions
Related Projects
detectron2Detectron2 is a platform for object detection, segmentation and other visual recognition tasks.
EasyOCRReady-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.
imgaugImage augmentation for machine learning experiments.
meshroomNode-based Visual Programming Toolbox
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.