Real-time 3D semantic reconstruction library for robotics, building dense metric-semantic maps from 2D sensor data.
Kimera-Semantics is an open-source library that performs real-time 3D semantic reconstruction from 2D sensor data, such as RGB-D or stereo cameras. It builds dense, voxel-based maps where each element is labeled with a semantic class (like 'wall' or 'floor'), enabling robots to understand their environment beyond just geometry. It solves the problem of creating rich, semantically annotated 3D maps for autonomous navigation and scene understanding.
Robotics researchers and engineers working on autonomous systems, semantic SLAM, 3D scene understanding, and robotic perception who need real-time, dense semantic mapping capabilities.
Developers choose Kimera-Semantics for its real-time performance, tight integration with the ROS ecosystem, and lean design that builds on the robust Voxblox library. It provides a practical, open-source solution for adding semantic layers to geometric maps without the overhead of custom mesh management.
Real-Time 3D Semantic Reconstruction from 2D data
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
The 'fast' integrator type achieves updates in approximately 0.1 seconds, an order of magnitude faster than the 'merged' method, enabling near-incremental processing for dynamic environments.
Assigns semantic labels like floor or wall to every voxel in the 3D reconstruction, providing rich scene understanding beyond pure geometry for autonomous systems.
Comes with pre-configured launch files and rviz setups, making deployment straightforward in ROS-based robotic pipelines, as shown in the simulation and Euroc dataset examples.
Leverages Voxblox for efficient ESDF management, avoiding custom meshing code and building on a proven library, which keeps the code lean and focused on semantics.
Installation and operation are tightly coupled with ROS, requiring catkin, wstool, and system dependencies, which adds complexity for non-ROS users or alternative frameworks.
Unlike Voxblox++, it lacks instance-aware segmentation capabilities, limiting its use for applications that require object detection or manipulation based on individual instances.
Installation involves multiple steps with wstool for dependencies and catkin builds, which can be error-prone and daunting for those new to ROS ecosystems.
The README does not detail support for custom semantic classes; it relies on input from semantic segmentation models, potentially restricting flexibility for novel use cases.