se(3)-TrackNet
A neural network for real-time 6D object pose tracking in video using RGB-D data, trained only on synthetic images.
What is se(3)-TrackNet?
se(3)-TrackNet is a neural network architecture for 6D pose tracking that estimates the position and orientation of known objects in video sequences using RGB-D data. It solves the challenges of occlusion, lack of real-world annotated data, and error drift in long-term tracking by training exclusively on synthetic images. The approach optimizes relative pose between current observations and synthetic renderings conditioned on previous estimates.
Researchers and engineers in robotics, computer vision, and AR/VR who need real-time, robust 6D pose tracking for manipulation, model-based reinforcement learning, or human-robot interaction.
It outperforms alternatives in robustness and computational efficiency (90.9 Hz) while requiring only synthetic data for training, eliminating the need for costly real-world pose annotations. Its novel architecture reduces domain shift and uses Lie Algebra for effective orientation representation.
Overview
[IROS 2020] se(3)-TrackNet: Data-driven 6D Pose Tracking by Calibrating Image Residuals in Synthetic Domains
Use Cases
Best For
- Robotic manipulation tasks requiring real-time object pose feedback
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
