MLKit - ARKit

MLKit-ARKit is a Swift-based example project that integrates Firebase ML Kit's object detection with Apple's ARKit to create an augmented reality labeling experience. It processes live camera frames from ARKit to identify objects and renders corresponding 3D text labels in the AR scene using SceneKit.

Key Features

• Real-time Object Detection — Continuously analyzes ARKit camera frames via Firebase ML Kit on a background thread to maintain AR performance.
• 3D AR Labeling — Places detected object names as 3D text labels at ARKit feature points, anchored in the real-world view.
• Threaded Processing — Uses a dedicated dispatch queue to run ML inference without blocking ARKit's 60 FPS rendering loop.
• Tap-to-Label Interaction — Allows users to tap to place labels based on the most recent detection at the screen's center raycast point.

Philosophy

The project demonstrates a practical integration of on-device machine learning with augmented reality, prioritizing real-time performance by decoupling ML processing from AR rendering threads.