How to install rpg_trajectory_evaluation on Ubuntu 20.04?

Installation is straightforward via cloning the repository, but you'll need to manually install dependencies like numpy, matplotlib, colorama, and ruamel.yaml using pip for Python 2. Note that Python 2 may require additional setup or virtual environments on newer Ubuntu versions.

rpg_trajectory_evaluation vs EVO for trajectory analysis

rpg_trajectory_evaluation is tailored for VO/VIO research with KITTI-standard relative errors and batch comparison features, while EVO is more general-purpose with Python 3 support and easier command-line usage. Choose rpg for academic benchmarking focused on odometry, or EVO for broader SLAM evaluation with modern Python.

How to convert ROS bag files for use with this toolbox?

Use the provided bag_to_pose.py script under scripts/dataset_tools to extract PoseStamped messages into the required stamped_groundtruth.txt format. Ensure timestamps are in seconds and quaternions have the w component last as specified in the README.

What alignment method should I use for monocular visual odometry?

For monocular VO, use the sim3 (similarity transformation) alignment to account for scale ambiguity, as recommended in the evaluation parameters section. This is the default if no eval_cfg.yaml is provided.

How to batch evaluate multiple algorithms across datasets?

Create a YAML configuration file under scripts/analyze_trajectories_config specifying algorithms and datasets, then run analyze_trajectories.py with parameters like --output_dir and --platform. The README provides an example command for Euroc datasets.

Does rpg_trajectory_evaluation support Python 3?

No, the README clearly states that only Python 2 is currently supported, which is a major limitation. Users have reported forks or manual patches for Python 3, but the official version lacks this compatibility.

How to change the sub-trajectory lengths for relative error calculation?

Modify the RelDistances or RelDistancePercentages fields in the YAML configuration file for analyze_trajectories.py, or set subtraj_lengths directly when using the Trajectory class programmatically as described in the customization section.

Open-Awesome

rpg_trajectory_evaluation

MITPythonv0.1

A Python toolbox for quantitative evaluation of visual(-inertial) odometry trajectories using alignment methods and error metrics.

GitHub

1.2k stars374 forks0 contributors

What is rpg_trajectory_evaluation?

rpg_trajectory_evaluation is a Python toolbox for quantitatively evaluating the accuracy of trajectories produced by visual odometry (VO) and visual-inertial odometry (VIO) algorithms. It computes standard error metrics like Absolute Trajectory Error (ATE) and Relative Error (RE) after aligning estimated trajectories with ground truth data. The toolbox automates the generation of plots and statistical tables, streamlining the benchmarking process for SLAM and odometry research.

Target Audience

Researchers and developers working on visual odometry, visual-inertial odometry, SLAM, or robotics who need to rigorously evaluate and compare the performance of their trajectory estimation algorithms.

Value Proposition

It provides a standardized, easy-to-use implementation of common trajectory evaluation methods, saving time and ensuring consistency in benchmarking. The ability to batch-evaluate multiple algorithms and datasets with customizable parameters makes it particularly valuable for academic research and algorithm competitions.

Overview

Toolbox for quantitative trajectory evaluation of VO/VIO

Use Cases

Best For

Benchmarking VO/VIO algorithms against ground truth trajectories
Generating publication-ready plots and error statistics for research papers
Comparing the performance of multiple odometry algorithms across different datasets
Evaluating trajectory accuracy in robotics and autonomous systems
Automating the quantitative analysis of SLAM system outputs
Converting and analyzing trajectory data from formats like EuRoC or ROS bags

Not Ideal For

Projects requiring real-time error feedback during algorithm execution
Development environments that have migrated exclusively to Python 3
Teams with proprietary or highly custom data formats lacking built-in converters
Users who need a graphical interface for interactive analysis and visualization

Pros & Cons

Pros

Standardized Benchmark Metrics

Implements Absolute Trajectory Error (ATE) and Relative Error (RE) exactly as defined in the KITTI benchmark, ensuring consistency with widely accepted research standards.

Batch Evaluation Capabilities

Can compare multiple algorithms across datasets with a single command using YAML configuration files, automating the generation of comparative plots and LaTeX tables.

Flexible Trajectory Alignment

Supports SE3, Sim3, and posyaw alignment methods to handle different sensor setups like monocular, stereo, or visual-inertial systems, as detailed in the evaluation parameters.

Paper-Ready Outputs

Directly produces publication-quality PDF plots and formatted LaTeX tables for RMSE statistics, reducing manual post-processing for academic papers.

Built-in Data Converters

Includes utilities to transform common formats like EuRoC datasets and ROS bags into the required pose file structure, simplifying data preparation.

Cons

Python 2 Only

The README explicitly states support only for Python 2, which is outdated and incompatible with modern Python 3 ecosystems, limiting adoption and maintenance.

Rigid Folder Structure

Requires specific file naming (e.g., stamped_groundtruth.txt) and directory organization, making ad-hoc or dynamic analyses cumbersome without restructuring data.

Complex Configuration

Setup involves multiple YAML files for parameters and datasets, with command-line options that are extensive and may require trial-and-error for new users.

No Real-Time Integration

Designed purely for post-processing evaluation; it cannot be easily embedded into live systems for online trajectory monitoring or debugging.

Frequently Asked Questions

Related Projects

Kalman-and-Bayesian-Filters-in-Python

Kalman Filter book using Jupyter Notebook. Focuses on building intuition and experience, not formal proofs. Includes Kalman filters,extended Kalman filters, unscented Kalman filters, particle filters, and more. All exercises include solutions.

Stars19,026

Forks4,494

Last commit1 year ago

evo

Python package for the evaluation of odometry and SLAM

Stars4,249

Forks791

Last commit6 days ago

PROJ

PROJ - Cartographic Projections and Coordinate Transformations Library

Stars1,985

Forks891

Last commit4 days ago

robot_localization

robot_localization is a package of nonlinear state estimation nodes. The package was developed by Charles River Analytics, Inc. Please ask questions on answers.ros.org.

Stars1,903

Forks1,020

Last commit1 month ago

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub