How do I set up Marathon for testing on iOS simulators?

You need to create a Marathondevices file listing simulators by UDID or device profile, and specify iOS app bundles in the Marathonfile. This allows Marathon to manage device pools, but it requires manual configuration of simulators, which can be complex for large teams.

Marathon or Firebase Test Lab: which is better for mobile testing?

Marathon offers more control over execution and cost, especially with custom device farms, while Firebase Test Lab is a fully managed service. Choose Marathon if you need fine-grained optimization and have infrastructure to manage, or Firebase for a simpler, cloud-native approach.

Does Marathon support testing for web applications?

No, Marathon is specifically designed for Android and iOS mobile applications only. For web testing, you would need to use other tools like Selenium or Playwright, as it doesn't extend beyond mobile platforms.

What are the system requirements to run Marathon?

Marathon requires Java Runtime Environment 17 or higher to execute. For iOS testing, you also need access to macOS instances or simulators, which can be local or remote via SSH, as mentioned in the configuration documentation.

How does Marathon handle flaky tests in CI/CD pipelines?

It uses preventive retries based on real-time statistical data to queue retries for tests likely to fail, and post-factum retries with quotas for unexpected failures. This helps stabilize test runs, but relies on historical data which may need time to accumulate.

Can I integrate Marathon with Jenkins or GitHub Actions?

Yes, Marathon can be integrated into CI/CD pipelines by executing it via command line in scripts. The output reports in the specified directory can be used for further processing, though direct plugin support may be limited compared to native tools.

Open-Awesome

Marathon Runner

GPL-2.0Kotlin0.10.4

A fast, platform-independent test runner for Android and iOS that optimizes for execution speed and stability.

Visit Website GitHub

639 stars123 forks0 contributors

What is Marathon Runner?

Marathon is a cross-platform test runner that executes tests for Android and iOS applications with a focus on speed and reliability. It addresses flakiness in tests and environments through strategies like preventive retries, test sorting, and sharding, optimizing the balance between performance and stability. The tool provides platform-specific configurations and supports custom hardware farms via its API.

Target Audience

Mobile developers and QA engineers working on Android and iOS applications who need efficient, reliable test execution in CI/CD pipelines. It's particularly useful for teams dealing with flaky tests or large test suites requiring parallel execution.

Value Proposition

Developers choose Marathon for its fine-grained control over test execution stability and performance, reducing test run times while managing flakiness. Its platform-independent design and support for custom hardware farms offer flexibility beyond standard cloud testing services.

Overview

Cross-platform test runner

Use Cases

Best For

Running large test suites for Android and iOS apps in parallel
Reducing flaky test failures in CI/CD pipelines
Optimizing test execution time for mobile applications
Managing test runs across custom device farms or simulators
Balancing test stability with execution speed and cost
Integrating automated testing for cross-platform mobile projects

Not Ideal For

Projects requiring testing for web or desktop applications
Teams with very small test suites that don't need advanced parallelization features
Organizations that prefer fully managed, cloud-native testing solutions without infrastructure management

Pros & Cons

Pros

Advanced Performance Optimization

Implements test batching, sharding, and sorting to reduce execution time, prioritizing longer tests to minimize retry impact, as described in the performance overview.

Robust Stability Features

Uses preventive retries based on real-time statistics and post-factum retries with quotas to handle flaky tests effectively, addressing environment and test instability.

Cross-Platform Mobile Support

Provides easy-to-use configurations for Android and iOS via Marathonfile, and an API for custom hardware farms, enabling flexible test execution across platforms.

Full Control Over Execution

Allows teams to balance stability, performance, and cost through customizable configurations, aligning with its philosophy of optimizing testing workflows.

Cons

Complex iOS Device Setup

Requires manual provision of simulator UDIDs or profiles via a Marathondevices file, which can be tedious and error-prone, especially for dynamic environments.

Java Runtime Dependency

Requires JRE 17 or higher, adding an extra layer of infrastructure management for teams not already using Java in their stack.

Limited Platform Scope

Focused solely on Android and iOS, so it lacks built-in support for other application types like web or desktop, limiting its versatility.

Frequently Asked Questions

Related Projects

Kif

Keep It Functional - An iOS Functional Testing Framework

Stars6,243

Forks917

Last commit19 hours ago

EarlGrey

:tea: iOS UI Automation Test Framework

Bluepill is a reliable iOS testing tool that runs UI tests using multiple simulators on a single machine

Stars3,205

Forks230

Last commit1 year ago

ViewInspector

Runtime introspection and unit testing of SwiftUI views

Stars2,617

Forks189

Last commit3 months ago

Community-curated · Updated weekly · 100% open source

Found a gem we're missing?

Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.

Submit a project Star on GitHub