treebeard
A GitHub Action that automatically tests Jupyter notebooks from top to bottom using nbmake and pytest.
What is treebeard?
nbmake-action is a GitHub Action that automatically tests Jupyter notebooks by running them from top to bottom. It integrates with pytest via the nbmake package to catch execution errors and ensure notebook reliability in CI/CD pipelines. The tool helps maintain the quality of scientific and technical documentation by automating notebook validation.
Scientists and developers who write documentation or code in Jupyter notebooks and want to automatically test them after every commit using GitHub Actions.
Developers choose nbmake-action for its simplicity and seamless integration with GitHub workflows, providing an easy way to automate notebook testing without complex setup. It offers a straightforward alternative to manually configuring nbmake directly, though advanced users may prefer the flexibility of the underlying package.
Overview
GitHub Action for testing notebooks
Use Cases
Best For
- Automating Jupyter notebook testing in GitHub CI/CD pipelines
- Ensuring scientific documentation remains executable after updates
- Catching runtime errors in notebook-based tutorials and examples
- Integrating notebook validation into data science project workflows
- Maintaining quality of educational or research notebook repositories
- Testing notebooks across multiple directories with glob patterns
Not Ideal For
- Advanced users needing fine-grained control over pytest or nbmake configurations
- Projects using CI/CD platforms other than GitHub Actions, like GitLab CI or Jenkins
- Teams with complex existing workflows who prefer scripting notebook tests directly
Pros & Cons
Pros
Offers a simple YAML configuration for quickly adding notebook testing to GitHub workflows, as demonstrated in the Quick Start example with minimal setup steps.
Executes notebooks sequentially from top to bottom to catch runtime errors, ensuring scientific documentation remains reliable after every commit.
Allows specifying directories and glob patterns to test specific notebooks, using parameters like 'path' and 'notebooks' for precise control.
Integrates with the nbmake package via pytest, enabling compatibility with existing testing setups and tools for consistent validation.
Cons
As noted in the README, users with GitHub Actions experience might prefer nbmake directly for more control over configurations, making this action less adaptable for advanced needs.
The action is exclusive to GitHub Actions, so it cannot be used with other CI/CD systems, limiting its applicability in diverse development environments.
Being a wrapper around nbmake, updates or new features from the underlying package might not be immediately available, causing delays in bug fixes or enhancements.
Frequently Asked Questions
Related Projects
pointblankData validation toolkit for assessing and monitoring data quality.
nbvalA py.test plugin to validate Jupyter notebooks
ipytestPytest in IPython notebooks.
treonEasy to use test framework for Jupyter Notebooks
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.