How to test a custom mypy plugin with pytest-mypy-plugins?

Use the YAML spec to define test cases with the plugin imported, set MYPYPATH in the env section to point to your plugin, and run pytest. The README shows examples with environment variables for simulating import paths.

pytest-mypy-plugins vs writing mypy tests in Python – which is better?

pytest-mypy-plugins is better for structured, YAML-based testing integrated with pytest, while writing tests in Python offers more flexibility and debugability. Choose based on team preference and need for automation.

How do I set up environment variables for mypy tests in YAML?

Add an 'env' key in your YAML test case with variables like PYTHONPATH, as shown in the README. This allows simulating different paths without modifying the system environment.

Can pytest-mypy-plugins be used with Django type stubs?

Yes, it's commonly used for testing Django type stubs; examples in the repository include Django-related tests. You can define files and env vars to mimic Django project structures.

What are the performance implications of using pytest-mypy-plugins?

Running mypy in a subprocess adds overhead, and using --mypy-same-process can cause import cache issues. It's best for CI/CD where isolated runs are acceptable, but may slow down local testing.

How to handle relative paths in pytest-mypy-plugins tests?

Convert relative paths to absolute ones using tools like $(pwd) in shell commands or set --mypy-same-process flag, as advised in the README to avoid temporary directory issues.

pytest-mypy-plugins — Mypy Testing Pytest Plugin

What is pytest-mypy-plugins?

pytest-mypy-plugins is a pytest plugin designed for testing mypy's type checking functionality. It allows developers to write test cases in YAML format to verify that mypy correctly infers types, validates custom stubs, and works with plugins. This tool is essential for ensuring that type annotations and mypy configurations behave as intended across different code scenarios.

Target Audience

Python developers and library maintainers who use mypy for static type checking and need to test custom type stubs, plugins, or complex type annotations. It is particularly useful for projects with extensive type definitions or those developing mypy plugins.

Value Proposition

Developers choose pytest-mypy-plugins because it integrates seamlessly with pytest, providing a familiar testing workflow while offering a specialized framework for type safety validation. Its YAML-based test specification makes it easy to write and maintain comprehensive type tests, reducing the risk of type-related bugs.

pytest plugin for testing mypy types, stubs, and plugins

Use Cases

Best For

Testing custom mypy plugins for correct type inference
Validating type stubs for third-party libraries
Ensuring complex type annotations behave as expected in large codebases
Automating type safety checks in CI/CD pipelines
Writing parametrized tests for multiple type scenarios
Debugging mypy configuration and environment issues

Not Ideal For

Projects that do not use mypy or any static type checking
Teams preferring to write type tests directly in Python code rather than YAML specifications
Environments using unittest or other testing frameworks instead of pytest
Simple type validation where running mypy on source files is sufficient without automated test suites

Pros & Cons

Pros

YAML Test Specification

Allows writing test cases in a simple YAML format with code snippets and expected mypy outputs, making it easy to define and maintain comprehensive type tests.

Flexible Type Assertions

Supports both custom reveal_type for detailed output matching and standard typing.assert_type, providing multiple ways to verify type inferences based on project needs.

Parametrized Testing Support

Enables parameterization similar to pytest's @pytest.mark.parametrize, allowing efficient testing of multiple scenarios without code duplication, as shown in the YAML examples.

Environment Configuration

Permits setting environment variables like PYTHONPATH and MYPYPATH directly in YAML specs, facilitating testing of plugins and stubs in simulated import paths.

Schema Validation

Provides a JSONSchema for YAML files, offering editor auto-completion and error checking to reduce mistakes in test specifications.

Cons

Path Configuration Hassles

Relative paths for PYTHONPATH or MYPYPATH often fail when mypy runs in a subprocess, requiring absolute paths or same-process mode, which complicates setup and debugging.

Niche to Mypy

Exclusively designed for mypy testing, making it irrelevant for projects using alternative type checkers like Pyright or pytype, limiting its broader utility.

YAML Learning Curve

Relies on YAML files for test definitions, which can be less intuitive and harder to debug for developers accustomed to writing tests in Python with standard IDE tools.

Frequently Asked Questions

What is pytest-mypy-plugins?

Target Audience

Value Proposition

Use Cases

Best For

Testing custom mypy plugins for correct type inference
Validating type stubs for third-party libraries
Ensuring complex type annotations behave as expected in large codebases
Automating type safety checks in CI/CD pipelines
Writing parametrized tests for multiple type scenarios
Debugging mypy configuration and environment issues

Not Ideal For

Projects that do not use mypy or any static type checking
Teams preferring to write type tests directly in Python code rather than YAML specifications
Environments using unittest or other testing frameworks instead of pytest
Simple type validation where running mypy on source files is sufficient without automated test suites

Pros & Cons

Pros

YAML Test Specification

Allows writing test cases in a simple YAML format with code snippets and expected mypy outputs, making it easy to define and maintain comprehensive type tests.

Flexible Type Assertions

Supports both custom reveal_type for detailed output matching and standard typing.assert_type, providing multiple ways to verify type inferences based on project needs.

Parametrized Testing Support

Enables parameterization similar to pytest's @pytest.mark.parametrize, allowing efficient testing of multiple scenarios without code duplication, as shown in the YAML examples.

Environment Configuration

Permits setting environment variables like PYTHONPATH and MYPYPATH directly in YAML specs, facilitating testing of plugins and stubs in simulated import paths.

Schema Validation

Provides a JSONSchema for YAML files, offering editor auto-completion and error checking to reduce mistakes in test specifications.

Cons

Path Configuration Hassles

Relative paths for PYTHONPATH or MYPYPATH often fail when mypy runs in a subprocess, requiring absolute paths or same-process mode, which complicates setup and debugging.

Niche to Mypy

Exclusively designed for mypy testing, making it irrelevant for projects using alternative type checkers like Pyright or pytype, limiting its broader utility.

YAML Learning Curve

Relies on YAML files for test definitions, which can be less intuitive and harder to debug for developers accustomed to writing tests in Python with standard IDE tools.

Frequently Asked Questions

pytest-mypy-plugins

What is pytest-mypy-plugins?

Overview

Use Cases

Best For

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions

Related Projects

Found a gem we're missing?

pytest-mypy-plugins

What is pytest-mypy-plugins?

Overview

Use Cases

Best For

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions

Related Projects

Found a gem we're missing?