Eurybia
A Python library for monitoring model and data drift over time, generating insightful HTML reports for AI governance.
What is Eurybia?
Eurybia is a Python library that monitors model drift and data drift over time, helping to secure model deployment with data validation. It generates HTML reports to visualize changes between baseline and production datasets, aiding in model auditing and AI governance. The tool uses a binary classifier to detect drift and provides explainability for feature contributions.
Data scientists and ML engineers who need to monitor and validate machine learning models in production, as well as teams focused on AI governance and model auditing.
Developers choose Eurybia for its comprehensive, automated drift detection and insightful visual reports that facilitate collaboration between technical and non-technical stakeholders. Its integration with explainability tools like Shapash and focus on production ML lifecycle management set it apart.
Overview
⚓ Eurybia monitors model drift over time and securizes model deployment with data validation
Use Cases
Best For
- Monitoring data drift in production machine learning models
- Validating data before deploying ML models to production
- Generating audit reports for AI governance and compliance
- Collaborating between data scientists and analysts on model performance
- Tracking model performance evolution over time
- Explaining feature contributions to data drift
Not Ideal For
- Projects requiring real-time, low-latency drift detection for immediate model intervention
- Teams needing concept drift analysis without waiting for future updates (as noted in the roadmap)
- Environments where Python is not the primary language or where minimal library dependencies are critical
Pros & Cons
Pros
Combines data drift and model drift detection using a binary classifier with AUC metrics, providing a unified view of model health over time, as highlighted in the quickstart and features section.
Integrates with Shapash to offer detailed feature importance and contribution analysis for the drift classifier, helping prioritize impactful changes and enhancing transparency, as shown in the report visualizations.
Generates interactive HTML reports with dynamic visualizations like scatter plots and distribution comparisons, facilitating discussion between technical and non-technical stakeholders, as emphasized in the philosophy and features.
Validates data before deployment by comparing production datasets to baseline, ensuring consistency and reducing risks, with step-by-step examples in the tutorials.
Cons
As admitted in the roadmap, Eurybia does not yet support concept drift analysis, limiting its ability to detect changes in the relationship between features and target without manual workarounds.
Designed for periodic reporting via schedulers like Airflow, not for real-time monitoring, which may not suit dynamic production environments needing instant alerts.
Relies on libraries like Catboost and Panel, which can lead to compatibility issues and increased installation complexity, as noted in the installation instructions with warnings about Python versions.
Frequently Asked Questions
Related Projects
PyTorch - Tensors and Dynamic neural networks in Python with strong GPU accelerationTensors and Dynamic neural networks in Python with strong GPU acceleration
kerasDeep Learning for humans
streamlitStreamlit — A faster way to build and share data apps.
gradioBuild and share delightful machine learning apps, all in Python. 🌟 Star to support our work!
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.