LiFT
A Scala/Spark library for measuring fairness and mitigating bias in large-scale machine learning workflows.
What is LiFT?
LiFT (LinkedIn Fairness Toolkit) is a Scala/Spark library for measuring and mitigating fairness and bias in machine learning workflows. It provides tools to evaluate biases in training data, assess model performance across demographic subgroups, and apply post-processing techniques to promote equality of opportunity. The library is designed to handle large-scale datasets efficiently within existing ML pipelines.
Machine learning engineers, data scientists, and researchers working on large-scale ML systems who need to audit and improve fairness, particularly in enterprise or production environments.
LiFT offers a production-ready, scalable solution for fairness analysis integrated directly with Apache Spark. Unlike many research-oriented tools, it provides configuration-driven Spark jobs and APIs that fit seamlessly into existing ML workflows, with robust support for both measurement and mitigation.
Overview
The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.
Use Cases
Best For
- Auditing training datasets for demographic biases before model training
Related Projects
Found a gem we're missing?
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
