A curated list of algorithms and academic papers for auditing black-box algorithms like recommendation systems and classifiers.
Awesome Audit Algorithms is a curated repository of academic papers and algorithmic techniques designed for auditing black-box algorithms. It provides methodologies to scrutinize remote, proprietary systems—like recommendation engines and classifiers—where internal logic is hidden from users and institutions. The collection helps researchers and auditors infer information about these algorithms through queries and analysis of their input-output behavior.
Researchers, data scientists, and practitioners in AI ethics, algorithmic fairness, and security who need to audit proprietary or third-party machine learning systems. It's also valuable for academics studying transparency, accountability, and adversarial testing of algorithms.
It centralizes cutting-edge research on black-box auditing, saving time for professionals who need proven methods to analyze opaque algorithms. Unlike generic AI resources, it specifically focuses on audit techniques, offering a specialized toolkit for ensuring algorithmic accountability and compliance.
A curated list of algorithms and papers for auditing black-box algorithms.
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Curates papers from 2005 to 2026, spanning fairness, privacy, model extraction, and adversarial attacks, as evidenced by the extensive yearly listings under 'Papers'.
Specifically targets algorithms for auditing remote, proprietary systems, addressing a critical gap in AI transparency, with methods like query-based inference and API interactions detailed throughout.
Includes papers from top-tier conferences such as NeurIPS, ICML, and AAAI, ensuring peer-reviewed, state-of-the-art methodologies for reliable research.
Covers diverse topics from bias detection to model stealing, making it valuable for researchers in ethics, security, and machine learning, as highlighted in sections like 'Fairness & Bias Detection' and 'Model Extraction'.
Regularly updated with recent papers, including 2025 and 2026 entries, and links to related events, keeping the resource current with evolving audit trends.
Primarily a bibliography with few code links; most papers are theoretical, requiring significant effort to translate research into working tools, which limits immediate usability for practitioners.
The list is raw and unsynthesized, offering no summaries, comparisons, or learning paths, forcing users to navigate complex papers independently without curated insights.
As a static GitHub repository, it lacks forums, discussions, or active support beyond paper additions, reducing opportunities for collaboration and real-world feedback.
Heavily focused on theoretical research with minimal case studies or applied examples, making it less accessible for teams needing hands-on audit solutions in industry settings.