Question 1

How to audit a black-box algorithm for bias using research papers?

Accepted Answer

Start with papers like 'Fairness Auditing with Multi-Agent Collaboration' (2024) or 'Certifying and Removing Disparate Impact' (2015), which detail statistical tests and query-based methods. Implement techniques by adapting code from linked repositories, though be prepared for academic complexity.

Question 2

What are the best papers on model extraction attacks in this list?

Accepted Answer

Key papers include 'Stealing Machine Learning Models via Prediction APIs' (2016) and 'Knockoff Nets: Stealing Functionality of Black-Box Models' (2019). These provide foundational approaches for reverse-engineering models through API queries, with some offering code links for implementation.

Question 3

Awesome Audit Algorithms vs Awesome AI Ethics: which is better for audit methods?

Accepted Answer

Awesome Audit Algorithms is superior for specialized, black-box audit techniques, as it curates focused research papers. In contrast, Awesome AI Ethics covers broader ethics topics, so choose based on your need for depth in auditing versus general ethical frameworks.

Question 4

How to implement membership inference attacks from these papers?

Accepted Answer

Refer to 'Membership Inference Attacks Against Machine Learning Models' (2017) for methodology. Check for GitHub links in the paper descriptions, but note that implementations often require adaptation, as this list doesn't provide ready-to-run tools.

Question 5

Is this list useful for GDPR compliance auditing?

Accepted Answer

Yes, it includes papers like 'Privacy Auditing with One (1) Training Run' (2023) and 'Auditing Black-Box Prediction Models for Data Minimization Compliance' (2021), which offer methods to assess data leakage and minimization, aiding regulatory compliance checks.

Question 6

What tools can I build based on these audit algorithms?

Accepted Answer

You can develop tools for fairness testing, model extraction, or adversarial attacks by studying papers like 'GeoDA: a geometric framework for black-box adversarial attacks' (2020). However, expect to write custom code, as few pre-built tools are provided.

Audit Algorithms

What is Audit Algorithms?

Overview

Use Cases

Best For

Related Projects

Found a gem we're missing?

Not Ideal For

Pros & Cons

Pros

Cons

Frequently Asked Questions