Bias auditing

How it works

Where counterfactual probing swaps one attribute in a matched pair, a bias audit zooms out to the population: run many demographically labelled inputs through the system and measure outcome rates, error rates, and sentiment across groups, the way a fair-lending or hiring audit would. The output is a set of disparity metrics with confidence intervals — evidence that holds up where a single anecdote wouldn't, and the form regulators increasingly expect.

When to use it

Any evaluative or decision-support use touching people — screening, moderation, recommendation, sentiment — especially where there is legal exposure.

Limitations

Only audits the attributes and groups you chose to measure; intersectional and unmeasured biases slip through, and a clean audit on tested dimensions is not a clean bill of health.

Method yield

Findings

1

Versions spanned

5

Yield score

4

Severity-weighted across the published findings below. Why we measure this →

Findings it surfaces (1)

Documented failures this method catches — the evidence it works.

• Name-based demographic bias in outputsHigh

  Swapping only a name (signalling gender or ethnicity) changes evaluative outputs like screening or sentiment.

  How it found it: Auditing outcome rates across a population of name-swapped prompts quantifies the disparate treatment as a statistic.

  Bias

References & further reading

• Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification

  Joy Buolamwini, Timnit Gebru · PMLR v81 (FAccT 2018) · January 21, 2018

Cite this