FairStress: A deeper evaluation of fairness in LLM decision-making

Hugging Face has released FairStress, a benchmark designed to assess demographic fairness in large language models (LLMs) beyond their final decisions. The tool evaluates how consistently models adhere to evidence when demographic identities are introduced, even in identical scenarios. It is integrated into Olmo-eval, the company’s open evaluation workbench, and is available for public use.

Models are increasingly deployed in high-stakes settings such as hiring, lending, and healthcare, where their outputs can significantly impact individuals’ opportunities. FairStress examines whether models maintain symmetric decision-making when demographic identities are altered, including scenarios where one candidate is clearly stronger. The benchmark also tests how models respond to persuasive arguments favoring different demographic groups.

FairStress includes 300 realistic two-candidate decision scenarios across 29 domains, with variations in how demographic identity is presented—ranging from no identity cues to explicit labels. The benchmark expands to over 13 million test cases by combining these variations with adversarial prefixes that challenge model decisions. A subset, FairStress-Core, reduces computational costs while retaining much of the benchmark’s diagnostic power.

Experiments with FairStress revealed that models often abandon correct decisions more readily when doing so favors minority-coded candidates, a pattern termed anti-stereotypical over-correction. For example, when a majority-coded candidate was clearly stronger, models were more likely to switch to a weaker minority-coded candidate under persuasive pressure. Differences in behavior were more pronounced with explicit demographic labels than with implied identities.