On (assessing) the fairness of risk score models
Publikation: Bidrag til bog/antologi/rapport › Konferencebidrag i proceedings › Forskning › fagfællebedømt
Dokumenter
- Preprint
Indsendt manuskript, 927 KB, PDF-dokument
Recent work on algorithmic fairness has largely focused on the fairness of discrete decisions, or classifications. While such decisions are often based on risk score models, the fairness of the risk models themselves has received considerably less attention. Risk models are of interest for a number of reasons, including the fact that they communicate uncertainty about the potential outcomes to users, thus representing a way to enable meaningful human oversight. Here, we address fairness desiderata for risk score models. We identify the provision of similar epistemic value to different groups as a key desideratum for risk score fairness, and we show how even fair risk scores can lead to unfair risk-based rankings. Further, we address how to assess the fairness of risk score models quantitatively, including a discussion of metric choices and meaningful statistical comparisons between groups. In this context, we also introduce a novel calibration error metric that is less sample size-biased than previously proposed metrics, enabling meaningful comparisons between groups of different sizes. We illustrate our methodology - which is widely applicable in many other settings - in two case studies, one in recidivism risk prediction, and one in risk of major depressive disorder (MDD) prediction.
| Originalsprog | Engelsk |
|---|---|
| Titel | Proceedings of the 6th ACM Conference on Fairness, Accountability, and Transparency, FAccT 2023 |
| Antal sider | 13 |
| Forlag | Association for Computing Machinery, Inc. |
| Publikationsdato | 2023 |
| Sider | 817-829 |
| ISBN (Elektronisk) | 9781450372527 |
| DOI | |
| Status | Udgivet - 2023 |
| Begivenhed | 6th ACM Conference on Fairness, Accountability, and Transparency, FAccT 2023 - Chicago, USA Varighed: 12 jun. 2023 → 15 jun. 2023 |
Konference
| Konference | 6th ACM Conference on Fairness, Accountability, and Transparency, FAccT 2023 |
|---|---|
| Land | USA |
| By | Chicago |
| Periode | 12/06/2023 → 15/06/2023 |
| Navn | ACM International Conference Proceeding Series |
|---|
Bibliografisk note
Funding Information:
The authors would like to thank Vibe Frøkjær, Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, and Merete Osler, Department of Public Health, University of Copenhagen, for help with the MDD study conception. The authors would also like to thank Emily Beaman, Department of Biomedical Sciences, University of Copenhagen, for assistance with the data analysis in the MDD study. Work on this project was partially funded by the Independent Research Fund Denmark (DFF, grant number 9131-00097B), Denmark’s Pioneer Centre for AI (DNRF grant number P1), and the Novo Nordisk Foundation through the Center for Basic Machine Learning Research in Life Science (MLLS, grant number NNF20OC0062606). The funding agencies had no influence on the writing, submission, or publication of this manuscript.
Funding Information:
The authors would like to thank Vibe Frøkjær, Neurobiology Research Unit, Copenhagen University Hospital, Rigshospitalet, and Merete Osler, Department of Public Health, University of Copenhagen, for help with the MDD study conception. The authors would also like to thank Emily Beaman, Department of Biomedical Sciences, University of Copenhagen, for assistance with the data analysis in the MDD study. Work on this project was partially funded by the Independent Research Fund Denmark (DFF, grant number 9131- 00097B), Denmark's Pioneer Centre for AI (DNRF grant number P1), and the Novo Nordisk Foundation through the Center for Basic Machine Learning Research in Life Science (MLLS, grant number NNF20OC0062606). The funding agencies had no influence on the writing, submission, or publication of this manuscript.
Publisher Copyright:
© 2023 ACM.
Antal downloads er baseret på statistik fra Google Scholar og www.ku.dk
ID: 359976468