Counterfactual Audit of Ethnic-Origin Bias and Architectural Mitigation in Generative AI Assistants for Debt Collection in Savings and Credit Cooperatives

Debt-collection automation requires assessment of differential treatment between equivalent profiles. This pre-deployment counterfactual audit evaluates tone severity and empathy in generative AI outputs for Ecuador’s Popular and Solidarity Financial Sector. Five models generated 2400 responses under unmitigated and fairness-instruction conditions (A/B), plus 400 surname-masked control tasks (C), across 40 synthetic scenarios and three surname pairs. None of the 12 primary contrasts remained statistically significant after Holm correction. This does not establish absence of bias, and the 40-scenario sample may limit detection of small effects. Results for instruction-based mitigation apply only to the tested Protocol V4 template; alternative formulations were not evaluated. Masking prevented surname exposure in all 400 control tasks without establishing general fairness or identical generated text. The primary human analysis used seven complete raters and 630 paired-item evaluations; three additional raters contributed 29 partial evaluations excluded from that analysis. Inter-rater agreement was near zero and judge–human alignment was weak, limiting interpretation of automated scores as measures of perceived severity or empathy. One complete rater was also a coauthor. The study provides scoped audit results and an exposure-control check rather than evidence of deployment effectiveness or general model fairness.