Alignment with demographic subgroups can look good for single survey questions, yet miss the correlation structure of cultural values.
Tristan Williams, Sebastian Padó, Alan Akbik and I propose a 2-level eval framework and apply it to demographically aligned LLMs.
arxiv.org/abs/2601.15755
Paper accepted to #EACL2026 main conference 🎉
@taniseceron.bsky.social, Sebastian Padó and I test multilingual LLMs before and after English-only fine-tuning and find strong cross-lingual political opinion transfer across five Western languages.
www.arxiv.org/abs/2508.05553
While the responses for different cues are highly correlated, we find stronger biases for more explicit but less externally valid cues. We advise future work to focus on externally valid cues such as human written conversation histories or to base their findings on a mixture of cues (2/2)
Different sociodemographic cues for gender (male, female, and non-binary personas) result in different LLM answers to a medical advice request.
Even with identical sociodemographic info, how LLMs are given it changes downstream bias results. Our new preprint (w/ @veraneplenbroek.bsky.social, Jan Batzner & Sebastian Padó) tests cues with varying external validity across 10 personas, 4 tasks & 7 LLMs: arxiv.org/abs/2601.18572
