The Double Bind of Content Moderation in Visual GenAI Tools for Social Care
Research Output:
Conference Article in Proceeding or Book/Report chapter
Article in proceedings
Research
Peer-reviewOpen access
Abstract
People with intellectual and cognitive disabilities rely on visual supports to live more independently, and social care workers produce these from fixed image libraries with limited customization. GenAI may address this, yet how it performs when practitioners produce visual supports is not well understood. Through eight participatory design workshops (NrParticipants = 18) in Denmark and Sweden. We find familiar patterns from GenAI/HCI research, but with functional rather than aesthetic stakes, namely whether a client can learn a daily living skill. Our central finding is the double bind of content moderation: the same filters that protect children and people with intellectual and cognitive disabilities from harmful imagery also block the professional visual supports these populations depend on. This bind keeps GenAI from delivering value precisely where it could genuinely improve people’s lives. We mark it as a design-and-governance challenge requiring design research and policy to move together.
Publication Information
Output type
Research Output:
Conference Article in Proceeding or Book/Report chapter
Article in proceedings
Research
Peer-reviewOriginal language
EnglishArticle number
100Pages from-to (Number of pages)
Pages 1-13 (13 pages)Publication milestones
- Accepted/In press - 30/06/2026
- Published - 28/09/2026
Publication status
Published - 28/09/2026
Publisher
Association for Computing Machinery, United StatesBook series
- Book series name: Proceedings of NordiCHI
ISBN (Print)
979-8-4007-2373-5Host publication title
NordiCHI '26: Proceedings of the 14th Nordic Conference on Human-Computer InteractionFunding Details
This research was partially funded by R&D Östergötland and by the Novo Nordisk Foundation (Grant Number NNF20OC0066119). We would like to thank all workshop participants and their respective municipalities. Generative AI (Claude Opus 4.8, Anthropic) tools were used to assist with editing and revising author-written text. All content was reviewed and verified by the authors, who take full responsibility for it.
FundersFunding numbers
R&D Östergötland
Grant Number NNF20OC0066119
Novo Nordisk Foundation
Grant Number NNF20OC0066119

Access to documents
Related Event
Title
Nordic Human-Computer Interaction Conference 2026
Event type
ConferenceDegree of recognition
International eventDate
05/10/2026 - 07/10/2026Location
University of VaasaVaasaFinland
