Skip to search boxSkip to navigationSkip to main content

The Double Bind of Content Moderation in Visual GenAI Tools for Social Care

Research Output:
Conference Article in Proceeding or Book/Report chapter
Article in proceedings
Research
Peer-review

Open access

Abstract

People with intellectual and cognitive disabilities rely on visual supports to live more independently, and social care workers produce these from fixed image libraries with limited customization. GenAI may address this, yet how it performs when practitioners produce visual supports is not well understood. Through eight participatory design workshops (NrParticipants = 18) in Denmark and Sweden. We find familiar patterns from GenAI/HCI research, but with functional rather than aesthetic stakes, namely whether a client can learn a daily living skill. Our central finding is the double bind of content moderation: the same filters that protect children and people with intellectual and cognitive disabilities from harmful imagery also block the professional visual supports these populations depend on. This bind keeps GenAI from delivering value precisely where it could genuinely improve people’s lives. We mark it as a design-and-governance challenge requiring design research and policy to move together.

Publication Information

Output type

Research Output:
Conference Article in Proceeding or Book/Report chapter
Article in proceedings
Research
Peer-review

Original language

English

Article number

100

Pages from-to (Number of pages)

Pages 1-13 (13 pages)

Publication milestones

  • Accepted/In press - 30/06/2026
  • Published - 28/09/2026

Publication status

Published - 28/09/2026

Publisher

Association for Computing Machinery, United States

Book series

  • Book series name: Proceedings of NordiCHI
979-8-4007-2373-5

Host publication title

NordiCHI '26: Proceedings of the 14th Nordic Conference on Human-Computer Interaction

Funding Details

This research was partially funded by R&D Östergötland and by the Novo Nordisk Foundation (Grant Number NNF20OC0066119). We would like to thank all workshop participants and their respective municipalities. Generative AI (Claude Opus 4.8, Anthropic) tools were used to assist with editing and revising author-written text. All content was reviewed and verified by the authors, who take full responsibility for it.
FundersFunding numbers
R&D Östergötland
Grant Number NNF20OC0066119
Novo Nordisk Foundation
Grant Number NNF20OC0066119

Related Event

Title

Nordic Human-Computer Interaction Conference 2026

Event type

Conference

Degree of recognition

International event

Date

05/10/2026 - 07/10/2026

Location

University of VaasaVaasaFinland