From Prompts to Patches: A Vocabulary for Bridging Interpretability and Interaction

When designing interactions with AI systems, the technology is often approached as human-like agents that are engaged using humans’ natural language - text prompts. Drawing on post-phe-nomenological framings of AI systems for design, we argue that such artificial ‘others’ obstruct designerly access to the aesthetic and functional potential of their internal mechanics. Hidden Layer Interaction proposes an alternative to classical prompting techniques to make models’ interpretable mechanistic features accessible to users. In this paper, we frame the approach as a bridging concept for human-computer interaction and provide a vocabulary as a demonstration of how it lets us bridge theoretical knowledge of models’ mechanistic functioning with interaction design practice. With the proposed vocabulary, we aim to develop a language for making the inner workings of AI models affordant for designing interactive systems.