Pectogram is an AI-augmented communication tool that extends the Picture Exchange Communication System (PECS) to support more expressive, personalized, and adaptive interaction for users of augmentative and alternative communication (AAC) technologies.
Description
Communication is a deeply personal and context-dependent act, yet many AAC systems remain rigid, relying on pre-defined symbol sets that can limit users’ expressive range. Pectogram explores how generative AI can enhance pictogram-based communication by helping users create new symbols and adapt existing ones to reflect their own experiences, preferences, and identities.
The system allows users to assemble pictogram “sentences” to convey ideas visually, then uses language and image generation models to propose variations, synonyms, or visual enhancements that align with the user’s intended meaning. This design promotes bidirectional communication, enabling conversation partners to interpret and respond more fluidly while maintaining the visual structure that AAC users depend on. Our research combines human-centered design, AI ethics, and accessibility principles to examine how augmentative systems can be both creative and respectful of user autonomy. Early findings suggest that Pectogram supports more personalized expression and fosters reflection on communication preferences among both AAC users and their conversation partners.
This work contributes to ongoing discussions in HCI and accessibility about how AI can be responsibly integrated into assistive technologies—enhancing user agency rather than replacing it. Pectogram demonstrates how creativity, accessibility, and intelligence can coexist in a single communication system.
Demonstration Video
Presentations
E. Connolly, S. Lera, L. Reed, S. Morrison-Smith. 2025. Pectogram: Enhancing PECS with Generative AI to Support Unfamiliar Communication Contexts’. Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’25), ACM (2025).
Soren Lera, Leah Reed, Sarah Morrison-Smith. 2025. Pectogram: Exploring AI-Augmented Visual Communication in AAC Systems. Presented at Hamilton College Family Weekend Research Poster Session and the 13th NY6 Undergraduate Research Conference, St. Lawrence University.
Team
Faculty
- Dr. Sarah Morrison-Smith
Undergraduate Researchers
- Soren Lera
- Leah Reed
Previous Undergraduate Researchers
- Oluwayanmife (Uriah) Adeniran
- Charles (Charlie) Beard
- Ethan Connolly