How Users Choose LLMs

AI has become increasingly more commonplace and, as such, has been scrutinized for its environmental impacts. Efficiency, ease of use, UI/UX design, etc., all contribute to user preference. Presenting data for each preference metric to users may change their perception of the model and, subsequently, their choice of model. We aim to discover whether participants note presented ecological data and use that data when selecting an LLM for everyday tasks.

This project focuses on a survey and interview model. First, participants fill out a survey regarding their values and opinions towards AI. They then complete three tasks using a different LLM model for each task and provide feedback on each model’s performance and usability. Environmental impact is one of the metrics provided by the interface that participants use, but, using a deception study design, it was not revealed to participants that environmental impact was the focus of this study. The participants review and rank the models’ performance, and afterwards, they are told about the true aim of the study. Finally, the participants are interviewed about their experience and opinions after knowing the study’s purpose.

Researchers combine quantitative analysis and reflexive thematic analysis as part of a broader investigation into how people make choices when they do not feel social pressure to make “environmentally conscious” choices, especially surrounding AI.

Team

Faculty

Undergraduate Researchers