Last updated on October 29, 2020
By Ariel Goldman
There are many ways to gather Human-Computer Interaction (HCI) data, each with their benefits [1]. One such method is semi-structured interviews. Semi-structured interviews are valuable data collection tools due to their versatility and due to the give-and-take that they allow between the interviewer and the participant [2]. In this blog post, I will provide an overview as to how to successfully conduct a semi-structured interview.
What is a Semi-Structured Interview?
A semi-structured interview is a middle ground between a structured and completely unstructured interview [1]. In a structured interview, all questions are pre-determined [1]. On the other end of the spectrum, a completely unstructured interview is a conversation with a focus and an intent [1]. A semi-structured interview, being in between these two extremes, relies on prepared questions but also allows the interviewer to add in questions throughout the interview in response to interesting concepts the participant brings up [1].
Recruiting Participants
Ethics
While recruiting participants for semi-structured interviews it is always important to make sure you are behaving ethically. All American universities that accept federal funding have to have an Institutional Review Board (IRB) committee [3]. Protecting human subjects (in this case your interview participants) is the IRB board’s main function, so assuring you receive IRB approval from your university crucial to running an ethical study [3]. One consideration in running an ethical study is making sure that no one is coerced into participating in your study [1], [3]. It is also important that you take into consideration how you will protect participant privacy in your study, especially when participants fear retribution if their answers were publicly associated with them [1].
Sampling
There are many ways to go about recruiting the participants for your semi-structured interview, and the method you choose will depend both on the data you need and the practical constraints of your study [1]. One option is purposive sampling, also known as judgment sampling [1]. In purposive sampling, the researchers select participants based on what set of participants they think will most efficiently answer the research question [1]. Another option is theoretical sampling, which can be done when data collection and data analysis are being done concurrently [1]. When employing theoretical sampling, researchers choose participants that will help them to test and expand on the theory that is emerging from the data they already have [1]. Snowball sampling is another method of recruiting participants, where participants who you have already recruited connect you to others who can participate in the study [1]. Snowball sampling is useful in helping researchers recruit from populations that are for some reason hard to reach, for instance, if the group is very small and specific [1]. However, snowball sampling risks recruiting less diverse participants, which could lead to findings based on one’s sample that don’t generalize well to the larger population being studied [1]. Based on the goals of and the constraining factors on a researcher’s semi-structured interviews, the researcher needs to select the recruiting method, or combination of methods, that best suit their needs [1].
Data Saturation
In research, you want to conduct enough interviews to achieve data saturation [4]. Data saturation has been achieved when additional interviews do not lead to new insights [4], [5]. For a homogeneous group, it takes about 12 interviews to achieve data saturation, resulting in about 15 interviews needing to be conducted, since you need to do interviews past the point of saturation to realize that you have reached the point of saturation [4]. Latham describes homogeneous as meaning “a particular ‘position’ or level (e.g., top-level executives) in the organization, a particular type of employee (e.g., customer service representatives), so on and so forth” [4]. If you have different types of participants, you will need to reach data saturation for each type [4].
Preparing for the Semi-Structured Interviews
While semi-structured interviews provide the flexibility to add questions and topics to the interview in real-time, the interviewer should prepare questions and topics ahead of time, based on their research questions [1]. Blandford describes how having “grounded” questions, such as questions that ask participants to think back to the most recent time they did a task you are interested in, helps interviewers to obtain detailed descriptions of how the participant completes a task [1]. The questions should prompt the participant to speak about specific incidences rather than make generalizations [1]. It is also important that the interview questions are not leading questions, and it is often useful to ask open rather than closed questions, as open questions promote more detail [1]. When trying to solicit feedback on ideas for the design of future tools, it is helpful to first provide a detailed scenario of the design being used to the participant, and then to ask the participants questions about this scenario [1].
Holtzblatt and Beyer, in explaining contextual design, state the principle that “People are experts at what they do – but are unable to articulate their own work practice” [6]. In planning for your interviews and creating interview questions, this is an important principle to keep in mind. Your questions should treat the participants as experts in their work practice and seek to coax out the details of their work practice that, while they do them in completing their tasks, they are not consciously aware of [6]. If possible, it is also helpful if the researcher can plan to complete the interview in the environment where the participant usually completes the task being studied and/or arrange for the participant to have access to the technology they use in the task being investigated during the interview [1], [6].
The prepared questions should be given an intended ordering to create an interview guide [2]. Kallio describes that the interview guide should be pilot-tested exposing “the interview guide to critique and scrutiny to see if changes [are] needed” [2]. Kallio proposes three different options for this pilot testing: internal testing, expert assessment, and field-testing [2]. Internal testing is when the researchers collaborate to assess the interview guide; this can include the researchers interviewing one another using the research guide [2]. Expert assessment is when the researchers show the guide to outside experts to get feedback on the guide and to gain new insights about the guide by discussing it [2]. Field-testing is when the interview guide is tested by using it with potential study participants to discover flaws and ways to improve the guide [2]. In addition to being adjusted during pilot-testing, the interview guide can also be modified in response to actual interviews the researchers conduct during the study [2].
Conducting the Interview
When first meeting the participant, the interviewer’s main goal is to start a conversation, make the participant feel comfortable, and to let the participant know that the interviewer values their expertise on the interview topic and is there to learn from them [1]. The interviewer should next introduce the research and its purpose. The interviewer should then obtain the participant’s consent to participate in the research and make it clear to the participant that they can withdraw from the study at any time [1]. At this point, if the participant consents to it, the interviewer should begin recording the conversation (either audio-only or with video, depending on what is appropriate and what the participant consents to) [1]. If the interview is not being recorded, then the interviewer should be prepared to take notes during the discussion [1]. Now the semi-structured interview can begin.
At the beginning of the interview, interviewers typically start by gathering background information from the participant [1]. This serves the dual purpose of providing the interviewer context for the rest of the interview and helping the participant feel more comfortable being interviewed [1]. For example, the interviewer could ask the participant about their job or their technology use at this stage of the interview [1]. Once the interview gets underway, the interviewer should use their prepared questions as a starting point for the conversation, but should also take advantage of this interview only being semi-structured by asking follow-up questions that arise from the comments of the participant [1]. In creating these follow-up questions, the advice for creating prepared questions in the section above generally still applies [1], [6].
During the interview, the interviewer should expect and be able to adjust to the participant bringing topics up earlier in the interview than the interviewer had planned for [1]. The interviewer should also be prepared that the participant may feel nervous or uncomfortable about discussing difficulties they have using technology [1]. To combat this issue, interviewers should assure participants that it is the technology being scrutinized in this study, not the participants [1]. The interviewer should also be sure to be an attentive, non-judgmental listener [1].
At the end of the interview, the interviewer should provide the participant with the opportunity to provide any additional thoughts they might have related to the topics of the interview and to add in any information they realized they forgot earlier in the interview [1]. The interviewer should also thank the participant [1]. Before they part ways, the interviewer should also tell the participant a bit about how their data will be utilized, as it commonly makes people happy to hear how their participation in the study might help improve the technology they use [1].
Works Cited
[1] A. Blandford, D. Furniss, and S. Makri, “Qualitative HCI Research: Going Behind the Scenes,” Synth. Lect. Hum.-Centered Inform., vol. 9, no. 1, pp. 1–115, Apr. 2016, doi: 10.2200/S00706ED1V01Y201602HCI034.
[2] H. Kallio, A.-M. Pietilä, M. Johnson, and M. Kangasniemi, “Systematic methodological review: developing a framework for a qualitative semi-structured interview guide,” J. Adv. Nurs., vol. 72, no. 12, pp. 2954–2965, Dec. 2016, doi: 10.1111/jan.13031.
[3] A. Bruckman, “Research Ethics and HCI,” in Ways of Knowing in HCI, J. S. Olson and W. A. Kellogg, Eds. New York, NY: Springer New York, 2014, pp. 449–468.
[4] John Latham, “Qualitative Sample Size-How Many Participants is Enough?,” John Latham. https://www.drjohnlatham.com/many-participants-enough/#:~:text=Enough%20is%20Enough&text=Consequently%2C%20the%20“sweet%20spot”,in%20the%20next%20few%20weeks (accessed Oct. 09, 2020).
[5] J. Matthes, C. S. Davis, and R. F. Potter, Eds., The International Encyclopedia of Communication Research Methods, 1st ed. Wiley, 2017.
[6] Karen Holtzblatt and Hugh R. Beyer, “Chapter 8: Contextual Design,” in The Encyclopedia of Human-Computer Interaction, 2nd ed., .
Featured Image Provided by Delesign Graphics
