UX and Voice Assistant (VA) Politeness 🗣️

A mixed-method study to understand how VA politeness affects UX

OVERVIEW

Given how VAs have been adopted into the everyday lives of many, I conducted a literature review to understand the research that has been conducted into UX. A vast amount of research has explored how the anthropomorphism of VAs affects UX. However, little research has explored the politeness of VAs. Thus, I decided to conducted research to understand how VA politeness affects UX.

ROLE

I was the sole UX researcher working on this project from scratch by planning the project, conducting analysis of the research problem, designing and implementing the methodology and evaluating the research conducted.

I summarised the project within a full research paper and presented the findings to my supervisor, a faculty member and a fellow student for evaluation.

My responsibilities included conducting a literature review, forming hypotheses, designing the methodology, designing and administering surveys, conducting in-depth interviews, conducting quantitative and qualitative analysis to understand the collected data, interpreting the findings, forming recommendations for design and evaluating the project.

DURATION

3 months

RESEARCH AIM

The purpose of this present study was to investigate and compare participants’ perceptions of trust, humanlike-ness, behavioural traits and engagement when listening to a user interact with both a polite and neutral VA.

METHODOLOGY

A mixed-method research design was used to collect both quantitative and qualitative data. Thus, the study involved two parts: (1) a quantitative part to compare the differences in social manners and engagement, perceived humanlike traits, perceived behavioural traits, and trust, and (2) a qualitative part to aid interpretation of the quantitative outcomes.

Within part 1, a within-subjects design was adopted. The independent variable is social manners, with two levels (polite and neutral). All participants took part in each condition. Order-effects were counterbalanced by changing the order of conditions for each participant. The dependent variables were humanlike traits, behavioural traits, engagement and trust.

Two audio recordings were presented to participants within this study. In condition one, the audio recording involved a ‘polite’ VA in a user-VA interaction. The length of this audio recording was 1 minute 55 seconds. In condition two, the audio recording involved a more ‘neutral’ VA in a user-VA interaction. The length of this audio recording was 1 minute 32 seconds.

Within part 2, in-depth interviews using a semi-structured format were conducted. Open- ended questions were posed to participants to understand their views on politeness of VAs. The average length of the interview across the twenty participants was 12 minutes and 47 seconds.

DATA ANALYSIS

To analyse the survey data, statistical analyses were performed using SPSS Statistics V28.

Due to the ordinal property of the data, a non-parametric test 'Wilcoxon'signed rank test' was chosen to conduct the analysis. This was used to investigate the significant statistical differences between the user engagement, humanlike, behavioural and trust scores between the polite and neutral conditions.

Spearman’s rho was used to investigate whether there is a significant correlation between median behavioural scores and median trust, engagement and humanlike scores in both polite and neutral conditions.

To analyse the interview data, thematic analysis was used to form themes.

KEY FINDINGS

Significant difference in user engagement between the polite and neutral conditions, suggesting that users find polite VAs more engaging.

Users voiced that polite VA sounds less robotic and thus more engaging

No significant difference in trust between the polite and neutral VA.

Some users voiced that politeness enhances their trust in VAs
One user felt as though a politer VA makes them feel as though the VA is listening to their commands, making them more confident and trusting in their response
A few participants expressed a sense of skepticism towards polite communication from a VA
Many participants raised that their trust does not lie with the VA itself but the developer of the software

RECOMMENDATIONS

Recommendations can be formed to aid designers in the development of VAs.

The findings of this present study suggest that users perceive politer VAs as more engaging than less polite, or neutral VAs. However, many users reported feeling indifferent to the politeness expressed by the VA and find efficiency and timely responses more engaging. Thus, designers have the challenge of striking a balance between the benefits of politeness and efficiency within user-VA interactions.

Designers should first ensure clear communication from the VA, while maintaining politeness. In this way, commands from the VA are concise and efficient. This ensures that the user is receiving fast responses. Using strategies outlined within the politeness theory (Brown & Levinson, 1978) designers can include polite language in the VA responses. This can include indirect language to soften communication and gratitude such as ‘please’, ‘thank you’ and ‘sorry’. The VA should also be designed to offer alternative options, emphasising that the user is able to make their own decisions. They can also offer suggestions to assist the user in achieving goals, such as recommendations for booking times. These factors ensure that the VA is proactive, which users mentioned to increase engagement. Designers should note that the VA should not be designed to be ‘overly’ polite, to the extent that the system sounds unnatural and artificial.

LIMITATIONS AND FUTURE RESEARCH

This present study provided an insight into whether users’ perceptions differ when listening to a user interact with a polite VA and neutral VA. It also allowed for a richness of data surrounding perceptions of VA politeness, VA-trust, and engagement. However, further research can be conducted to explore this in more depth to understand how VA politeness affects user experience.

Within this study, participants were presented with the audio recordings of a user interacting with both the polite and neutral VA. Although the users were asked to answer survey questions relating to these said interactions, they were unable to assess the usability of the VA and it lacked realism. Future research can prototype a polite VA to interact directly with the participants, which will increase realism and allow for scenarios to be adapted according to the outputs of the users. Prototypes of varying levels of politeness can be designed to explore which VAs are more engaging and trustworthy, aiding in the future design of VAs.

Although politeness is universal, the patterns of behaviours, politeness principles and perceptions may have cross-cultural differences which were not accounted for within this present study. Thus, future research can be conducted to investigate how user experiences with VA politeness differ between cultures to help inform design choices suited towards all cultures.

Data was collected from 20 participants within this present study. While this allowed for a richness of data surrounding perceptions of VA politeness within in-depth interviews, a larger sample size would benefit the quantitative aspect of the research. Future research could be conducted using a larger sample to ensure that the findings are generalisable.