Psychological monitoring of mental disorders through conversation

The thesis investigates the perception of various aspects of virtual agents and chatbots by users with different sociodemographic features and in various communicative settings. The long-term aim is to achieve the use of chatbots as assistants for users with mental disorders such as depression and anxiety. The thesis consists of seven chapters. As can be seen from the previous paragraphs, the first chapter introduces the main topic, the characteristics of anxiety and depressive disorders, the two European projects to which the thesis has been connected and describes some aspects related to Human-Chatbot Interaction. The second chapter proposes a scheme for annotating dialogue acts, the MENHIR dialogue scheme, for the development of chatbot systems to assist people with depression and anxiety disorders in managing their conditions. The proposed dialogue acts have been selected by re-using existing dialogue act schemes and supplying them with specific dialogue acts individuated from the analysis of relevant conversational data. To realize an annotation, it has been necessary to elaborate in parallel, transcriptions and the corresponding audio files by using Praat software (Boersma and Weenink, 2018). The transcriptions and the audio files used for 12 this study referred to a previous experiment carried out within the Menhir project. This experiment led to the collection of interviews conducted by counselors and addressed to two different groups of participants. The first group (experimental group) consisted of participants with mental disorders (clients of the Action Mental Health partner) and the control group consisted of healthy participants. To annotate these files a dialogue act category list was drawn. In the phase of drafting of the list, the audios of counsellor-patient interactions were listened to deepen and better understand aspects of the conversation that may be important for the MENHIR’s context and find inspiration for introducing new dialogue act categories. Then, the dialogue acts under each category were ordered and reviewed to delete doubled, or unnecessary labels and produce a Final List. Therefore, this work led to the creation of two lists containing specific codes corresponding to the dialogue act name. A first codes list is specific to counselors while the other codes list is intended for patients/participants. The third chapter refers to a study devoted to assessed participants’ preferences toward two different levels of synthetic voices’ qualities (high and low quality) and synthetic voice gender differences. The investigation involved 186 participants divided in 4 groups equally balanced by gender (Group 1 composed by 45 adolescents; Group 2 by 47 young adults; Group 3 by 45 middle-aged and Group 4 by 49 seniors). The experiment was planned to evaluate the degree of acceptance to four selected synthetic Italian voices (two female and two male voices) characterized by two different levels of synthetic voice quality (low vs high). Participants had to evaluate other than voices also the preferred age and the tasks entrusted to the proposed voices. This second study entitled “How synthetic voices' qualities engage the perception of differently aged users (adolescents, young adults, middle-aged and seniors)” and presented in the third chapter of this thesis is published in proceedings of 12th IEEE International Conference on Cognitive Infocommunications, winning the Prize Best Paper Award. Unfortunately, due to fact that the scope of some papers were deemed to be outside of the scope of IEEE Xplore, the IEEE has 13 made the decision not to include this year's proceedings in IEEE Xplore, but to make it open-access. Therefore, the fourth chapter incorporates a paper, "Identifying synthetic voices' qualities for conversational agents" that adapts the paper from the second chapter, and also benefits from peer review. The fifth chapter concerns a work that saw the PhD student involved in data analysis and in the drafting of a paper, whose data were collected from three different countries involved in the mentioned European project called Empathic. As previously mentioned, the project aims to develop an Empathic Virtual Coach (VC) capable of promoting a healthy lifestyle and seniors’ independent aging. To reach this goal, it is fundamental to understand user preferences towards the implemented automatic system. Specifically, the countries involved in the investigation were: France, Norway and Spain for a total of 71 volunteers. Five virtual agents (VAs) named Natalie, Alice, Lena, Christian and Adam were purposely designed and user preferences were evaluated using the shortened version of the Virtual Agent Acceptance Questionnaire (VAAQ). This specific tool, will be suitably described in the aforementioned chapter. Regarding the VAAQ, a specific section (section VI) concerning voice preferences, represents a valuable contribution to the PhD student’ thesis project, providing additional information derived from other cultural contexts. The study is published in proceedings of 7th IEEE Cyber Science and Technology Congress (CyberSciTech 2022), Special session Social and Biometric data for Applications in human machine interactions: Models and algorithms (SOBIOAPPS). The sixth chapter is aimed at the description of a study carried out during the visiting period, spent at the Centre of Research of Information and Communication Technologies of the University of Granada (CITIC-UGR), Spain, under the supervision of Professor Zoraida Callejas Carrión. This research involved 43 Spanish young adults aged between 20 and 35 years old who were asked to express their preferences regarding 4 synthetic voices created purposely for the experiment, by using the shortened and digitized version of the Virtual Agent Voice Acceptance Questionnaire (VAVAQ). This specific tool, developed by 14 members of the EMPATHIC project, includes 7 different sections. In addition to the gender difference, the voices also differed in voice quality levels: there were in fact two high quality voices (Elena and Pablo) and two low quality voices (Maria and Juan). Due to the pandemic situation, participants were mainly contacted via email. Each participant was provided with a specific link. When the visiting period ended, the data belonging to other 4 Italian groups were collected in Italy. The Italian experimental protocol was the same described for the Spanish experiment. The Italian synthetic voices differ equally in gender and level of voice quality (Giulia and Antonio high quality voices vs Clara and Edoardo low quality voices). Therefore, a group consisting of 45 young adults aged between 20 and 35 years old was recruited for the study to compare these data with them collecting in Spain. Additionally, other three groups of participants (45 adolescents, 45 middle-aged and 45 seniors, respectively) were involved to collect data and validate the Italian shortened and digitalized version of the Virtual Agent Voice Acceptance Questionnaire (VAVAQ). In the last chapter, some considerations relating to the results obtained from each individual study will be discussed. Regarding the present thesis work, the PhD student has contributed in multiple roles: she took part in formulating and planning the overall research goals, in development and design of methodology, in performing the experiments and data collection. Subsequently, she paid particular attention to the data analysis and interpretation by applying statistical techniques. Therefore, she oversaw the drafting of the various articles that compose her PhD thesis, under the careful guidance and supervision of her tutor, Professor Esposito. Undoubtedly, valuable are the reviews and suggestions of the various co-authors cited in the works presented.