Computer-Human Interaction Research and Applications

This study examined the effect of mass customization of a socially assistive robot (SAR) on older-adult users’ attitudes and behaviors toward the robot. Mass customization, actively modifying aspects of a product by users before use, was proven to increase positive reactions towards products. Thirty-one older-adult participants were invited one at a time to explore new applications of personal robots for domestic use utilizing the Temi robot. We divided them into two groups that differed in their ability to manipulate the robot’s visual design using various add-ons before starting the one-on-one interaction with the robot. Results of the thematic analysis and questionnaires suggest that allowing mass customization can increase users’ enjoyment, help in forming human-robot relationships, and lead to proactive Interaction.

Social isolation and loneliness are risk factors of morbidity and mortality for the elderly, whose number will increase in the future. Smart devices can offer solutions to stimulate activities and social contact to fight these threats, on condition that the real needs, expectations and capacities of the target users are considered. Among smart devices, smart wooden furniture provides a sustainable way forward that can be easily integrated and accepted into the domestic environment. The article presents a module in a compact 3D-printed box with a smart tactile icon and visual and auditory feedback, built to be integrated in a wooden piece of furniture. A variety of simple scenarios can be programmed and the pattern of the icon can be changed according to the user’s needs and preferences. Various tests to validate the design have been performed and are presented. The electronic components are accessible for repair and the aim is low consumption, according to eco-design recommendations. The module showed to be a promising simple, robust and customizable tool to promote effective interaction with the elderly.

Human-Computer Interaction (HCI) is crucial in crafting effective, cost-efficient, and user-friendly e-learning systems. In the realm of e-learning, HCI focuses on creating interfaces and experiences that optimize learner engagement, interaction, and overall learning outcomes. Educators are actively pursuing initiatives to enhance student motivation, engagement, and academic achievement, particularly in Science, Technology, Engineering, and Mathematics (STEM) disciplines. However, learners often lack active engagement with electronic content when adapting and interpreting learning materials. This paper examines the Quality of Experience (QoE) in Science education content via a Technology Enhanced Multimedia Learning (TEML) Web portal featuring multisensory effects like rosemary scent, vibration, and airflow. Our ongoing research explores learners’ emotional states during Mulsemedia-based learning. In this study, we present initial insights into the developed Mulsemedia test bed using IoT devices and a learning Web portal, based on self-assessed QoE questionnaires administered to 60 participants divided into experimental and control groups. The results indicate that both groups had a positive experience in the Mulsemedia-based learning environment, demonstrating its suitability for STEM subjects.

Traveling is known to improve a person’s well-being and happiness, and tourism experiences should be offered to all types of tourists, including those with different disabilities. At the same time, technology is evolving and tourism applications are spreading. In this paper, we focus our attention on accessible mobile applications that can support a tourism experience. We first reviewed and classified a wide range of applications dedicated to users with visual, hearing, motor, cognitive impairments, and also others dedicated to the elderly. We have included this last category of users as they represent an interesting but at the same time specific target for the tourism industry. To understand how these dedicated apps could be better designed to improve accessibility we have then run a study collecting empirical data through questionnaires proposed to 210 users with diverse cognitive abilities, and to 50 elderly users, respectively. We have investigated and analyzed their approach to using tourist apps, also trying to understand the challenges they most commonly encounter when using them, and discussing possible improvements.

In the avionics industry, software test automation is a crucial component, as it plays a significant role in accelerating the development process. Since all automation systems are error-prone, the automation systems that affect flight safety should be supervised by humans as they must be reliable. In this study, an augmented reality based software verification system is designed to observe the visual verification of cockpit screens and detect possible automation errors. In the designed system, an observer can instantly see the test steps and automation results using an augmented reality headset as a hologram on the cockpit screen and can report the errors they observe in test automation. The performance of the proposed system was analyzed qualitatively and quantitatively. The experimental results demonstrated that the use of augmented reality in cockpit display verification systems accelerated the testing process, reduced users’ cognitive load, and improved the system’s usability.

In this paper, we present a Q-learning-based multi-agent system designed for Dynamic Difficulty Adjustment (DDA) in a 3D fighting game. Our primary goal is to enhance the player’s gaming experience through dynamic game difficulty adjustments based on their performance. We leverage the Unity game development platform and the ML-Agents framework to implement the Q-learning algorithm, training intelligent agents to adapt the game’s difficulty. Our findings underscore the efficacy of Q-learning and multi-agent systems in improving DDA for video games. In the conclusion section, we discuss potential implications and future directions for our research.

With the advent of improved Artificial Intelligence (AI) algorithms and the availability of large datasets, researchers worldwide are developing numerous AI-based applications to replicate human capabilities. One such application is automating the task of Speech Language Pathologists (SLPs) and building automated speech therapy tools for children with Speech Sound Disorder (SSD). However, this development of AI focused on imitating human capabilities brings concerns such as algorithmic discrimination or biased algorithms, job displacements, and privacy issues. To address these challenges, researchers advocate for Human-Centered AI (HCAI) and have proposed various frameworks for AI-based systems. Although the proposed frameworks were developed for generalized AI application, it is not clear about its relevance in specialized AI application such as speech therapy. This study aims to establish HCAI goals and a goal hierarchy specific to an HCAI-based Speech Therapy Tool (HCAI-STT) designed for children with SSD. Through an Affinity Mapping exercise, we identify seven top-level goals and sub-goals, which include fairness, responsibility and accountability, human-centered empowerment, trustworthiness, privacy, unbiased funding, and security. Our findings highlight the importance of considering not only the technical capabilities of AI systems, but also their ethical and social implications. By prioritizing these goals, we can help ensure that AI-based speech therapy tools are developed and deployed in a responsible and ethical manner that aligns with the needs and values of their users. Our findings have broader implications for the development and deployment of AI systems across domains, and future research can build on our findings by exploring how the goal hierarchy we developed can be operationalized in practice.

The increasing ageing population in any country necessitates high-quality healthcare services. Smartphone-based healthcare applications can play a pivotal role in meeting this requirement. Previous studies indicated that older adults exhibit lower acceptance of smartphone-based healthcare applications. In contrast, older adults popularly use smartphone applications for social interaction. WhatsApp is one such popular application used by older adults for social interaction. This study aims to enhance the ease of use of a dedicated healthcare application among older adults by incorporating WhatsApp's user interface design attributes. The present study comprises three phases. In the first phase, we assessed the usability of WhatsApp and observed its good usability characteristics for older adults. In the second phase, we identified the design attributes of WhatsApp's user interface by employing the affinity mapping method. In the third phase, we applied the identified design attributes to develop a prototype of a healthcare application and tested it with older adults. We found that the developed prototype of the healthcare application is easy to use among older adults. This study recommends developing a WhatsApp-inspired dedicated healthcare application for older adults. The study demonstrates how older adults perceive a new healthcare application as user-friendly, inspired by a familiar user interface. This research will aid designers and developers in creating dedicated healthcare applications tailored to the needs of older adults. Consequently, it will enhance the adoption of digital healthcare applications among the ageing population.

Electric micromobility is a promising part of the transition towards more sustainable and ecologic transportation systems – especially as a complement to public transportation on the last mile. However, so far adoption of such services is still not ideal, especially in rural areas. By means of a quantitative study with 137 users, our work-in-progress paper investigates user acceptance in urban and rural areas. Based on a structural equation model, our research shows that rural municipalities and providers should especially consider aspects that address the performance expectation and the perceived collective environmental impact of potential users, as these have been found to be especially relevant for the intention to use. Our study thus contributes to the theoretical understanding of e-micromobility, which has so far been mostly investigated in urban areas.

In order to improve online learning outcomes, a Learning Dashboard (LD) for online students is being developed as a plugin for the learning management system Moodle to support self-regulation. The project itself focuses on the factors that lead to success and failure in online learning. Using a user-centered design approach, the LD will provide students with feedback and functional elements through different cards. 24 online students completed a three-part term paper in which they examined the elements of two wireframes of the LD in relation to Wertheimer's Gestalt Laws and in terms of factual and interaction problems. We also received 11 card designs from them as a voluntary bonus assignment. Assignments 1 & 2 had to be completed successfully in order to be admitted to the exam. The study was designed to encourage student participation and improve accessibility by taking into account their expertise. The results showed that clearer overviews, clarification of how content elements fit together, more compact solutions, and intuitive controls improved clarity and usability.

College freshmen often face difficulties adjusting to the new academic and social environment of university life. It is critical to help them adapt to academic and personal life, while also improving their sense of belonging and engagement with the university. In this paper, we focus on the context of an international joint venture university, Xi’an Jiaotong-Liverpool University (XJTLU), and present a participatory design approach to identify potential solutions collaboratively. We conducted three participatory design workshops with freshmen in undergraduate and postgraduate studies, where we discovered specific challenges, developed serious game content and design alternatives, and delivered a board game that supports academic and social integration at XJTLU. To evaluate the effectiveness of the board game, we collected both quantitative and qualitative data. The quantitative analysis revealed that the board game is effective in improving freshmen’s knowledge acquisition of academic affairs, increasing their familiarity with the environment and resources, and enhancing their ability to access information and resources. The board game also received high scores in system usability and user experience. The qualitative analysis indicated that the board game was engaging, interesting, and well-received by students. They found the board game helpful in their academic and social integration and expressed a desire to play it again in the future. Our participatory design approach and the resulting board game provide a promising avenue for universities to support freshmen’s transition to university life.

Designing interactive technology to support safety-critical systems poses multiple challenges with respect to security, access to operators and the proprietary nature of the data. We conducted a two-year participatory design project with French power grid operators to both understand their specific needs and to collaborate on the design of a novel collaborative tool called StoryLines . Our primary objective was to capture detailed, in-context data about operators’ work practices as part of a larger project designed to provide bi-directional assistance between an intelligent agent and human operator. We targeted handovers between shifts to take advantage of the operators’ existing practice of articulating the current status of the grid and expected future events. We use information that would otherwise be lost to gather valuable information about the operator’s decision rationale and decision-making patterns. This paper describes how we combined a bottom-up participatory design approach with a top-down generative theory approach to design StoryLines, an interactive timeline that helps operators collect information from diverse tools, record reminders and share relevant information with the next shift’s operator. We conclude with a discussion of the challenges of working with users in safety-critical environments and directions for future research.

One of the characteristics of big data is its internal complexity and also variety manifested in many types of datasets that are to be managed, searched, or analyzed. In their natural forms, some of the data entities are unstructured, such as texts or multimedia objects, while some are structured but too complex. In this paper, we have investigated how visualizations of various complex datasets perform in the role of universal data representations for both human users and deep learning models. In a user study, we have evaluated several visualizations of complex relational data, where some proved their superior performance with respect to the precision and speed of classification by human users. Moreover, the same visualizations also led to effective classification performance when used with deep learning models.

This paper presents a novel web platform designed to investigate the relationship between sounds, colors, and emotions, with the overarching goal of enhancing the sensory experience of impaired individuals in visual art museums. Taking advantage of the principles of sensory substitution, the project aims to bridge the gap between auditory and visual perception, allowing individuals with visual or cognitive impairments to engage with visual art through alternative sensory modalities. The platform’s architecture is centered around the delivery of short sound stimuli to participants, who then provide feedback on the associations they perceive between these auditory cues and both colors and emotional dimensions. The collected data will be analyzed to discern patterns and correlations, shedding light on how auditory stimuli can be used to evoke visual and emotional responses. This paper outlines the technical and methodological aspects of the web platform, including its design, development, and implementation. It discusses the selection of sound stimuli and the integration of user-friendly interfaces to ensure a seamless experience for participants. Preliminary results from volunteer tests are briefly presented, highlighting intriguing findings regarding the associations between sounds, colors, and emotions.

In recent years, there has been growing interest in understanding what makes a review valuable, as such reviews are vital in guiding consumer and business decision-making. The purpose of this study was to determine the role that the user experience of mobile applications plays in fostering review helpfulness as well as stimulating managerial responses to reviews of these applications. This study proposes a measure of UX richness for online reviews and finds that both positive and negative UX-rich reviews contribute to enhancing the helpfulness of reviews as well as the likelihood that they will receive a response from the application provider. The study further demonstrates the moderating role of UX richness in the prominent effects of review length and review rating on both the helpfulness and managerial response to mobile app reviews. The study culminates with a discussion of the implications of these findings.

The current paper aims to analyse the complex array of practices entailed by teams and esports professionals by looking at one of the most peculiar phenomena of the esports field: gaming houses, i.e., “co-operative living arrangement[s] where several players of video games, usually professional esports players, live in the same residence” [1]. Representing one of the first attempts to assess the role of gaming houses as emerging esports spaces based on new forms of playbour and production of and by users, the paper comprises an innovative adaptation of PRISMA protocol for literature and scoping reviews to shed light on how the technological, material, and social elements are enacted through gaming houses’ activities, which mirror the ones entailed by digital platforms. In fact, through the three moves of encoding, aggregating and computing users’ interactions [2], gaming houses (re)produce virtual and analogical goods, translating consumer practices and profoundly influencing the broader esports ecosystem. Finally, by framing themselves as ideal hives for pro players, i.e., a prototypical breeding ground for esports professionals, these structures push for new paradigms of work-life balance and users’ production, thus leading to a further reflection on the nature of play and working practices in our contemporary network society [3].

AI win prediction is widely used in the live streaming of Esports games, with the assumption that it is capable of significantly enhancing the viewing experience and providing valuable information to spectators. However, there is very little empirical research to demonstrate the actual attitudes and feelings of spectators towards AI win prediction. This paper describes an ongoing study from the perspective of communication effectiveness that aims to bridge this gap and explore some possible influencing factors, which could provide a scientific basis for better presenting AI prediction information in future Esports live streaming, thus further improving the viewing experience and engagement of spectators. This study has not yet officially begun on a large scale, so this paper reports primary results from in-depth interviews with 12 hardcore fans of League of Legends, as a pilot study for the formal survey experiment. The perceived usefulness, the balance between credibility, accuracy, and dramatic effects, and the anthropomorphic image are mainly discussed.

Social interactions and the sense of presence are important for the spectatorship experience in live-streams. In large audiences, communication gets harder and viewers participate less. This paper explores the possibility of representing an audience using animated avatars to increase the sense of presence and potentially move some traffic from the chat window to the avatars. We discuss the motivations for and the challenges in creating an audience avatar interface.

Esports professionals have to cope with a lot of stress and really need to be in sync with their teammates in order to perform at their highest capability. As part of an ongoing project, we are working toward a battery of minigames aimed at helping esports professionals train their teamwork skills and improve their mental health. Proposed ideas include modules focused on meditation, teaching coping mechanisms for difficult social scenarios via visual novel, and synchronized breathing exercises. Each of these has positive points and negative points. With these ideas, we hope to further develop several engaging minigames which can later be user tested.

Game streaming platforms like Twitch could benefit from more user control and transparency in recommendations. In this paper, we highlight the importance of allowing users to customise their streaming experience through three design goals: Social Interaction, Captivation, and Knowledge Acquisition, the latter addressing both skill improvement and serendipity. We discuss the preliminary results of our on-going iterative and user-centred design process aimed at improving the exploration experience for game spectators. More specifically we report on the results of co-design research to explore the parameters necessary for game spectators’ enhanced control over their game stream discovery experience.