articles+ search results

Computer Science - Human-Computer Interaction

Crowdsourcing can collect many diverse ideas by prompting ideators individually, but this can generate redundant ideas. Prior methods reduce redundancy by presenting peers' ideas or peer-proposed prompts, but these require much human coordination. We introduce Directed Diversity, an automatic prompt selection approach that leverages language model embedding distances to maximize diversity. Ideators can be directed towards diverse prompts and away from prior ideas, thus improving their collective creativity. Since there are diverse metrics of diversity, we present a Diversity Prompting Evaluation Framework consolidating metrics from several research disciplines to analyze along the ideation chain - prompt selection, prompt creativity, prompt-ideation mediation, and ideation creativity. Using this framework, we evaluated Directed Diversity in a series of a simulation study and four user studies for the use case of crowdsourcing motivational messages to encourage physical activity. We show that automated diverse prompting can variously improve collective creativity across many nuanced metrics of diversity.
Comment: CHI 2021

Computer Science - Human-Computer Interaction and Computer Science - Artificial Intelligence

The advent of artificial intelligence (AI) and machine learning (ML) bring human-AI interaction to the forefront of HCI research. This paper argues that games are an ideal domain for studying and experimenting with how humans interact with AI. Through a systematic survey of neural network games (n = 38), we identified the dominant interaction metaphors and AI interaction patterns in these games. In addition, we applied existing human-AI interaction guidelines to further shed light on player-AI interaction in the context of AI-infused systems. Our core finding is that AI as play can expand current notions of human-AI interaction, which are predominantly productivity-based. In particular, our work suggests that game and UX designers should consider flow to structure the learning curve of human-AI interaction, incorporate discovery-based learning to play around with the AI and observe the consequences, and offer users an invitation to play to explore new forms of human-AI interaction.

Computer Science - Human-Computer Interaction

Crowdsourced design feedback systems are emerging resources for getting large amounts of feedback in a short period of time. Traditionally, the feedback comes in the form of a declarative statement, which often contains positive or negative sentiment. Prior research has shown that overly negative or positive sentiment can strongly influence the perceived usefulness and acceptance of feedback and, subsequently, lead to ineffective design revisions. To enhance the effectiveness of crowdsourced design feedback, we investigate a new approach for mitigating the effects of negative or positive feedback by combining open-ended and thought-provoking questions with declarative feedback statements. We conducted two user studies to assess the effects of question-based feedback on the sentiment and quality of design revisions in the context of graphic design. We found that crowdsourced question-based feedback contains more neutral sentiment than statement-based feedback. Moreover, we provide evidence that presenting feedback as questions followed by statements leads to better design revisions than question- or statement-based feedback alone.

Computer Science - Human-Computer Interaction and Computer Science - Artificial Intelligence

Despite the intricacies involved in designing a computer as a teampartner, we can observe patterns in team behavior which allow us to describe at a general level how AI systems are to collaborate with humans. Whereas most work on human-machine teaming has focused on physical agents (e.g. robotic systems), our aim is to study how humans can collaborate with information agents. We propose some appropriate team design patterns, and test them using our Collaborative Intelligence Analysis (CIA) tool.
Comment: 4 pages

Computer Science - Human-Computer Interaction

Uncertainty is widely acknowledged as an engaging gameplay element but rarely used in exergames. In this research, we explore the role of uncertainty in exergames and introduce three uncertain elements (false-attacks, misses, and critical hits) to an exergame. We conducted a study under two conditions (uncertain and certain), with two display types (virtual reality and large display) and across young and middle-aged adults to measure their effect on game performance, experience, and exertion. Results show that (1) our designed uncertain elements are instrumental in increasing exertion levels; (2) when playing a motion-based first-person perspective exergame, virtual reality can improve performance, while maintaining the same motion sickness level as a large display; and (3) exergames for middle-aged adults should be designed with age-related declines in mind, similar to designing for elderly adults. We also framed two design guidelines for exergames that have similar features to the game used in this research.
Comment: Accepted to ACM 2021 CHI Conference on Human Factors in Computing Systems (CHI 2021)

Computer Science - Machine Learning, Computer Science - Human-Computer Interaction, 68U35, and H.5.0

This paper discusses modern Auto Machine Learning (AutoML) tools from the perspective of a person with little prior experience in Machine Learning (ML). There are many AutoML tools both ready-to-use and under development, which are created to simplify and democratize usage of ML technologies in everyday life. Our position is that ML should be easy to use and available to a greater number of people. Prior research has identified the need for intuitive AutoML tools. This work seeks to understand how well AutoML tools have achieved that goal in practice. We evaluate three AutoML Tools to evaluate the end-user experience and system performance. We evaluate the tools by having them create models from a competition dataset on banking data. We report on their performance and the details of our experience. This process provides a unique understanding of the state of the art of AutoML tools. Finally, we use these experiences to inform a discussion on how future AutoML tools can improve the user experience for neophytes of Machine Learning.
Comment: 10 pages, 3 tables, 3 figures. First author is a high school senior

Computer Science - Human-Computer Interaction

Wearable Cognitive Assistance (WCA) amplifies human cognition in real time through a wearable device and low-latency wireless access to edge computing infrastructure. It is inspired by, and broadens, the metaphor of GPS navigation tools that provide real-time step-by-step guidance, with prompt error detection and correction. WCA applications are likely to be transformative in education, health care, industrial troubleshooting, manufacturing, and many other areas. Today, WCA application development is difficult and slow, requiring skills in areas such as machine learning and computer vision that are not widespread among software developers. This paper describes Ajalon, an authoring toolchain for WCA applications that reduces the skill and effort needed at each step of the development pipeline. Our evaluation shows that Ajalon significantly reduces the effort needed to create new WCA applications.

Computer Science - Human-Computer Interaction, Computer Science - Artificial Intelligence, K.3.1, and I.2.1

Personalized gamification explores knowledge about the users to tailor gamification designs to improve one-size-fits-all gamification. The tailoring process should simultaneously consider user and contextual characteristics (e.g., activity to be done and geographic location), which leads to several occasions to tailor. Consequently, tools for automating gamification personalization are needed. The problems that emerge are that which of those characteristics are relevant and how to do such tailoring are open questions, and that the required automating tools are lacking. We tackled these problems in two steps. First, we conducted an exploratory study, collecting participants' opinions on the game elements they consider the most useful for different learning activity types (LAT) via survey. Then, we modeled opinions through conditional decision trees to address the aforementioned tailoring process. Second, as a product from the first step, we implemented a recommender system that suggests personalized gamification designs (which game elements to use), addressing the problem of automating gamification personalization. Our findings i) present empirical evidence that LAT, geographic locations, and other user characteristics affect users' preferences, ii) enable defining gamification designs tailored to user and contextual features simultaneously, and iii) provide technological aid for those interested in designing personalized gamification. The main implications are that demographics, game-related characteristics, geographic location, and LAT to be done, as well as the interaction between different kinds of information (user and contextual characteristics), should be considered in defining gamification designs and that personalizing gamification designs can be improved with aid from our recommender system.
Comment: Submitted to IEEE Transactions on Learning Technologies. 14 pages, 2 figures, 8 tables

Computer Science - Human-Computer Interaction

The landscape of digital games is segregated by player ability. For example, sighted players have a multitude of highly visual games at their disposal, while blind players may choose from a variety of audio games. Attempts at improving cross-ability access to any of those are often limited in the experience they provide, or disregard multiplayer experiences. We explore ability-based asymmetric roles as a design approach to create engaging and challenging mixed-ability play. Our team designed and developed two collaborative testbed games exploring asymmetric interdependent roles. In a remote study with 13 mixed-visual-ability pairs we assessed how roles affected perceptions of engagement, competence, and autonomy, using a mixed-methods approach. The games provided an engaging and challenging experience, in which differences in visual ability were not limiting. Our results underline how experiences unequal by design can give rise to an equitable joint experience.
Comment: 21 pages, 1 figure. Manuscript submitted to ACM Conference on Human Factors in Computing Systems (CHI 21)

Computer Science - Multimedia and Computer Science - Human-Computer Interaction

Connected vehicles, whether equipped with advanced driver-assistance systems or fully autonomous, are currently constrained to visual information in their lines-of-sight. A cooperative perception system among vehicles increases their situational awareness by extending their perception ranges. Existing solutions imply significant network and computation load, as well as high flow of not-always-relevant data received by vehicles. To address such issues, and thus account for the inherently diverse informativeness of the data, we present Augmented Informative Cooperative Perception (AICP) as the first fast-filtering system which optimizes the informativeness of shared data at vehicles. AICP displays the filtered data to the drivers in augmented reality head-up display. To this end, an informativeness maximization problem is presented for vehicles to select a subset of data to display to their drivers. Specifically, we propose (i) a dedicated system design with custom data structure and light-weight routing protocol for convenient data encapsulation, fast interpretation and transmission, and (ii) a comprehensive problem formulation and efficient fitness-based sorting algorithm to select the most valuable data to display at the application layer. We implement a proof-of-concept prototype of AICP with a bandwidth-hungry, latency-constrained real-life augmented reality application. The prototype realizes the informative-optimized cooperative perception with only 12.6 milliseconds additional latency. Next, we test the networking performance of AICP at scale and show that AICP effectively filter out less relevant packets and decreases the channel busy time.
Comment: Submitted to ICDCS'21

Computer Science - Machine Learning, Computer Science - Artificial Intelligence, Computer Science - Human-Computer Interaction, and Computer Science - Multiagent Systems

In order for agents trained by deep reinforcement learning to work alongside humans in realistic settings, we will need to ensure that the agents are \emph{robust}. Since the real world is very diverse, and human behavior often changes in response to agent deployment, the agent will likely encounter novel situations that have never been seen during training. This results in an evaluation challenge: if we cannot rely on the average training or validation reward as a metric, then how can we effectively evaluate robustness? We take inspiration from the practice of \emph{unit testing} in software engineering. Specifically, we suggest that when designing AI agents that collaborate with humans, designers should search for potential edge cases in \emph{possible partner behavior} and \emph{possible states encountered}, and write tests which check that the behavior of the agent in these edge cases is reasonable. We apply this methodology to build a suite of unit tests for the Overcooked-AI environment, and use this test suite to evaluate three proposals for improving robustness. We find that the test suite provides significant insight into the effects of these proposals that were generally not revealed by looking solely at the average validation reward.

Computer Science - Robotics and Computer Science - Human-Computer Interaction

Over the past years, extensive research has been dedicated to developing robust platforms and data-driven dialogue models to support long-term human-robot interactions. However, little is known about how people's perception of robots and engagement with them develop over time and how these can be accurately assessed through implicit and continuous measurement techniques. In this paper, we investigate this by involving participants in three interaction sessions with multiple days of zero exposure in between. Each session consists of a joint task with a robot as well as two short social chats with it before and after the task. We measure participants' gaze patterns with a wearable eye-tracker and gauge their perception of the robot and engagement with it and the joint task using questionnaires. Results disclose that aversion of gaze in a social chat is an indicator of a robot's uncanniness and that the more people gaze at the robot in a joint task, the worse they perform. In contrast with most HRI literature, our results show that gaze towards an object of shared attention, rather than gaze towards a robotic partner, is the most meaningful predictor of engagement in a joint task. Furthermore, the analyses of long-term gaze patterns disclose that people's mutual gaze in a social chat develops congruently with their perceptions of the robot over time. These are key findings for the HRI community as they entail that gaze behavior can be used as an implicit measure of people's perception of robots in a social chat and of their engagement and task performance in a joint task.
Comment: 20 pages, 7 figures, This work has been submitted to Frontiers in Robotics and AI (Human-Robot Interaction)

Computer Science - Computation and Language, Computer Science - Artificial Intelligence, and Computer Science - Human-Computer Interaction

The impact of user satisfaction in policy learning task-oriented dialogue systems has long been a subject of research interest. Most current models for estimating the user satisfaction either (i) treat out-of-context short-texts, such as product reviews, or (ii) rely on turn features instead of on distributed semantic representations. In this work we adopt deep neural networks that use distributed semantic representation learning for estimating the user satisfaction in conversations. We evaluate the impact of modelling context length in these networks. Moreover, we show that the proposed hierarchical network outperforms state-of-the-art quality estimators. Furthermore, we show that applying these networks to infer the reward function in a Partial Observable Markov Decision Process (POMDP) yields to a great improvement in the task success rate.
Comment: Accepted at the Human in the Loop Dialogue Systems, 34st Conference on Neural Information Processing Systems (NeurIPS 2020). Paper updated with minor changes

Computer Science - Human-Computer Interaction and Computer Science - Computer Vision and Pattern Recognition

The popularity of racket sports (e.g., tennis and table tennis) leads to high demands for data analysis, such as notational analysis, on player performance. While sports videos offer many benefits for such analysis, retrieving accurate information from sports videos could be challenging. In this paper, we propose EventAnchor, a data analysis framework to facilitate interactive annotation of racket sports video with the support of computer vision algorithms. Our approach uses machine learning models in computer vision to help users acquire essential events from videos (e.g., serve, the ball bouncing on the court) and offers users a set of interactive tools for data annotation. An evaluation study on a table tennis annotation system built on this framework shows significant improvement of user performances in simple annotation tasks on objects of interest and complex annotation tasks requiring domain knowledge.

Computer Science - Computation and Language, Computer Science - Artificial Intelligence, and Computer Science - Human-Computer Interaction

We present EventPlus, a temporal event understanding pipeline that integrates various state-of-the-art event understanding components including event trigger and type detection, event argument detection, event duration and temporal relation extraction. Event information, especially event temporal knowledge, is a type of common sense knowledge that helps people understand how stories evolve and provides predictive hints for future events. EventPlus as the first comprehensive temporal event understanding pipeline provides a convenient tool for users to quickly obtain annotations about events and their temporal information for any user-provided document. Furthermore, we show EventPlus can be easily adapted to other domains (e.g., biomedical domain). We make EventPlus publicly available to facilitate event-related information extraction and downstream applications.

Computer Science - Computers and Society and Computer Science - Human-Computer Interaction

With the rise of COVID-19 cases globally, many countries released digital tools to mitigate the effects of the pandemic. In Germany the Robert Koch Institute (RKI) published the Corona-Data-Donation-App, a virtual citizen science (VCS) project, to establish an early warning system for the prediction of potential COVID-19 hotspots using data from wearable devices. While work on motivation for VCS projects in HCI often presents egoistic motives as prevailing, there is little research on such motives in crises situations. In this paper, we explore the socio-psychological processes and motivations to share personal data during a pandemic. Our findings indicate that collective motives dominated among app reviews (n=464) and in in-depth interviews (n=10). We contribute implications for future VCS tools in times of crises that highlight the importance of communication, transparency and responsibility.

Computer Science - Human-Computer Interaction

Many accessibility features available on mobile platforms require applications (apps) to provide complete and accurate metadata describing user interface (UI) components. Unfortunately, many apps do not provide sufficient metadata for accessibility features to work as expected. In this paper, we explore inferring accessibility metadata for mobile apps from their pixels, as the visual interfaces often best reflect an app's full functionality. We trained a robust, fast, memory-efficient, on-device model to detect UI elements using a dataset of 77,637 screens (from 4,068 iPhone apps) that we collected and annotated. To further improve UI detections and add semantic information, we introduced heuristics (e.g., UI grouping and ordering) and additional models (e.g., recognize UI content, state, interactivity). We built Screen Recognition to generate accessibility metadata to augment iOS VoiceOver. In a study with 9 screen reader users, we validated that our approach improves the accessibility of existing mobile apps, enabling even previously inaccessible apps to be used.

Computer Science - Software Engineering and Computer Science - Human-Computer Interaction

Quality of website design is one of the influential factors of website success. How the design helps the users using effectively and efficiently website and satisfied at the end of the use. However, it is a common tendency that websites are designed based on the developer's perspectives and lack considering user importance. Thus, the degree of website usability tends to be low according to user perceptions. This study purposed to understand the user experiences using an institutional repository (IR) website in a public university in Indonesia. The research was performed based on usability testing framework as the usability testing method. About 12 participants were purposely involved concerning their key informant characteristics. Following three empirical data collection techniques (i.e., query technique, formal experiment, and thinking aloud), both descriptive analysis using usability scale matric and content analysis using qualitative data analysis (QDA) Miner Lite software were used in the data analysis stage. Lastly, several visual design recommendations were then proposed at the end of the study. In terms of a case study, besides the practical recommendations which may contextually useful for the next website development; the clarity of the research design may also help scholars how to combine more than one usability testing technique within a multi-technique study design.
Comment: 7 pages, The 2nd International Science and Mathematics Conference (SMIC 2020) : Transforming Research and Education of Science and Mathematics in the Digital Age

Computer Science - Human-Computer Interaction

In today's fast-paced world, stress has become a growing health concern. While more automatic stress tracking technologies have recently become available on wearable or mobile devices, there is still a limited understanding of how they are actually used in everyday life. This paper presents an empirical study of automatic stress-tracking technologies in use in China, based on semi-structured interviews with 17 users. The study highlights three challenges of stress-tracking data engagement that prevent effective technology usage: the lack of immediate awareness, the lack of pre-required knowledge, and the lack of corresponding communal support. Drawing on the stress-tracking practices uncovered in the study, we bring these issues to the fore, and unpack assumptions embedded in related works on self-tracking and how data engagement is approached. We end by calling for a reconsideration of data engagement as part of self-tracking practices with technologies rather than simply looking at the user interface.
Comment: 13 pages, 2 figures, 1 table, Accepted at ACM 2021 CHI Conference on Human Factors in Computing Systems (CHI 2021)

Computer Science - Human-Computer Interaction

Industrial inspection automation in aerospace presents numerous challenges due to the dynamic, information-rich and regulated aspects of the domain. To diagnose the condition of an aircraft component, expert inspectors rely on a significant amount of procedural and tacit knowledge (know-how). As systems capabilities do not match high level human cognitive functions, the role of humans in future automated work systems will remain important. A Cyber-Physical-Social System (CPSS) is a suitable solution that envisions humans and agents in a joint activity to enhance cognitive/computational capabilities and produce better outcomes. This paper investigates how a work-centred approach can support and guide the engineering process of a CPSS with an industrial use case. We present a robust methodology that combines fieldwork inquiries and model-based engineering to elicit and formalize rich mental models into exploitable design patterns. Our results exhibit how inspectors process and apply knowledge to diagnose the component`s condition, how they deal with the institution`s rules and operational constraints (norms, safety policies, standard operating procedures). We suggest how these patterns can be incorporated in software modules or can conceptualize Human-Agent Teaming requirements. We argue that this framework can corroborate the right fit between a system`s technical and ecological validity (system fit with operating context) that enhances data reliability, productivity-related factors and system acceptance by end-users.