publications

Data

1. 

   Korean Emoticons: Understanding How Subtle Emotional Differences Are Evoked Online

   Kang, Chowon, Hong, Jong-ok, Chang, Wooje, Suk, Hyeon-jeong, and Hong, Hwajung

   CSCW ’22, Nov 2022

   Abs

   Online conversations through text have limitations in expressing emotions that could cause miscommunications across cultures. In this work, we study the Korean emotional expressions in text focusing on how people perceive emotional intentions through the use of emotion-expressing Korean characters. We define them as Korean Emoticons (‘ㅋ’, ‘ㅎ’, ‘ㅠ’), onomatopoeic characters often used to express emotions for text-based communication. We examine the participants’ understanding and usage of Korean Emoticons by conducting an online survey asking to evaluate emotional contents of given sentences and interviews to explain personal experiences. We found that the different numbers of Korean emoticons used evoke different emotions, and that negative emoticons amplify positive emotions in positive contexts and positive emoticons alleviate negative emotions in negative contexts, while emoticons in neutral contexts have varying impacts depending on the context. We further discuss design implications on how text suggestion tools can support users taking emotional intentions into account.

2. GeniAuti: Toward Data-Driven Interventions to Challenging Behaviors of Autistic Children through Caregivers’ Tracking

   Jo, Eunkyung, park, Seora, Bang, Hyeonseok, Hong, Youngeun, Kim, Yeni, Choi, Jungwon, Kim, Bung Nyun, Epstein, Daniel A., and Hong, Hwajung

   CSCW ’22, Apr 2022

   Abs HTML

   Challenging behaviors significantly impact learning and socialization of autistic children and can stress and burden their caregivers. Documentation of challenging behaviors is fundamental for identifying what environmental factors influence them, such as how others respond to a child’s such behaviors. Caregiver-tracked data on their child’s challenging behaviors can help clinical experts make informed recommendations about how to manage such behaviors. To support caregivers in recording their children’s challenging behaviors, we developed GeniAuti, a mobile-based data-collection tool built upon a clinical data collection form to document challenging behaviors and other clinically relevant contextual information such as place, duration, intensity, and what triggers such behaviors. Through an open-ended deployment with 19 parent-child pairs and three expert collaborators, caregivers found GeniAuti valuable for (1) becoming more attentive and reflective to behavioral contexts, including their own response strategies, (2) discovering positive aspects of their children’s behaviors, and (3) promoting collaboration with clinical experts around the caregiver-tracked data to develop tailored intervention strategies for their children. However, participant experiences surface challenges of logging behaviors in social circumstances, conflicting views between caregivers and clinical experts around the structured recording process, and emotional struggles resulting from recording and reflecting on intensely negative experiences. Considering the complex nature of caregiver-based health tracking and caregiver–clinician collaboration, we suggest design opportunities for facilitating negotiations between caregivers and clinicians and accounting for caregivers’ emotional needs.

3. Sticky Goals: Understanding Goal Commitments for Behavioral Changes in the Wild

   Lee, Hyunsoo, Kim, Auk, Hong, Hwajung, and Lee, Uichin

   CHI ’21, May 2021

   Abs HTML PDF

   A commitment device, an attempt to bind oneself for a successful goal achievement, has been used as an effective strategy to promote behavior change. However, little is known about how commitment devices are used in the wild, and what aspects of commitment devices are related to goal achievements. In this paper, we explore a large-scale dataset from stickK, an online behavior change support system that provides both financial and social commitments. We characterize the patterns of behavior change goals (e.g., topics and commitment setting) and then perform a series of multilevel regression analyses on goal achievements. Our results reveal that successful goal achievements are largely dependent on the configuration of financial and social commitment devices, and a mixed commitment setting is considered beneficial. We discuss how our findings could inform the design of effective commitment devices, and how large-scale data can be leveraged to support data-driven goal elicitation and customization.

4. ADIO: An Interactive Artifact Physically Representing the Intangible Digital Audiobook Listening Experience in Everyday Living Spaces

   Lee, Kyung-Ryong, Kim, Beom, Kim, Junyoung, Hong, Hwajung, and Park, Young-Woo

   CHI ’21, May 2021

   Abs HTML PDF

   Although audiobooks are increasingly being used, people tend to perceive audiobook experiences as ’not real reading’ due to its intangibility and ephemerality. In this paper, we developed ADIO, a device augmenting audiobook experience through representing personal listening state in the form of an interactive physical bookshelf. ADIO displays a user’s listening progress through a pendant’s changing length and the user’s digital audiobook archive titles. The result of our four-week in-field study with six participants revealed that ADIO provided proof of the user’s listening-to, which brought a sense of reading and gave a trigger for recalling the listened-to audiobook content. Additionally, audiobooks’ improved visibility reminded participants to listen to them, and ADIO’s physical interaction allowed participants to form personal patterns for listening to audiobooks. Our findings proposed new methods for augmenting the audiobook listening experience at three stages and further implications for designing physical curation on users’ digital archives.

5. Understanding Parenting Stress through Co-designed Self-Trackers

   Jo, Eunkyung, Toombs, Austin L., Gray, Colin M., and Hong, Hwajung

   CHI ’20, Apr 2020

   Abs HTML PDF

   New parents often experience significant stress as they take on new roles and responsibilities. Stress management and mental wellbeing are two areas in which personal informatics (PI) research has gained attention, and there is an opportunity to investigate how parenting stress can be mitigated through PI practices. In this paper, we present the results of a co-designed technology probe study through which we deployed individualized self-trackers with new parents. We investigate the stress management topics new parents are interested in tracking and how — and with what goals—they engage in self-directed PI practices. Our findings indicate that PI practices can potentially enable parents to: re-discover positive aspects of their everyday lives; identify better-suited stress management strategies; and facilitate spousal communication about shared responsibilities. We discuss how self-tracking experiences for the mental wellness of parents can be better designed.

6. Toward Future-Centric Personal Informatics: Expecting Stressful Events and Preparing Personalized Interventions in Stress Management

   Lee, Kwangyoung, Cho, Hyewon, Toshnazarov, Kobiljon, Narziev, Nematjon, Rhim, So Young, Han, Kyungsik, Noh, YoungTae, and Hong, Hwajung

   CHI ’20, Apr 2020

   Abs HTML PDF

   Stress is caused by a variety of events in our daily lives. By anticipating stressful situations, we can prepare and better cope with stressors when they actually occur. However, many past-centric personal informatics (PI) tools focus on capturing events that already happened and analyzing the data. In this work, we examine how anticipation — a future-centric self-tracking practice — could be used to manage daily stress levels. To address this, we built MindForecaster, a calendar- mediated stress anticipation application that allows users to expect stressful events in advance, generates activities to mitigate stress, and evaluates actual stress levels compared to previously estimated stress levels. In a 30-day deployment with 47 users, the users who explicitly planned and executed coping interventions reported reduced stress more than those who only expected stressful events. We suggest design implications for stress management by incorporating the properties of anticipation into current PI models.

7. MindNavigator: Exploring the stress and self-interventions for mental wellness

   Lee, Kwangyoung, and Hong, Hwajung

   CHI ’18, Apr 2018

   Abs HTML PDF

   Mental wellness is a desirable health outcome for students. However, current personal informatics systems do not adequately support students in creating concrete mental health-related goals and turning them into actionable plans. In this paper, we introduce MindNavigator - a workshop in which groups of college students were invited to generate behavioral change goals to manage daily life stress and practice personalized interventions for two weeks. We describe the manner in which participants identified both stressors and pleasures to create actionable, engaging, and open-ended behavioral plans that aided in stress relief. We found that the social nature of the workshop helped participants understand themselves and execute self-intervention in new ways. Through this practice, we build on prior studies to propose an analytical framework of personal informatics for mental wellness.

8. Designing for Self-Tracking of Emotion and Experience with Tangible Modality

   Lee, Kwangyoung, and Hong, Hwajung

   CHI ’17, Jun 2017

   Abs HTML PDF

   Self-tracking technologies have been developed to understand the self. Emotions are critical to understanding one’s daily life; however, tracking the emotion is challenging due to the implicit form of data. In this paper, we introduce MindTracker, an approach for tracking emotion through a tangible interaction with plasticine clay. We explored the benefits and challenges of MindTracker via a two-week data collection study with 16 college students as well as via interviews with three clinical mental health experts. MindTracker is designed for users to craft a form that represents emotion using clay and to describe the experience that evokes the emotion using a diary. We found that the tangible modality of MindTracker motivated the participants to express various aspects of emotions. In addition, MindTracker’s data collection and reflection process could have therapeutic properties, such as expressive therapy, self-soothing, and emotional self-regulation. We conclude this paper by discussing the design features of emotion-tracking tools and opportunities to use MindTracker to promote mental health.

AI / Algorithm

1. Exploring the Use of a Voice-based Conversational Agent to Empower Adolescents with Autism Spectrum Disorder

   Cha, Inha, Kim, Sung-In, Hong, Hwajung, Yoo, Heejeong, and Lim, Youn-kyung

   CHI ’21, May 2021

   Abs HTML PDF

   Voice-based Conversational Agents (VCA) have served as personal assistants that support individuals with special needs. Adolescents with Autism Spectrum Disorder (ASD) may also benefit from VCAs to deal with their everyday needs and challenges, ranging from self-care to social communications. In this study, we explored how VCAs could encourage adolescents with ASD in navigating various aspects of their daily lives through the two-week use of VCAs and a series of participatory design workshops. Our findings demonstrated that VCAs could be an engaging, empowering, emancipating tool that supports adolescents with ASD to address their needs, personalities, and expectations, such as promoting self-care skills, regulating negative emotions, and practicing conversational skills. We propose implications of using off-the-shelf technologies as a personal assistant to ASD users in Assistive Technology design. We suggest design implications for promoting positive opportunities while mitigating the remaining challenges of VCAs for adolescents with ASD.

2. How Do You Feel Online: Exploiting Smartphone Sensors to Detect Transitory Emotions during Social Media Use

   Ruensuk, Mintra, Cheon, Eunyong, Hong, Hwajung, and Oakley, Ian

   IMWUT ’20, Dec 2020

   Abs HTML PDF

   Emotions are an intrinsic part of the social media user experience that can evoke negative behaviors such as cyberbullying and trolling. Detecting the emotions of social media users may enable responding to and mitigating these problems. Prior work suggests this may be achievable on smartphones: emotions can be detected via built-in sensors during prolonged input tasks. We extend these ideas to a social media context featuring sparse input interleaved with more passive browsing and media consumption activities. To achieve this, we present two studies. In the first, we elicit participant’s emotions using images and videos and capture sensor data from a mobile device, including data from a novel passive sensor: its built-in eye-tracker. Using this data, we construct machine learning models that predict self-reported binary affect, achieving 93.20% peak accuracy. A follow-up study extends these results to a more ecologically valid scenario in which participants browse their social media feeds. The study yields high accuracies for both self-reported binary valence (94.16%) and arousal (92.28%). We present a discussion of the sensors, features and study design choices that contribute to this high performance and that future designers and researchers can use to create effective and accurate smartphone-based affect detection systems.

Inclusion / Health

1. 

   Prediction for Retrospection: Integrating Algorithmic Stress Prediction into Personal Informatics Systems for College Students’ Mental Health

   Kim, Taewan, Kim, Haesoo, Lee, Ha Yeon, Goh, Hwarang, Abdigapporov, Shakhboz, Jeong, Mingon, Cho, Hyunsung, Han, Kyungsik, Noh, Youngtae, Lee, Sung-Ju, and Hong, Hwajung

   CHI ’22, Apr 2022

   Abs HTML

   Reflecting on stress-related data is critical in addressing one’s mental health. Personal Informatics (PI) systems augmented by algorithms and sensors have become popular ways to help users collect and reflect on data about stress. While prediction algorithms in the PI systems are mainly for diagnostic purposes, few studies examine how the explainability of algorithmic prediction can support user-driven self-insight. To this end, we developed MindScope, an algorithm-assisted stress management system that determines user stress levels and explains how the stress level was computed based on the user’s everyday activities captured by a smartphone. In a 25-day field study conducted with 36 college students, the prediction and explanation supported self-reflection, a process to re-establish preconceptions about stress by identifying stress patterns and recalling past stress levels and patterns that led to coping planning. We discuss the implications of exploiting prediction algorithms that facilitate user-driven retrospection in PI systems.

2. 

   “It’s not wrong, but I’m quite disappointed”: Toward an Inclusive Algorithmic Experience for Content Creators with Disabilities

   Choi, Dasom, Lee, Uichin, and Hong, Hwajung

   CHI ’22 Apr 2022

   Abs HTML PDF

   YouTube is a space where people with disabilities can reach a wider online audience to present what it is like to have disabilities. Thus, it is imperative to understand how content creators with disabilities strategically interact with algorithms to draw viewers around the world. However, considering that the algorithm carries the risk of making less inclusive decisions for users with disabilities, whether the current algorithmic experiences (AXs) on video platforms is inclusive for creators with disabilities is an open question. To address that, we conducted semi-structured interviews with eight YouTubers with disabilities. We found that they aimed to inform the public of diverse representations of disabilities, which led them to work with algorithms by strategically portraying disability identities. However, they were disappointed that the way the algorithms work did not sufficiently support their goals. Based on findings, we suggest implications for designing inclusive AXs that could embrace creators’ subtle needs.

3. 

   Sad or just jealous? Using Experience Sampling to Understand and Detect Negative Affective Experiences on Instagram

   Ruensuk, Mintra, Kim, Taewan, Hong, Hwajung, and Oakley, Ian

   CHI ’22 Apr 2022

   Abs HTML

   Social Network Services (SNSs) evoke diverse affective experiences. While most are positive, many authors have documented both the negative emotions that can result from browsing SNS and their impact: Facebook depression is a common term for the more severe results. However, while the importance of the emotions experienced on SNSs is clear, methods to catalog them, and systems to detect them, are less well developed. Accordingly, this paper reports on two studies using a novel contextually triggered Experience Sampling Method to log surveys immediately after using Instagram, a popular image-based SNS, thus minimizing recall biases. The first study improves our understanding of the emotions experienced while using SNSs. It suggests that common negative experiences relate to appearance comparison and envy. The second study captures smartphone sensor data during Instagram sessions to detect these two emotions, ultimately achieving peak accuracies of 95.78% (binary appearance comparison) and 93.95% (binary envy).

4. Understanding University Students’ Experiences, Perceptions, and Attitudes Toward Peers Displaying Mental Health–Related Problems on Social Networking Sites: Online Survey and Interview Study

   Kim, Taewan, and Hong, Hwajung

   JMIR Mental Health ’21, Apr 2021

   HTML PDF
5. Comparing Perspectives Around Human and Technology Support for Contact Tracing

   Lu, Xi, Reynolds, Tera L., Jo, Eunkyung, Hong, Hwajung, Page, Xinru, Chen, Yunan, and Epstein, Daniel A.

   CHI ’21, May 2021

   Abs HTML PDF

   Various contact tracing approaches have been applied to help contain the spread of COVID-19, with technology-based tracing and human tracing among the most widely adopted. However, governments and communities worldwide vary in their adoption of digital contact tracing, with many instead choosing the human approach. We investigate how people perceive the respective benefits and risks of human and digital contact tracing through a mixed-methods survey with 291 respondents from the United States. Participants perceived digital contact tracing as more beneficial for protecting privacy, providing convenience, and ensuring data accuracy, and felt that human contact tracing could help provide security, emotional reassurance, advice, and accessibility. We explore the role of self-tracking technologies in public health crisis situations, highlighting how designs must adapt to promote societal benefit rather than just self-understanding. We discuss how future digital contact tracing can better balance the benefits of human tracers and technology amidst the complex contact tracing process and context.

6. Enriched Social Translucence in Medical Crowdfunding

   Kim, Jennifer G., Kong, Ha-Kyung, Hong, Hwajung, and Karahalios, Karrie

   DIS ’20, Jul 2020

   Abs HTML PDF

   Social translucence theory argues that online collaboration systems should make contributors’ activities visible to better achieve a common goal. Currently in medical crowdfunding sites, various non-monetary contributions integral to the success of a campaign, such as campaign promotions and offline support, are less visible than monetary contributions. Our work investigates ways to enrich social translucence in medical crowdfunding by aggregating and visualizing non-monetary contributions that reside outside of the current crowdfunding space. Three different styles of interactive visualizations were built and evaluated with medical crowdfunding beneficiaries and contributors. Our results reveal the perceived benefits and challenges of making the previously invisible non-monetary contributions visible using various design features in the visualizations. We discuss our findings based on the social translucence framework–visibility, awareness, and accountability–and suggest design guidelines for crowdfunding platform designers.

7. MAMAS: Supporting Parent–Child Mealtime Interactions Using Automated Tracking and Speech Recognition

   Jo, Eunkyung, Bang, Hyeonseok, Ryu, Myeonghan, Sung, Eun Jee, Leem, Sungmook, and Hwajung Hong,

   CSCW ’20, May 2020

   Abs HTML PDF

   Many parents of young children find it challenging to deal with their children’s eating problems, and parent–child mealtime interaction is fundamental in forming children’s healthy eating habits. In this paper, we present the results of a three-week study through which we deployed a mealtime assistant application, MAMAS, for monitoring parent–child mealtime conversation and food intake with 15 parent–child pairs. Our findings indicate that the use of MAMAS helped 1) increase children’s autonomy during mealtime, 2) enhance parents’ self-awareness of their words and behaviors, 3) promote the parent–child relationship, and 4) positively influence the mealtime experiences of the entire family. The study also revealed some challenges in eating behavior interventions due to the complex dynamics of childhood eating problems. Based on the findings, we discuss how a mealtime assistant application can be better designed for parents and children with challenging eating behaviors.

8. In Helping a Vulnerable Bot, You Help Yourself: Designing a Social Bot as a Care-Receiver to Promote Mental Health and Reduce Stigma

   Kim, Taewan, Ruensuk, Mintra, and Hong, Hwajung

   CHI ’20, Apr 2020

   Abs HTML PDF

   Helping others can have a positive effect on both the giver and the receiver. However, supporting someone with depression can be complicated and overwhelming. To address this, we proposed a Facebook-based social bot displaying depressive symptoms and disclosing vulnerable experiences that allows users to practice providing reactions online. We investigated how 55 college students interacted with the social bot for three weeks and how these support-giving experiences affected their mental health and stigma. By responding to the bot, the participants reframed their own negative experiences, reported reduced feelings of danger regarding an individual with depression and increased willingness to help the person, and presented favorable attitudes toward seeking treatment for depression. We discuss design opportunities for accessible social bots that could help users to keep practicing peer support interventions without fear of negative consequences.

9. Toward becoming a better self: Understanding self-tracking experiences of adolescents with autism spectrum disorder using custom trackers

   Kim, Sung-In, Jo, Eunkyung, Ryu, Myeonghan, Cha, Inha, Kim, Young-Ho, Yoo, Heejung, and Hong, Hwajung

   PervasiveHealth ’19, May 2019

   Abs HTML PDF

   Adolescence is a challenging period, particularly for individuals with autism spectrum disorder (ASD), due to having little knowledge about themselves. Self-management is a strategy to enhance self-understanding through continuous self-monitoring, which can support adaptive transitions to adulthood. Meanwhile, the advent of digital self-tracking tools enables users to collect and reflect on data about themselves. In this work, we investigated how adolescents with ASD kept track of their everyday lives to better understand themselves using a custom self-tracking platform, OmniTrack, over a two-week period. Our findings indicate that personalized self-tracking experiences enable adolescents to monitor the detailed contexts, causes, and consequences of problematic situations; regulate negative emotion and anxiety while interacting with the tracker; and communicate through data with their caregivers, teachers, and therapists. Building on these findings, we suggest the design of a new form of flexible, scaffolded self-tracking technique that can inform both researchers for designing pervasive health technologies for adolescents with ASD and clinicians for guiding adolescents with ASD toward better self-management using such technologies.