2021 Year-in-Review: Health and Happiness Design Lab at Drexel University

This is the second Year-in-Review of the Health and Happiness Design Lab. Many of our lab members developed non-trivial health issues and managed them, but we severed through and made another year of many accomplishments. Many new members joined for summer research, and we received some funding to get exciting projects started. Also, we learned many things around using emerging technologies, such as conversational agents, to support older adults and mental health. We started an exciting journey to rethinking existing ethical frameworks to understand research ethics around research involving human subjects in the context of evolving technological environments involving AI and algorithms.

More specifically, we built knowledge around how we might design emerging technologies in everyday lives, such as conversational agents, to help older adults and families get help on their caretaking roles and examine ways to support mental health. Given that the studies in conversational agents have historically been investigated with virtual agents in the past, we conducted several systematic reviews to build our knowledge on the precedents of conversational agents (Lu’s research on citation metrics on conversational agents to understand this better is forthcoming). Here we investigated what theoretical framework might have been identified about the factors that influence people’s trust toward these agents and how we might define different design components of agents’ design characteristics. The proliferation of third-party systems in many of these agent-building platforms has led us to investigate how developers and users can control users’ data being shared during the interaction with conversational agents. We received funding from the National Science Foundation in collaboration with American Psychological Association to understand whether current ethical frameworks (e.g., the Belmont Principle) apply to the emerging role of technologies in research involving human subjects. We have invited experts in the field to work with us to define working questions and generate potential starting points for the solutions to these difficult questions.

We also made some strides in advancing our collaboration with the College of Nursing and Health Professions in understanding the role of technologies in supporting the informal caregivers of ADRD. We examined nationally representative survey data to understand how the technology adoption levels vary across different segments of the older adult population. This effort would help us understand how we can provide the appropriate technology environment to support older adults and their caregivers. Our continued collaboration with Pace University and San Diego State University on the Telehealth Intervention Program for Seniors (TIPS) is starting to accelerate, with several new papers being published and some papers going under review and being prepared for submission.

Many exciting projects are brewing in the background, such as working with the Philadelphia community leaders to develop a mobile solution to help underprivileged youth’s mental health. Also, we submitted a grant proposal with folks from the engineering and autism institute to support adult autistic people to overcome social anxiety and support their transition into the workforce environment.

We start below with the new knowledge we contributed to various areas, including conversational agents, research ethics, older adults, and social technology design.

Conversational agents

MJ Rheu, JY Shin, W Peng, J. Huh-Yoo (2021). Trust-building Factors and Implications for Conversational Agent Design. International Journal of Human-Computer Interaction. 37:1, 81–96, DOI: 10.1080/10447318.2020.1807710

We conducted a systematic review of the experimental studies investigating the effect of conversational agents’ and users’ characteristics on trust. From a full-text review of 29 articles, we identified five agent design themes affecting trust toward conversational agents: social intelligence of the agent, voice characteristics and communication style, look of the agent, non-verbal communication, and performance quality. We also found that participants’ demographics, personality, or use context moderate the effect of these themes.

D Smriti, R Rathod, J Huh-Yoo. 2021. Toward Privacy Implications in Developing and Deploying Chatbots in Informatics Research: Analysis of Chatbot Development Platforms. American Medical Informatics Association. In Press.

Chatbots have become widely integrated with third-party applications. This integration allows these third-party platforms to collect end-users personal information so that chatbots can provide personalized marketing and experiences to the end-users. However, it is unclear which third-party applications are integrated with various chatbot development platforms and how explicit their involvement is to the end-users. We address this knowledge gap by first examining nine text-based chatbot development platforms and the information given at the set-up process regarding various parties’ access to end-users information. We found that third-party applications gain end-user information through sign-in, deployment, and developers’ preferences. We discuss the limitations of not making third-party applications’ access and control of end-user information explicit. Future work includes analyzing privacy policy and terms of service documents to gain implications about the involvement of third-party applications when researching chatbots.

Kao, T.-S. A., Ling, J., Smriti, D., Shin, J.Y., Peng, W., Huh-Yoo, J. (accepted for March 2021). The Feasibility of Technology-Adapted Motivational Interviewing (TAMI), Poster presentation. 2021 MNRS Virtual Annual Research Conference.

Obesity remains a national public health crisis, as it increases the risk for diabetes, hypertension, cardiovascular diseases, and cancer. Motivational Interviewing (MI), an evidence-based counseling technique, can improve overweight/obese individuals’ lifestyles by building self-efficacy for healthy eating. There are multiple ways to deliver MI remotely (e.g., the internet, phone calls, and texts). However, these approaches are costly and limited in their dissemination capacity without the significant involvement of trained medical providers. We developed an automated standalone Technology-Adapted MI (TAMI) device to incorporate the MI technique. This study aimed to test the feasibility and preliminary efficacy of TAMI. The results suggest that TAMI is feasible and well-accepted by overweight/obese parents to improve their healthy eating behavior positively. Thus, TAMI may be a promising platform to deliver MI remotely. The future work would further examine how this interaction can lead to effective, long-term behavior change.

Research ethics

Huh-Yoo J, Kadri R, Buis LR. Pervasive Healthcare IRBs and Ethics Reviews in Research: Going Beyond the Paperwork. IEEE Pervasive Computing. 2021 Mar 16;20(1):40–4.

In this piece, we unraveled the multifaceted processes of understanding and communicating technology research risks involving human subjects. Through focusing on the history and importance of IRB and ethics committee oversight, we highlighted the importance of going beyond our moral obligations to conduct ethical research and pointed out the practical and logistical reasons for adhering to research ethics review procedures. We urged the critical need to think proactively, rather than retroactively, of what risks we introduce to study participants, including potential physical, emotional, social, legal, and economic harms. We also examined how information flows to external entities outside the research process that need special consideration. Finally, informed consent can be reframed as a process beyond receiving signatures for legal liability purposes. It is an opportunity to communicate risks to participants and to put into place strategies to mitigate those risks.

J Huh-Yoo, S Panicker, E Chiauzzi, E Kim. Human Subject Research in the Era of Emerging Digital Technologies, Ubiquity of Algorithms, and Evolving Privacy Norms. 2021. American Medical Informatics Association. In Press.

We conducted a workshop at AMIA to explore the interests of the AMIA audience on how existing ethical frameworks may or may not work in addressing the challenges with the changing technology environments in human subjects research.

Older adults and technology

MK. Schiaffino, Z. Zhang, D. Sachs, J. Migliaccio, J. Huh-Yoo, Predictors of Retention for Community-Based Telehealth Programs: A Study of the Telehealth Intervention Program for Seniors (TIPS). 2021. American Medical Informatics Association. In Press.

Community-based telehealth programs (CTPs) allow patients to monitor health at community-based facilities regularly. Evidence from community-based telehealth programs is scarce. We assessed retention factors — patients remaining active participants — in a CTP called the Telehealth Intervention Programs for Seniors (TIPS). We analyzed 5-years of data on social, demographic, and multiple chronic conditions among participants from 17 sites (N=1878). We modeled a stratified multivariable logistic regression to test the association between self-reported demographic factors, caregiver status, multiple chronic conditions, and TIPS retention status by limited English proficient (LEP) status. Overall, 59.5% of participants (mean age: 75.8yrs, median 77yrs, SD 13.43) remained active. Significantly higher odds of retention were observed among LEP females, English-speaking diabetics, and English proficient (EP) participants without a caregiver. We discuss the impact of CTPs in the community, the role of caregiving, and recommendations for retaining recruited non-English speaking participants.

L Wang, J Huh-Yoo. The Representativeness of “People Also Ask” of Google Web Search on the Information Needs Concerning Alzheimer’s Disease and Related Dementias. 2021. American Medical Informatics Association. In Press.

Researchers applied multiple research methods, including interviews, diary studies, focus groups, and questionnaires, to understand the information needs of people with Alzheimer’s disease and related dementias (ADRD) and their caregivers. Systematic reviews of consolidating existing studies can be helpful, but they are costly in terms of time and effort. Google Web Search, as the most widely used search engine in the United States, introduced the “People also ask” (PAA) section, which applied machine learning models, RankBrain, and BERT to identify search patterns, predict individual searches, and offer suggestions according to a search query. This can be an economical solution to understanding people’s information needs. We compared the results of systematic reviews and that of PAA to investigate how much PAA represents the review results. A clean-installed Chrome browser was applied to avoid the influence of personalized recommendations. The representativeness of PAA varied among the themes of information needs, possibly resulting from different usage habits on different platforms. PAA can be an efficient tool to identify specific areas of information needs. Future work should investigate the patterns of which information needs PAA represents better than others. Furthermore, we can explore combining behavior statistics from multiple aggregate search behavior tools (e.g., Google Trends, Alexa).

P. Nguyen, HW Choi, MK Schiaffino, Z Zhang, J Huh-Yoo. Qualitative Coding Framework for Analyzing Alert Notes from the Telehealth Intervention Program for Seniors (TIPS). 2021. American Medical Informatics Association. In Press

We analyzed and developed patient call and call-back workflows from 24,931 alert notes from the Telehealth Intervention Program for Seniors (TIPS), a community-based telehealth program in the tri-state area of the northeastern U.S. The workflow model presents multiple sequences of events, from an alert to a nurse’s call to the resulting plan for the patient. Based on the identified workflows, we developed a qualitative coding framework consisting of 20 codes and a regular expression-based information extraction approach for automating the coding of the alerts. We evaluated our automated alert triaging model with manually annotated alert notes for scalability on a stratified sampling of 762 alert notes for precision and recall analysis. The high-performance results show the scalability potential for this approach. The resulting coded alert notes can be cross analyzed with patients’ health monitoring results to generate predictive models and triaging of false alerts. The findings build steps toward developing an automated alert triaging model to identify alert types in remote health monitoring and telehealth systems.

J. Huh-Yoo, J. Sefcik, M. Coates, RA DiMaria-Ghalili. (2021) Disparity on Online Patient-Provider Communication and Implications for Post-COVID Era. As part of “Symposium: Preparing for the New Normal: Chronicling the Impact of COVID-19 on Older Adults and Providers.” Gerontological Society of America 2021. In Press.

Online patient-provider communication (OPPC) has emerged as a promising venue for addressing unmet health communication needs among individuals who need continued monitoring and self-care. OPPC increases access to health-related information enhances clinical management and self-care, and reduces healthcare expenditures. We conducted a descriptive cohort analysis of National Health and Aging Trends Study (NHATS) data collected between 2011–2016 (Waves 1–5). The data consist of a nationally representative sample of over 8,000 Medicare beneficiaries aged 65 and older. We focused on three survey items: (1) whether the respondent uses the internet (yes, no), (2) whether the respondent communicates with providers online (yes, no), and (3) questionnaires repeated during the COVID pandemic (2020). The cohorts were grouped into those who: (1) own computers (High Tech), (2) own cell phone only or don’t own any computer or cell phones (Low Tech), and (3) just adopted cell phone or computers (Transition). We found that the disparity between High Tech users versus Low Tech and Transition users was immensely high in terms of regular internet use, and this trend was steady throughout five years. More importantly, only the High Tech users increased their OPPC level by 75% over the five years, but other users remained the same. NHATS respondents noted they used more computers, emails, texts, and the internet than pre-COVID. While it is promising to gain more Transition users through the COVID pandemic, whether the disparity on OPPC and internet use, in general, will reduce is still in question.

Social technology design

JY. Shin, MJ Rheu, J. Huh-Yoo, W. Peng. Designing Technologies to Support Parent-Child Relationships: A Review of Current Status and Future Directions. Proceedings of the ACM on Human-Computer Interaction. 2021 (CSCW1) In Press.

We found two common challenges that designers should address in technology designed to support parent-child relationships: discrepancies in expected communication between parents and child(ren) and the complex emotions of parents toward parenting caused by their busy schedules. Challenges specific to families living apart included being physically distant and having limited access to communication resources. The following factors commonly helped facilitate parent-child relationships: (1) reciprocity norms of the family (2) reinforcement of transparency, affection, and trust, (3) a physical proxy of each other through an object or interface design, (4) accessibility, the sophistication level of technology, and communication resources, (5) enjoyable, age-appropriate shared content among parents and children, and (6) situational awareness and routine as ways to increase parent-child relationships. Media richness and synchronicity in system design and privacy preservation without interruption facilitated families’ parent-child relationships. These findings contribute to understanding opportunities for technological innovation for physically co-located families and the importance of considering children’s age and developmental stages in designing technology for parent-child relationships.

A. Bernstein, J. Huh-Yoo, A. Forte. Parent and Paraprofessional Use of Commercial Augmentative and Alternative Communication Devices with Non-verbal Autistic Young People. 2021. American Medical Informatics Association. In Press

We interviewed parents and speech-pathologists of non-verbal individuals on their experiences of aided augmented and alternative communication (AAC) devices and software. Commonly dedicated devices among the participants included: Accent Program with Unity, BIGmack, LAMP Words For Life, Avaz, Proloquo2Go. We found four major themes regarding participants’ experiences with AAC and a methodological insight. (1) Customization: It was critical to choose what devices and strategies work for non-verbal individuals, as one participant explained: “We say their voice, their choice.” (2) Maintenance: When a dedicated device breaks, it needs to be sent back to the company and can take weeks to be returned to the individual. The severity of disruption associated with inconsistent access was detrimental to user experience. (3) Two-way Communication: Caretakers reflected that they, too, not just non-verbal individuals, need to learn how to communicate effectively with the devices. There was a lack of resources for caretakers to learn how to use the device(s). (4) Usability: Low frustration tolerance is typical among people on the spectrum and can lead to distress, making choice overload an egregious design problem that can erect critical barriers to use. Usability problems such as labeling issues and communication symbol sets contributed to this frustration. Lastly, we learned that the participants who work with non-verbal individuals closely in a 1:1 relationship and their concerted efforts to achieve intersubjective understanding with learners linked their experiences in a way that is uniquely powerful among informants in qualitative research.

A Papoutsaki, H MacLeod, J Huh-Yoo, L Mamykina, A Miller, L Yarosh, D Epstein. The Future of Research on Online Health Communities: Discussing Membership, Structure, and Support. 2021. ACM CSCW. In Press.

We also organized a workshop to discuss the future of online health communities research. Results of this are forthcoming.

Mental health

Smriti, D., Ambulkar, S., Meng, Q., Kaimal, G., Ramotar, K., Park, S., Huh-Yoo, J. (in press). Creative Arts Therapies for the Mental Health of Emerging Adults: A Systematic Review. The Arts in Psychotherapy.

Emerging adults face a significant risk of suffering from mental health disorders. Creative and expressive art therapies (CATs) help provide a destigmatized approach to this condition. Art therapy and poetry therapy were the most frequently studied CATs. Fewer studies on drama therapy and music therapy were found, with no studies on dance movement therapy. The majority of art therapy studies reported statistical significance on its efficacy.

B. Choi, H. Kim, J. Huh-Yoo (2021). Examining YouTube Videos to Address Stigma in Seeking Mental Health Support among College Students. Journal of Medical Internet Research Formative Research. In Press.

YouTube videos on college students’ mental health can be well differentiated by the types of posters and the purpose of the videos. Individuals’ videos that shared personal stories and experiential knowledge (i.e., tips and advice) engaged more viewers in the videos’ short-term and long-term life. Individuals’ videos on YouTube showed the potential to support college students’ mental health in unique ways, such as providing social support, validating experience, and sharing the positive experience of help-seeking.

Fundings received

To fund the work stated here, we received the following awards:

2021–2022 NSF SBE Ethical and Responsible Research #22124894 ($45,450)
PI: Jina Huh-Yoo Co-PIs: Sangy Panicker, Edward Kim
New Frameworks for Ethical Research Amidst Emerging Digital Technologies, Algorithms, and Privacy Norms.

2021–2022 VoiceXML Forum ($25,000)
PI: Jina Huh-Yoo
VoiceXML-based Conversational Agent for the Caregivers of Alzheimer’s and Alzheimer’s Related Dementia Patients.
Collaborators: Aleksandra Sarcevic, Erjia Yan, Rose Ann Dimaria-Ghalili, Laura Gitlin

Presentations

Our work has been disseminated to the following invited venues:

American Medical Informatics Association, San Diego, CA. 10/28/2021–11/3/2021

Designing Smart Health Technologies To Support Family Health. Daly Lab, Department of Psychology, Drexel University. Online. 12/8/2021

Designing Smart Health Technologies To Support Family Health. Seoul National University, Seoul, Korea. Online. 11/10/2021

It’s the wild, wild west: Lessons learned from IRB member’s risk perceptions toward digital research data. Michigan State University. Online. 7/15/2021

Designing Smart Health Technologies To Support Family Health. College of Nursing and Health Professionals, Drexel University. 5/17/2021

New and continuing members

We had several new members join for summer research experience:

• Meredith Montgomery, HCC masters student, Drexel University
• Shirley Qian, HCC masters student, Drexel University
• Kasey Meredith, HCC masters student, Drexel University
• Rahil Rathod, Information Systems masters student, Drexel University
• Emilia Edwards, HCC masters student, Drexel University
• Noor Huda, HCC masters student, Drexel University
• Kasey Meredith, HCC masters student, Drexel University, College of Computing
• Sukhi Sandhu, HCC masters student, Drexel University, College of Computing
• Vincent Savarese, co-op undergraduate student, Drexel University, College of Computing (Funded by SmartCup project)
• Mohammed Alam, co-op undergraduate student, Drexel University, College of Computing (Funded by VoiceXML Forum grant)
• Vyshnavi Anandan, medical student, Drexel University
• Rishabh Matta, medical student, Drexel University

With the continuing members:

• Diva Smriti, Ph.D. student, Drexel University (Congratulations on accepting to start an internship in UX Research at Amazon Summer of 2022!)
• Lu Wang, Ph.D. student, Drexel University (Never got to visit Drexel yet due to COVID, and Lu will be finally physically joining our lab in January of 2022 from China!)

Jina Huh-Yoo, MHCI, PhD is currently an assistant professor in the College of Computing and Informatics at Drexel University. She directs the Health and Happiness Design Lab.