Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 12.08.20 in Vol 7, No 8 (2020): August

Preprints (earlier versions) of this paper are available at, first published Nov 07, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    Designing a Clinician-Facing Tool for Using Insights From Patients’ Social Media Activity: Iterative Co-Design Approach

    1School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA, United States

    2The Zucker Hillside Hospital, Northwell Health, Glen Oaks, NY, United States

    3The Feinstein Institutes for Medical Research, Manhasset, NY, United States

    4The Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, United States

    Corresponding Author:

    Dong Whi Yoo, MFA, MS

    School of Interactive Computing

    Georgia Institute of Technology

    756 W Peachtree St NW

    Atlanta, GA, 30308

    United States

    Phone: 1 4043858603



    Background: Recent research has emphasized the need for accessing information about patients to augment mental health patients’ verbal reports in clinical settings. Although it has not been introduced in clinical settings, computational linguistic analysis on social media has proved it can infer mental health attributes, implying a potential use as collateral information at the point of care. To realize this potential and make social media insights actionable to clinical decision making, the gaps between computational linguistic analysis on social media and the current work practices of mental health clinicians must be bridged.

    Objective: This study aimed to identify information derived from patients’ social media data that can benefit clinicians and to develop a set of design implications, via a series of low-fidelity (lo-fi) prototypes, on how to deliver the information at the point of care.

    Methods: A team of clinical researchers and human-computer interaction (HCI) researchers conducted a long-term co-design activity for over 6 months. The needs-affordances analysis framework was used to refine the clinicians’ potential needs, which can be supported by patients’ social media data. On the basis of those identified needs, the HCI researchers iteratively created 3 different lo-fi prototypes. The prototypes were shared with both groups of researchers via a videoconferencing software for discussion and feedback. During the remote meetings, potential clinical utility, potential use of the different prototypes in a treatment setting, and areas of improvement were discussed.

    Results: Our first prototype was a card-type interface that supported treatment goal tracking. Each card included attribute levels: depression, anxiety, social activities, alcohol, and drug use. This version confirmed what types of information are helpful but revealed the need for a glanceable dashboard that highlights the trends of these information. As a result, we then developed the second prototype, an interface that shows the clinical state and trend. We found that focusing more on the changes since the last visit without visual representation can be more compatible with clinicians’ work practices. In addition, the second phase of needs-affordances analysis identified 3 categories of information relevant to patients with schizophrenia: symptoms related to psychosis, symptoms related to mood and anxiety, and social functioning. Finally, we developed the third prototype, a clinical summary dashboard that showed changes from the last visit in plain texts and contrasting colors.

    Conclusions: This exploratory co-design research confirmed that mental health attributes inferred from patients’ social media data can be useful for clinicians, although it also revealed a gap between computational social media analyses and clinicians’ expectations and conceptualizations of patients’ mental health states. In summary, the iterative co-design process crystallized design directions for the future interface, including how we can organize and provide symptom-related information in a way that minimizes the clinicians’ workloads.

    JMIR Ment Health 2020;7(8):e16969





    Psychotic disorders, such as schizophrenia and schizoaffective disorder, constitute a serious menace to personal health [1]. Symptoms characterizing these conditions emerge during critical years of adolescence; interfere with the establishment of healthy social, educational, and vocational foundations; and are associated with devastating individual and familial loss [2]. However, there are no known cures for these conditions [3]. Once developed, they often become chronic and must be managed [4]. Subjective patient experiences in the form of clinical interviews, questionnaires, patient self-reports, and family observations have played a central role in the management of these conditions for over a century [5]. These approaches require that behavioral, emotional, or cognitive symptoms are recalled from a patient’s memory, a method prone to retrospective recall bias [6,7]. Patients also frequently withhold information from their clinicians because of stigma, guilt or shame, failure to remember relevant events, or to meet impression management goals [8]. In fact, the psychopathology of serious mental illnesses may affect the very mental, emotional, and social judgments on which patients’ self-reported responses are based [9]. Therefore, there is a need to access more objective information about patients to augment patients’ verbal reports in clinical settings.

    In recent years, patient-generated data, machine learning, and digital health technologies have been recognized to offer unprecedented opportunities to help consumers, clinicians, researchers, and other stakeholders measure, manage, and improve mental health [10-12]. In particular, digital records of people’s activities on social media, such as Facebook and Twitter, have been shown to enable the development of machine learning algorithms that can assess and even predict risk to a variety of mental health challenges [10], such as major depression [13], postpartum depression [14], posttraumatic stress [15,16], and schizophrenia [17]. Research has also demonstrated that a variety of symptomatic expressions and outcomes of clinical interest (eg, depression risk or suicidal ideation) may be gleaned from such data [18,19]. Although social media platforms may negatively affect users’ behaviors (eg, being confined to echo chambers [20], exposure to misinformation [21], and increased social comparison [22]), they constitute voluntarily shared and recorded data that include the feelings, opinions, and daily experiences of their users. Therefore, with social media, it is possible to unobtrusively and longitudinally gather insights about people’s social interactions, affect, and cognitive styles—signals characteristic of one’s mental health state and thus critical to management and treatment of mental illnesses but which may be challenging to assess via psychometric or self-reported means because of concerns of impression management and retrospective recall bias [10].

    Although researchers have argued that social media analysis can be used to support real-world mental health care settings, interventions, and clinical diagnosis and assessment [23,24], actual opportunities concerning such technology in the clinical setting have not been explored. As a way to bridge this gap, through this research, our ultimate goal is to develop a clinician-facing tool that can provide mental health insights derived from patients’ social media at the point of care. This future tool will collect patients’ social media data under their informed consent and analyze the collected data to distill clinically meaningful insights. The analyzed data will be shown in a way that can support mental health clinicians’ work practices without adding intellectual burden or information overload.


    To achieve the aforementioned design goal, there are many challenges we need to overcome: we must understand the current work practices of mental health clinicians; we need to explore clinicians’ potential needs, barriers, and concerns regarding the concept of this future interface; and, most importantly, we need to introduce the concept of delivering mental health insights from patients’ social media to clinicians—a practice that currently does not exist. Therefore, as a first step toward conceptualizing this future technology, drawing on approaches in the human-computer interaction (HCI) field, we aimed to create a series of low-fidelity (lo-fi) prototypes informed by the clinicians’ opinions and needs regarding these insights gleaned from social media. These include whether social media analysis can be useful in the context of their clinical practices, what kinds of insights from patients’ social media data may be useful, and how these insights can be presented so as to be seamlessly integrated into their workflows in the future.

    We, as an interdisciplinary research team, co-designed the lo-fi prototypes. Co-design, which is widely used in the design research community, can be defined as a dialog between end users and designers [25]. We extended the concept of co-design as a long-term discussion between end users and designers. The 4 clinical researchers in this team were the potential end users of the prototype. At the same time, they were also domain experts who could represent end users. The rest of the HCI researchers served as designers and builders of future tools that can support clinicians.

    Over the 6 months of co-design activities, we adopted the needs-affordances analysis framework to support the iterative process. Needs-affordances analysis is a method to refine concrete design directions based on both what the users want to have (needs) and what the technology/platform can provide (affordances) [26].

    We first discuss the concept of co-design and needs-affordances analysis as the method of this project. Then, we present the 3 different lo-fi prototypes that are the results of our co-design activities. Finally, we discuss the lessons learned from this project and future plans for building clinician-facing mental health technologies that are powered by patients’ social media data.


    Iterative Co-Design Process

    Our research explores the building of technologies to support clinicians in mental health treatment, specifically focusing on schizophrenia, as a case example. Schizophrenia is a chronic and severe mental disorder characterized by abnormal social behaviors and distorted perceptions of reality [2]. Clinical observations indicate that it is among the most challenging mental illnesses to manage, primarily because of its high likelihood of relapse. Most patients experience multiple relapses during the course of the illness [27]. Management of the illness, therefore, involves preventing the occurrence of a relapse as well as symptomatic exacerbation [28]. Previous research has demonstrated that we can computationally extract behavioral and linguistic patterns from social media posts of patients with schizophrenia [17,29]. These approaches help us to predict mental health states and clinical outcomes, such as relapse episodes in schizophrenia [30], or to identify a clinically valid diagnosis of the illness [31]. Given the rich evidence that social media can be helpful for understanding patients with schizophrenia, we deemed focusing on the context of schizophrenia, which allows us to examine whether such information may be used at the point of care through a technological interface.

    These empirical evidences are further supported by recent research that has revealed patients’ interest in and willingness to use their social media data to improve their treatment outcomes. Padrez et al [24] found that as many as 70.4% (1008/1432) of patients who reported using social media and were willing to participate in research also agreed to share these data and have them linked to their medical records for health research. Similarly, Rieger et al [32] reported that in a survey of 238 mental health patients, of those who had a social media account and were receiving therapy, 74.4% (99/133) indicated that they would be willing to share their social media posts with their therapists, if their therapists were concerned about how they were doing, including 53% (50/94) who were willing to share both public and private posts. Specific to psychotic disorders and most relevant to our context, based on semistructured interviews with 112 psychotic disorder patients, in a work by Birnbaum et al [33], 80.7% (88/109) of patients were open to sharing their social media data with clinical researchers to explore how it can potentially inform their treatment. Drawing on these findings, and because clinical researchers have advocated that it is important to first focus on issues related to the accuracy, interpretability, meaning, and actionability of these data such as social media [34], in this paper, we focus on co-design exercises with clinicians. Our rationale is further bolstered by Baier’s [35] observation, “Without judicious consideration, social media use by [clinicians] can lead to inadvertent self-disclosures to [patients] that risk damaging the therapeutic alliance, interfering with therapeutic processes, and placing both the [patient] and clinician at risk”.

    As mentioned earlier, this paper focuses particularly on the process of building a series of lo-fi prototypes toward the aforementioned goal. In its basic form, a prototype is an expression of design intent. Prototyping allows designers to present their designs and see them in action. In the context of digital technologies such as our case, a prototype is a simulation of the final interaction between the user (the clinician) and the interface (communicating insights from patients’ social media activities). lo-fi prototyping is a quick and easy way to translate high-level design concepts into tangible and testable artifacts. The first and most important role of lo-fi prototypes is to check and test functionality rather than the visual appearance of the product.

    To develop our lo-fi prototypes, an interdisciplinary research team collaborated for 6 months to envision future technologies through an iterative co-design process (Table 1 provides the details of the team members). The team members spoke for roughly an hour via video conference calling every 2 weeks because the team was geographically distributed. In earlier meetings, we discussed experts’ user needs and the potential affordances of our future technology. Experts included clinicians working at a large health care system in the northeast of the United States. After we (HCI researchers at a large public university in the southeast of the United States) developed design sketches based on those discussions, we exchanged our ideas and thoughts while reviewing our sketches via the video conferencing app. During the discussion, one of the researchers wrote the points discussed and shared the notes with other researchers. The written notes mainly informed our design decisions during the iterative process, and the decisions were reflected during the subsequent meetings. This research was approved by the institutional review boards of the relevant institutions.

    Table 1. Details of the backgrounds of the team members.
    View this table

    More elaborately, we adopted an iterative strategy, in which we created a design sketch led by the HCI researcher on the team, obtained feedback on it from the clinician collaborators, and then adjusted it accordingly to create a modified or new sketch. Over time, the clinicians felt insights from social media could work as collateral information [36,37], which could complement existing ways of gathering information from patients and thereby overcome some of their limitations. Collateral information, which is information obtained from sources such as family members, primary providers, and friends, in mental health is important because it may supply objective, finer-grained data; can facilitate time-sensitive monitoring of symptoms; and can provide an ability to detect subtle but burgeoning changes in a patient’s condition, eventually improving clinical assessments.

    Furthermore, we postulated Facebook as our target social media platform for discussions surrounding the co-design process. This is because Facebook is widely adopted among young adults [38], and previous research has established that it is the most widely used social media site among the target patient population [23]. Moreover, in contrast to other social media platforms, Facebook provides higher identity and social affordances and has a rich set of social interaction capabilities [39], which allows the possibility to glean a variety of potentially relevant mental health insights.

    The abovementioned iterative process of our co-design activities was open ended because of the exploratory nature of our study; however, we used the following questions as starting points when we evaluated our sketches:

    • Do the visualizations support the clinicians’ needs that are identified by a needs-affordances analysis?
    • Are the visualizations easy to understand?
    • Will the visualizations be compatible with clinicians’ work practices?
    • What are the more/less important categories?
    • What are the other categories that can be helpful for clinicians?
    • What are the areas of improvement?

    On the basis of the feedback from these questions, we decided the direction of the next iterations. The details of the 3 iterations are described in Results section.

    Needs-Affordances Analysis

    Affordance refers to the features of an object in the physical environment that may cause interactions between the object and an agent [26]. In other words, affordances basically represent the opportunities for action that an object offers. Moreno and D’Angelo [39] proposed an affordances framework to design platform-sensitive social media–based health interventions. They also presented empirical evidence supporting why the consideration of platform affordances can support future health intervention designs. We follow this line of thought: we use this framework to choose our target social media platform, Facebook, and then, we consider the affordances of Facebook to create our lo-fi prototypes for eventual tools that can be used to augment clinical care and treatment settings.

    Antonenko et al [26] used the concept of affordances to enable researchers to design/evaluate new technologies that can support user needs. They proposed a needs-affordances analysis as a framework developed within the field of education research to design and evaluate new education technologies. It helped them to define user needs, identify a potentially appropriate technology, and align technological affordances with the specific needs of target users. Technological affordances are the possibilities of interactions a new technology can offer to potential users, which can be similar to the affordances of interventions in Moreno and D’Angelo’s framework.

    In this study, we apply a needs-affordances analysis to the process of creating lo-fi prototypes. It plays 2 vital roles in the process. First, it provides a systematic structure to the discussion between HCI researchers and clinical researchers. This highlights the need for clinical researchers to define user needs to crystallize a future interface while providing opportunities for HCI researchers to explain what kinds of needs can be supported by social media data analysis. Second, although this analytical framework does not provide a detailed guideline for design decisions, its ability to reveal the alignment of needs and affordances provides a hint of what the prototypes should be.

    The combination of iterative co-design and needs-affordances analysis gave us 3 prototypes: (1) a card-type interface for treatment goal tracking, (2) a clinical state and trend visualization interface, and (3) a clinical summary dashboard.


    A Card-Type Interface for Treatment Goal Tracking

    On the basis of consensus in the early discussions that social media can be a source of collateral information, our first lo-fi prototype generation process began by exploring the needs of our possible user group: mental health clinicians. Clinical researchers, grounded in clinical theory and practice, made a list of common treatment goals [40] they care about in the real-world clinical setting (Table 2). We compiled a list of the different kinds of information and computational insights that can help to assess whether a patient is close to, progressing toward, or digressing from their treatment goals. Next, we discussed the affordances of our chosen social media platform (Facebook) that can be relevant to meeting these treatment goals. For example, social and emotional affordances, which are related to the opportunities to communicate with others and express their emotions on social media, can be relevant to assess whether the patients are increasing or decreasing their social interactions and/or sharing emotional vulnerability with their friends. On the basis of the affordances of Facebook, we then discussed the affordances of the first version of the future interface. In connection to the aforementioned example, this step translated to discussing how information gleaned from patients’ Facebook data can be supported by Facebook’s affordances such as tagging, messaging, and friending. The frequency of tagging others in a post, the volume of private messages patients exchanged with others, and the number of newly added Facebook friends could be indicative of social interaction in a patient’s life, a critical dimension of most patients’ treatment goals and illness management.

    Table 2. The first needs-affordances analysis.
    View this table

    In addition to the needs identified earlier, we referred to previous work that visualized patient-generated data for clinicians [41-43]. Previous studies used a dashboard design including a card-type interface where one card represents one type of data/information and users can modify the order and types of cards on the dashboard. We also adopted their design directions because a card-type interface provides flexibility, so researchers can iteratively add and change different types of information. Similar to the previous work, our first sketch (Figure 1) is a card-type visualization, where each card includes the level of the attribute, such as depression, anxiety, social activities, alcohol and drug use, and sleep—all dimensions identified by the clinicians to be important components of tracking common treatment goals in schizophrenia. For depression and anxiety, we envisioned a scale for each symptom that a machine learning approach could automatically infer from the posts (eg, based on the algorithm developed in a study by Saha et al [44]). If a clinician intended to read the actual posts for a specific day, they could hover their mouse over the bar and read the posts on the pop-up layer. For social activities, the page was mainly based on the social affordances of Facebook, such as likes, check-ins, and comments. For the alcohol and drug card, we assumed our interface to identify alcohol- and drug-related posts and present them as a circle on the timeline. For sleep, because social media may not be the best proxy for measuring sleep patterns, we chose to present social media activities after midnight, which may imply some level of sleep hygiene—previous work by De Choudhury et al [13], for instance, found that the degree of night-time social media activity, operationalized as an insomnia index, was positively correlated with increased depression risk.

    Taken together, clinical collaborators were able to see some value in this form of visualization while pointing out areas of improvement; they pointed out that clinicians may not have time to review all the different cards during an ongoing session with a patient. Furthermore, they noted that they are likely to be most interested in learning about trends around when and how specific mental health attributes showed a change for the patient in question. They also clarified that social media activities related to the quality of life measurement can be interesting to them [15]; hence, a goal of this second prototype was to explore how to present such information derived from the social media data of patients.

    Figure 1. The first prototype: a card-type interface for treatment goal tracking.
    View this figure

    A Clinical State and Trend Visualization Interface

    Incorporating feedback from clinical collaborators, we developed the second version of our sketch. This interface is a desktop app that clinicians can access using their personal machines during appointments. The interface consists of 3 different components: the first area is the left menu, which presents information about the patient and clinicians as well as a tree menu of different pages (Mood, Cognition, Social Functioning, and Language); the second area is the main content of the page, an overview dashboard as default; and the third area shows long-term trends of all relevant attributes at the bottom.

    To arrive at the different components of this interface, in addition to needs-affordances analysis, the widely studied stress-vulnerability model [45] of schizophrenia informed the process of selecting and organizing the additional pages of mood, cognition, social functioning, and language. The model assumes that individuals have an intrinsic level of psychosocial vulnerability, which, in turn, can decide the level of impact realized from different, external, and stressful events. When their vulnerability is high enough and the stressor is strong enough, they will experience transient intermediate states that can be characterized by processing capacity overload, autonomic hyperarousal, and impaired processing of social stimuli. Transient intermediate states can negatively impact a person’s social environment, which can result in repetitive experiences of these states. If this harmful cycle continues until the transient intermediate states surpass an individual’s threshold, it can cause the development of schizophrenic symptoms. Therefore, determining vulnerability is important in assessing and treating people who are suffering from schizophrenia. Previous work has identified various attributes as markers of psychosocial vulnerability: mood, cognition, linguistic style, social functioning, semantic disorganization, syntactic disorientation, and others [46,47], corresponding to each of which we created a page in our second prototype.

    In addition, recall that a necessity identified during the discussion of the first prototype centered on clinicians’ desire to learn about patients’ quality of life attributes. Typically, clinicians track such attributes via unstructured conversations with their patients during visits, and it can range over a variety of topics, such as information about their romantic life, work, family, socialization activities, and so on. Inclusion of the Mood, Cognition, Social Functioning, and Language pages in this prototype can thus enable tracking such trends over short and long periods in patients’ Facebook data.

    We now describe the different components in detail below.

    The overview page, in particular, shows a default view; as shown in Figure 2, the interface uses simple visualization techniques (eg, dot, bar, and line charts) to provide insights that can support common treatment goals, such as learning about trends and monitoring changes related to depression, anxiety, social functioning, drug and alcohol use, and sleep hygiene, since the patient’s last clinical visit.

    Figure 2. The second prototype: a clinical state and trend visualization interface – "Overview" dashboard.
    View this figure

    Moving on the specific pages in this interface, first, Figure 3 shows the Mood category, which provides detailed information about mood and affective states that can augment depression and anxiety information shown in the summary page. Following psychology literature, we provided the level of the 2 different facets of mood—valence (the extent of positivity or negativity of mood) and arousal (the intensity of mood) [48]—and the level of 11 different affective states, such as joviality and fatigue, which have been reliably inferred from computational analyses of social media data in previous work [49]. Owing to the complexity of the nature of mood and affective states, we created simple line graphs for each dimension of those categories.

    Figure 3. The second prototype: a clinical state and trend visualization interface – "Mood".
    View this figure

    Another category that originated from the stress-vulnerability model, Cognition, is important in understanding schizophrenia because the model defines stress as the perceived discrepancies between an individual and the environment surrounding the individual [45]. The additional page (Figure 4) provides scores related to the cognitive processes based on the well-validated computerized text analysis tool, Linguistic Inquiry and Word Count (LIWC) [17,50]. The validity of LIWC was initially approved by the level of agreement between human judges and LIWC’s objective word counting [51], and many studies using social media posts show that LIWC can be applicable to understanding mental health attributes [15,17,52].

    Figure 4. The second prototype: a clinical state and trend visualization interface – "Cognition".
    View this figure

    Next, Social Functioning provides 6 different levels of social media activities that can function as features of a hypothetical machine learning algorithm that can calculate the overall social function score (Figure 5). We selected interpersonal relationships that are accessible on Facebook (number of posts and number of friends) as well as work- and school-related posts because the clinician researchers highlighted the importance of being able to return to their daily activities. Social media activities include the number of posts they have written, the interactions with other Facebook users, work- or school-related posts, and their number of friends.

    Figure 5. The second prototype: a clinical state and trend visualization interface – "Social Functioning".
    View this figure

    Among the different Language-related pages, the linguistic style page (Figure 6) includes some categories from LIWC. The scores for a specific category (eg, 0.08 for first-person singular pronouns) are calculated from the aggregated texts they have posted on Facebook since their last clinical visit; this implies the fraction of words in those texts that are first-person singular pronouns. Previous work has shown that people who self-disclose themselves as patients with schizophrenia show different patterns in pronoun use [17]. The bottom of the page, the long-term trend window, shows temporal changes of a selected category of linguistic style (shown for preposition use) since the patient created their Facebook account.

    Figure 6. The second prototype: a clinical state and trend visualization interface – "Linguistic Style".
    View this figure

    Another Language page in this prototype, entitled semantic disorganization (Figure 7), shows the result of topic modeling applied to patients’ Facebook data. Topic models are algorithms for discovering the latent themes that pervade a large, and otherwise unstructured, collection of documents [53]. We used a visualization technique to use topic models as a navigator that enables users to explore the hidden structure that the algorithm reveals [53]. In Figure 7, the themes identified by the algorithm are listed. As the themes are grouped in a quantitative way, the label of the theme is a group of prevalent words in the theme rather than an intuitive, interpretable title. We envision that human annotation of the label of the theme can provide more meaningful/interpretable information, as has been leveraged in previous work [17]. When users click a theme on the list, the words in the theme and the actual posts in the theme are shown on the right side of the page.

    Figure 7. The second sketch: a clinical state and trend visualization interface – "Semantic Disorganization".
    View this figure

    Figure 8 presents the final page within the Language category, entitled syntactic disorientation, which contains information about linguistic coherence, readability, repeatability, and complexity. We have added this information because of empirical evidence from the literature. Previous work that used natural language analysis models of linguistic change proved that social media users, who self-disclosed themselves as patients with schizophrenia, showed higher repeatability and complexity while, at the same time, lowering coherence and readability when compared with a control group [17]. We selected a simple line graph to maintain the coherence of the visual elements throughout the sketches.

    Figure 8. The second prototype: a clinical state and trend visualization interface – "Syntactic Disorientation".
    View this figure

    Owing to the elaborate nature of this prototype, clinical researchers provided several rounds of feedback, spanning multiple weeks. Broadly speaking, they appreciated the richness of the information conceptualized to be presented at the interface, which spanned both common treatment goal markers as well as other potentially useful information that can be gleaned from patients’ social media data, which, in turn, could be relevant to gather an understanding of a patient’s quality of life. At the same time, they were skeptical about maintaining the linguistic style page because they felt that our potential users may not have the time to review these highly elaborate different bar charts; may not be familiar with specific language analytic tools such as LIWC, despite being commonplace in the research literature; and may lack knowledge of how the different linguistic analyses were performed. Consequently, they may not be able to interpret the line graphs in a clinically meaningful way. However, clinical researchers have found some value in the linguistic style page; for example, they believed that the temporal reference category, such as past tense, present tense, and future tense, can be meaningful because it is related to understanding the future orientation of patient’s thoughts; future orientation can often indicate an improvement in mental health state [54].

    Furthermore, the detailed navigation of patient postings that topic models provide intrigued clinical researchers; the topic model allowed clinical researchers to start thinking about psychosis-related symptoms. Although we did not discuss psychosis-related vulnerability attributes in the first needs-affordances analysis (Table 2), the clinical researchers realized that this information can help lead to a better understanding of their patients. Therefore, we decided to discuss more deeply how we can cover psychosis-related symptoms of schizophrenia in our next iterations of the prototype. We believe this is one of the positive results of the iterative co-design of prototypes; conceptualizing what we need from new technology is difficult even using needs-affordances analysis because domain experts have to ideate several different possible needs while dealing with an unfamiliar and complicated concept for a new, futuristic technology. As a result, the sketches provide a chance to both reflect on potential needs and consider missing needs. In our case, the second prototype focused on mood- and social functioning–related information, but the resulting sketch that highlighted the potential usefulness of the information related to psychosis was missing. Therefore, we decided to conduct a second needs-affordances analysis focusing on the symptoms of schizophrenia patients, which led us to develop a new sketch.

    A Clinical Summary Dashboard

    As noted earlier, because we discovered the value of psychosis-related symptoms after reviewing the second version of the sketches, before iterating over and creating the next sketch, we conducted a second needs-affordances analysis (Table 3). During this needs-affordances analysis, the clinical researchers first provided a list of symptoms of interest that are considered expert needs, and then, HCI researchers identified the affordances of platforms and the future interfaces that can correspond with the symptoms. For instance, for symptoms associated with psychosis, the clinical researchers have made a list of symptoms, such as delusions, preoccupations, and hallucinations. We assumed that these symptoms could be inferred from linguistic characteristics and interaction patterns in their Facebook use because of the identity and cognitive affordances of Facebook, as shown in previous computational work on social media and schizophrenia [17].

    This second needs-affordances analysis produced 3 groups of expert user needs—symptoms associated with psychosis [46], symptoms associated with mood/anxiety subject to the comorbidities of schizophrenia [55], and social functioning [56]. The first group, symptoms associated with psychosis, was the most deviant part from the previous prototype. The latter 2 groups were similar to treatment goals, such as eliminating depressed mood and improving existing relationships, but as a positive outcome of this iterative process, we were able to make a more detailed list of information needs of mental health clinicians that were represented by the clinical researchers in our team.

    From the same needs-affordances analysis, we also learned that clinicians may wish to have an extremely simple summary of social media analysis. Clinical researchers suggested just letting them know whether the severity of symptoms has changed since the last visit using plain text, rather than visuals such as bar graphs. The purpose of the plain text summary is to provide both concise information and opportunities to further explore details when they think the change is interesting, from the perspective of a particular patient meeting their desired treatment goal(s).

    Table 3. The second needs-affordances analysis.
    View this table

    Figure 9 shows the first page of this summative dashboard. Symptoms from the second needs-affordance analysis that have changed since the last visit are shown using colored circles and plain text. The green circles indicate the symptoms that have improved since the last visit, and the red circles indicate the deterioration of a symptom. The plain text tells which symptoms have changed, and if applicable, what the changes were (eg, there were 2 posts related to work/school for the work/school category in the illustration of Figure 9). We assumed that symptoms that have changed since the last few visits can be interesting starting points for clinicians; if they want to look at the details, they can click and see bar charts that are similar to those in Figure 2.

    Figure 9. The final (third) version of our prototype: A clinical summary dashboard.
    View this figure


    Lessons Learned

    The iterative nature of our co-design process enabled us to expand the potential use of the prototype. Clinical researchers’ immediate responses to the first sketches were ambivalent: they were positive about being able to gather more fine-grained collateral information about their patients, which may help them overcome the weaknesses of current self-report–based information gathering processes. At the same time, they were concerned about whether the potential information could overload their work practices, as social media–derived information could be seen as another layer of noise. After they reviewed the revised sketches that were more well aligned with their needs and concise enough to be used in a limited amount of time, they started to consider other details. For example, they were highly interested in social functioning–related information in general.

    Clinical researchers also envisioned potential alternative uses of the tool that can support their work practices. In earlier meetings, the clinical researchers suggested that future tools could be more beneficial for therapists than psychiatrists because therapists usually have longer interactions with patients that can be aided by the proposed technology. However, after they saw the second prototype—the clinical state and trend visualization interface (Figure 2)—they were of the opinion that they may be able to use the dashboard to facilitate conversations between the treatment team—a combination of psychiatrists, psychologists, therapists, and social workers—which meets periodically in smaller, more cutting-edge, care-centric mental health clinics.

    In addition, our co-design activities essentially revealed an information abstraction gap, that is, a gap between the computational abstraction of patients’ Facebook data and the clinical abstraction of patients’ states. We introduced computational approaches within our designs because clinicians expressed the busy nature of their job, and patient social media data, although perceived to be valuable, can be gigantic. Computational approaches allow converting raw social media data into a more consumable unit of information (ie, from the total number of posts to the number of depression-indicative posts). These types of abstracted information derived from computational approaches were successful when the information was intuitive and compatible with clinical concepts. One of the most successful cases was the measures in the overview page in the second prototype (Figure 2). However, at the same time, aside from this potential, clinical researchers were skeptical in other measures that are based on either machine learning algorithms (eg, the performance metrics of the algorithms) or psycholinguistic analysis (eg, LIWC scores) because interpreting these measures is not a part of the current clinical culture; some measures may not be clinically meaningful or align with how a clinician assesses a patient’s mental health state, whereas others may not support their current workflows.

    Therefore, our co-design focused on bridging the gaps between what the computational approaches can provide and what clinicians can actually use at the point of care. The 2 needs-affordances analyses crystalized the latter part—how they abstract the diverse types of patient information into clinical categories. The second analysis yielded the 3 categories that summarized clinicians’ mental models of how they navigate patients’ information to make a clinical decision. The second and third prototypes explored how computational approaches can be matched to the clinicians’ mental models. The simplified visual representation in the third prototype reconciled the vast amount of data with the clinicians’ needs for immediate, actionable insights from new technology.

    Although we were able to result in a meaningful bridge between the gaps, there are still interesting aspects to explore; most importantly, the flow of patient-clinician consultations varied based on the types of clinicians, patients, and treatment strategies. In addition, information abstraction gaps can exist between stakeholders, such as clinicians, patients, the managers of hospitals, family members, and other caregivers. We need to explore, in future work, how computational abstraction can support/hamper communications between the stakeholders. Finally, the information abstraction gap is a sociotechnical phenomenon that is situated in cultures, policies, and laws surrounding the computational artifact and the users of the artifact. Therefore, future research should take into account institutional and cultural support to make new technologies implementable and sustainable.

    Comparison With Previous Work

    The concept of the use of mental health insights from social media is relatively novel. Fisher et al [57] provided case reports where clinicians had begun to incorporate patients’ electronic communications, such as social media in their care. For instance, these authors reported that one clinical team reviewed an outpatient’s Facebook feed because the patient was vulnerable to self-harm and violent behavior. In the face of these isolated incidents, to understand systematically how social media data may augment treatment for mental health conditions, Hobbs et al [58] surveyed 115 outpatient psychotherapy clinicians associated with McLean Hospital in Belmont, Massachusetts. They found that 61.7% (71/115) of clinicians reported viewing at least one patient’s social or electronic media as part of psychotherapy, and 92% (65/71) of those endorsed being able to provide more effective treatment using this information.

    However, despite revealing these potentials, design opportunities for social media at the point of care have not yet been explored. Our work is based on evidence in recent literature that explored the use of digital phenotyping data [59-61]. For example, Luo et al [60] identified dietitians’ needs for patient information regarding food, reflection, symptoms, activity, and physical state. Similar to this work and others, our co-design approach also identified clinicians’ needs for more objective patient information and contextualized the use of social media for this purpose. Our findings regarding clinicians’ needs for symptoms and social functioning in patients with schizophrenia, in fact, resonate with similar observations in previous research. Specifically, we suggest that patient information can be grouped as symptoms related to psychosis, symptoms related to mood/anxiety, and social functioning. Furthermore, to address the cognitive overload of clinicians that has been identified in previous work [62], our co-design approach resulted in a clinical summary view that provides changes in an extremely simple form.

    Limitations and Future Research

    Even though we were able to learn valuable lessons from the co-design approach, we acknowledge that there are certain limitations in this work. First, we note that the computational approaches that are considered in our prototype are in the early stages of research and require more rigorous and clinical validation before being deployed in clinical settings. Second, the user needs we considered are the potential user needs that are inferred by a small number of clinical researchers. It is possible that their projections of user needs may not fully represent our target groups’ opinions. We believe this limitation can be overcome with subsequent studies to gather user needs with a representative group of participants. It should be noted that we have not evaluated our prototype with potential users as a form of user study that can yield diverse design implications and guidelines. As mentioned earlier, this is the very first stage of a long-term project where we envisioned a future tool as an interdisciplinary working group, and we considered using a long-term co-design activity for the first step. We believe this first step has informed our team to proceed to the next stage of our research, including building a medium-fidelity prototype and a user study using the functioning prototype.

    In addition to the abovementioned points, we also acknowledge that patient perspectives and ethical considerations are massive issues surrounding the concept of using patient-generated data, such as with our proposed technology. That said, as a formative step, we wanted to ensure that clinicians perceived the technology in question to have clinical value before involving patients directly. We know that patients’ autonomy and agency are critical in designing a technology that uses sensitive information from vulnerable populations. Future endeavors will augment these insights by including patients in an iterative design process to explore their perspectives, especially their autonomy and agency.

    As a next step, we intend to conduct a qualitative evaluation study using our prototype, where we will introduce the concept of the future interface to a diverse set of clinicians. At the same time, we will seek their opinions about adopting this future technology, potential barriers to using this concept, and other concerns. We believe our prototypes can be useful for exploring these opinions because we co-designed these prototypes with domain experts who can represent our potential user group. This was the result of a relatively long-term discussion between HCI researchers and domain experts, which enabled them to educate each other on their own expertise, ideate future directions, and discuss remaining concerns. It might not be possible to have this deep discussion during traditional user study sessions, such as one-time, one-hour long interviews. We believe our future participants in the user study sessions using our prototypes can provide rich responses to questions regarding the adoption of this concept because our prototypes can be well aligned with their potential needs and work practices. Specifically, we are interested in questions related to usability (whether they can understand and use the prototype without problems), compatibility (whether the tools can be used in their work practices without frictions), and perceived clinical usefulness (whether they foresee the clinical value of using the prototype). We hope this co-design process and future qualitative study using the prototypes can lead us to develop a usable and useful clinician-facing tool in the future that can be used efficaciously at the point of care.


    The authors appreciate the valuable feedback from Sindhu Ernala, Kelsie Belan, and other members of the Georgia Tech Social Dynamics and Well-Being Lab and Georgia Tech Ubicomp Group. This research was partly supported by funds from a cooperative research agreement issued by Northwell Health to Georgia Tech and a National Institute of Mental Health grant to MDC (R01MH117172).

    Conflicts of Interest

    None declared.


    1. van Os J, Kenis G, Rutten BP. The environment and schizophrenia. Nature 2010 Nov 11;468(7321):203-212. [CrossRef] [Medline]
    2. Amador XF, Strauss DH, Yale SA, Gorman JM. Awareness of illness in schizophrenia. Schizophr Bull 1991;17(1):113-132. [CrossRef] [Medline]
    3. Mechanic D. The challenge of chronic mental illness: a retrospective and prospective view. Hosp Community Psychiatry 1986 Sep;37(9):891-896. [CrossRef] [Medline]
    4. Bachrach LL. Defining chronic mental illness: a concept paper. Hosp Community Psychiatry 1988 Apr;39(4):383-388. [CrossRef] [Medline]
    5. Davies T. ABC of mental health. Mental health assessment. Br Med J 1997 May 24;314(7093):1536-1539 [FREE Full text] [CrossRef] [Medline]
    6. Hertwig R, Fanselow C, Hoffrage U. Hindsight bias: how knowledge and heuristics affect our reconstruction of the past. Memory 2003;11(4-5):357-377. [CrossRef] [Medline]
    7. Smith DM, Brown SL, Ubel PA. Mispredictions and misrecollections: challenges for subjective outcome measurement. Disabil Rehabil 2008;30(6):418-424. [CrossRef] [Medline]
    8. Sherman M, Trief P, Sprafkin R. Impression management in the psychiatric interview: quality, style, and individual differences. J Consult Clin Psychol 1975 Dec;43(6):867-871. [CrossRef] [Medline]
    9. Liddle PF. The symptoms of chronic schizophrenia. A re-examination of the positive-negative dichotomy. Br J Psychiatry 1987 Aug;151:145-151. [CrossRef] [Medline]
    10. Mohr DC, Zhang M, Schueller SM. Personal sensing: understanding mental health using ubiquitous sensors and machine learning. Annu Rev Clin Psychol 2017 May 8;13:23-47 [FREE Full text] [CrossRef] [Medline]
    11. Torous J, Staples P, Onnela J. Realizing the potential of mobile mental health: new methods for new data in psychiatry. Curr Psychiatry Rep 2015 Aug;17(8):602 [FREE Full text] [CrossRef] [Medline]
    12. Saha K, Chan L, de Barbaro K, Abowd GD, De Choudhury M. Inferring mood instability on social media by leveraging ecological momentary assessments. Proc ACM Interact Mob Wearable Ubiquitous Technol 2017 Sep 11;1(3):1-27. [CrossRef]
    13. De Choudhury M, Gamon M, Counts S, Horvitz E. Predicting Depression via Social Media. In: Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media. 2013 Presented at: AAAI'13; July 8-11, 2013; Cambridge, Massachusetts, USA.
    14. De Choudhury M, Counts S, Horvitz E, Hoff A. Characterizing and Predicting Postpartum Depression From Shared Facebook Data. In: Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing. 2014 Presented at: CSCW'14; February 15-19, 2014; Baltimore, Maryland, USA. [CrossRef]
    15. Coppersmith G, Harman C, Dredze M. Measuring Post Traumatic Stress Disorder in Twitter. In: Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media. 2014 Presented at: AAAI'14; June 1-4, 2014; Ann Arbor, Michigan, USA   URL:
    16. Saha K, De Choudhury M. Modeling stress with social media around incidents of gun violence on college campuses. Proc ACM Hum-Comput Interact 2017 Dec 6;1(CSCW):1-27. [CrossRef]
    17. Ernala SK, Rizvi AF, Birnbaum ML, Kane JM, De Choudhury M. Linguistic markers indicating therapeutic outcomes of social media disclosures of schizophrenia. Proc ACM Hum-Comput Interact 2017 Dec 6;1(CSCW):1-27. [CrossRef]
    18. De Choudhury M, Kiciman E. The Language of Social Support in Social Media and its Effect on Suicidal Ideation Risk. In: Proceedings of the Eleventh International AAAI Conference on Web and Social Media. 2017 Presented at: WSM'17; May 15-18, 2017; Montreal, Canada.
    19. De Choudhury M, Kiciman E, Dredze M, Coppersmith G, Kumar M. Discovering Shifts to Suicidal Ideation from Mental Health Content in Social Media. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 2016 Presented at: CHI'16; May 7-12, 2016; San Jose, California, USA   URL: [CrossRef]
    20. Gillani N, Yuan A, Saveski M, Vosoughi S, Roy D. Me, My Echo Chamber, and I: Introspection on Social Media Polarization. In: Proceedings of the 2018 World Wide Web Conference. 2018 Presented at: WWW'18; April 23-27, 2018; Lyon, France. [CrossRef]
    21. del Vicario M, Bessi A, Zollo F, Petroni F, Scala A, Caldarelli G, et al. The spreading of misinformation online. Proc Natl Acad Sci U S A 2016 Jan 19;113(3):554-559 [FREE Full text] [CrossRef] [Medline]
    22. Vogel EA, Rose JP, Roberts LR, Eckles K. Social comparison, social media, and self-esteem. Psychol Pop Media Cult 2014 Oct;3(4):206-222. [CrossRef]
    23. Birnbaum ML, Rizvi AF, Confino J, Correll CU, Kane JM. Role of social media and the internet in pathways to care for adolescents and young adults with psychotic disorders and non-psychotic mood disorders. Early Interv Psychiatry 2017 Aug;11(4):290-295 [FREE Full text] [CrossRef] [Medline]
    24. Padrez KA, Ungar L, Schwartz HA, Smith RJ, Hill S, Antanavicius T, et al. Linking social media and medical record data: a study of adults presenting to an academic, urban emergency department. BMJ Qual Saf 2016 Jun;25(6):414-423. [CrossRef] [Medline]
    25. Zimmerman J, Tomasic A, Garrod C, Yoo D, Hiruncharoenvate C, Aziz R, et al. Field Trial of Tiramisu: Crowd-Sourcing Bus Arrival Times to Spur Co-Design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2011 Presented at: CHI'11; May 7-12, 2011; Vancouver, BC, Canada. [CrossRef]
    26. Antonenko PD, Dawson K, Sahay S. A framework for aligning needs, abilities and affordances to inform design and practice of educational technologies. Br J Educ Technol 2016 Jun 9;48(4):916-927. [CrossRef]
    27. Harrow M, Sands JR, Silverstein ML, Goldberg JF. Course and outcome for schizophrenia versus other psychotic patients: a longitudinal study. Schizophr Bull 1997;23(2):287-303. [CrossRef] [Medline]
    28. Kane JM. Treatment strategies to prevent relapse and encourage remission. J Clin Psychiatry 2007;68(Suppl 14):27-30 [FREE Full text] [Medline]
    29. Birnbaum ML, Ernala SK, Rizvi AF, De Choudhury M, Kane JM. A collaborative approach to identifying social media markers of schizophrenia by employing machine learning and clinical appraisals. J Med Internet Res 2017 Aug 14;19(8):e289 [FREE Full text] [CrossRef] [Medline]
    30. Birnbaum ML, Ernala SK, Rizvi AF, Arenare E, van Meter AR, De Choudhury M, et al. Detecting relapse in youth with psychotic disorders utilizing patient-generated and patient-contributed digital data from Facebook. NPJ Schizophr 2019 Oct 7;5(1):17 [FREE Full text] [CrossRef] [Medline]
    31. Ernala S, Birnbaum M, Candan K, Rizvi A, Sterling W, Kane J, et al. Methodological gaps in predicting mental health states from social media: triangulating diagnostic signals. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019 Presented at: CHI'19; May 4-9, 2019; Glasgow, UK. [CrossRef]
    32. Rieger A, Gaines A, Barnett I, Baldassano CF, Connolly Gibbons MB, Crits-Christoph P. Psychiatry outpatients' willingness to share social media posts and smartphone data for research and clinical purposes: survey study. JMIR Form Res 2019 Aug 29;3(3):e14329 [FREE Full text] [CrossRef] [Medline]
    33. Birnbaum ML, Rizvi AF, Faber K, Addington J, Correll CU, Gerber C, et al. Digital trajectories to care in first-episode psychosis. Psychiatr Serv 2018 Dec 1;69(12):1259-1263 [FREE Full text] [CrossRef] [Medline]
    34. Seltzer E, Goldshear J, Guntuku SC, Grande D, Asch DA, Klinger EV, et al. Patients' willingness to share digital health and non-health data for research: a cross-sectional study. BMC Med Inform Decis Mak 2019 Aug 8;19(1):157 [FREE Full text] [CrossRef] [Medline]
    35. Baier AL. The ethical implications of social media: issues and recommendations for clinical practice. Ethics Behav 2018 Oct 11;29(5):341-351. [CrossRef]
    36. Carey KB, Simons J. Utility of collateral information in assessing substance use among psychiatric outpatients. J Subst Abuse 2000;11(2):139-147. [CrossRef] [Medline]
    37. Neimark G, Hurford MO, DiGiacomo J. The internet as collateral informant. Am J Psychiatry 2006 Oct;163(10):1842. [CrossRef] [Medline]
    38. Social Media Fact Sheet. Pew Research Center. 2019.   URL: [accessed 2019-06-02]
    39. Moreno MA, D'Angelo J. Social media intervention design: applying an affordances framework. J Med Internet Res 2019 Mar 26;21(3):e11014 [FREE Full text] [CrossRef] [Medline]
    40. Grosse M, Grawe K. Bern inventory of treatment goals: part 1. Development and first application of a taxonomy of treatment goal themes. Psychother Res 2002 Mar 1;12(1):79-99. [CrossRef] [Medline]
    41. Kim Y, Heo E, Lee H, Ji S, Choi J, Kim J, et al. Prescribing 10,000 Steps Like Aspirin: Designing a Novel Interface for Data-Driven Medical Consultations. In: Proceedings Proceedings of the 2017 CHI Conference on Human Factors in Computing Systemsof the 2017 CHI Conference on Human Factors in Computing Systems. 2017 Presented at: CHI'17; May 6-11, 2017; Denver, USA. [CrossRef]
    42. Ryokai K, Michahelles F, Kritzler M, Syed S. Communicating and Interpreting Wearable Sensor Data with Health Coaches. In: 9th International Conference on Pervasive Computing Technologies for Healthcare. 2015 Presented at: PervasiveHealth'15; May 20-23, 2015; Istanbul, Turkey. [CrossRef]
    43. Yoo DW, De Choudhury M. Designing Dashboard for Campus Stakeholders to Support College Student Mental Health. In: Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare. 2019 Presented at: PervasiveHealth'19; May 20-23, 2019; Trento, Italy. [CrossRef]
    44. Saha K, Sugar B, Torous J, Abrahao B, Kıcıman E, De Choudhury M. A social media study on the effects of psychiatric medication use. Proc Int AAAI Conf Weblogs Soc Media 2019 Jun 7;13:440-451 [FREE Full text] [Medline]
    45. Zubin J, Spring B. Vulnerability--a new view of schizophrenia. J Abnorm Psychol 1977 Apr;86(2):103-126. [CrossRef] [Medline]
    46. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull 1987;13(2):261-276. [CrossRef] [Medline]
    47. Sabb FW, van Erp TG, Hardt ME, Dapretto M, Caplan R, Cannon TD, et al. Language network dysfunction as a predictor of outcome in youth at clinical high risk for psychosis. Schizophr Res 2010 Feb;116(2-3):173-183 [FREE Full text] [CrossRef] [Medline]
    48. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol 1988 Jun;54(6):1063-1070. [CrossRef] [Medline]
    49. De Choudhury M, Counts S, Gamon M. Not All Moods are Created Equal! Exploring Human Emotional States in Social Media. In: Sixth International AAAI Conference on Weblogs and Social Media. 2012 Presented at: ICWSM'12; June 4-7, 2012; Dublin, Ireland.
    50. Tausczik YR, Pennebaker JW. The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol 2009 Dec 8;29(1):24-54. [CrossRef]
    51. Pennebaker JW, Francis ME. Linguistic Inquiry and Word Count Loose Leaf. New York, USA: Lawrence Erlbaum; 2016.
    52. Wongkoblap A, Vadillo MA, Curcin V. Researching mental health disorders in the era of social media: systematic review. J Med Internet Res 2017 Jun 29;19(6):e228 [FREE Full text] [CrossRef] [Medline]
    53. Blei DM. Probabilistic topic models. Commun ACM 2012 Apr 1;55(4):77-84. [CrossRef]
    54. Miloyan B, Pachana NA, Suddendorf T. Future-oriented thought patterns associated with anxiety and depression in later life: the intriguing prospects of prospection. Gerontologist 2017 Aug;57(4):619-625 [FREE Full text] [CrossRef] [Medline]
    55. Addington D, Addington J, Maticka-Tyndale E. Assessing depression in schizophrenia: the Calgary depression scale. Br J Psychiatry Suppl 1993 Dec(22):39-44. [Medline]
    56. Endicott J, Spitzer RL, Fleiss JL, Cohen J. The global assessment scale. A procedure for measuring overall severity of psychiatric disturbance. Arch Gen Psychiatry 1976 Jun;33(6):766-771. [CrossRef] [Medline]
    57. Fisher CE, Appelbaum PS. Beyond googling: the ethics of using patients' electronic footprints in psychiatric practice. Harv Rev Psychiatry 2017;25(4):170-179. [CrossRef] [Medline]
    58. Hobbs KW, Monette PJ, Owoyemi P, Beard C, Rauch SL, Ressler KJ, et al. Incorporating information from electronic and social media into psychiatric and psychotherapeutic patient care: survey among clinicians. J Med Internet Res 2019 Jul 12;21(7):e13218 [FREE Full text] [CrossRef] [Medline]
    59. Chung CF, Agapie E, Schroeder J, Mishra S, Fogarty J, Munson S. When Personal Tracking Becomes Socialxamining the Use of Instagram for Healthy Eating. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 2017 Presented at: CHI'17; May 6-11, 2017; Denver, Colorado, USA. [CrossRef]
    60. Luo Y, Liu P, Choe EK. Co-Designing Food Trackers with Dietitians: Identifying Design Opportunities for Food Tracker Customization. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 2019 Presented at: CHI'19; May 4-9, 2019; Glasgow, Scotland. [CrossRef]
    61. Zhu H, Luo Y, Choe EK. Making Space for the Quality Care: Opportunities for Technology in Cognitive Behavioral Therapy for Insomnia. In: Proceedings Proceedings of the 2017 CHI Conference on Human Factors in Computing Systemsof the 2017 CHI Conference on Human Factors in Computing Systems. 2017 Presented at: CHI'17; May 6-11, 2017; Denber, Colorado, USA. [CrossRef]
    62. Hall A, Walton G. Information overload within the health care system: a literature review. Health Info Libr J 2004 Jun;21(2):102-108 [FREE Full text] [CrossRef] [Medline]


    HCI: human-computer interaction
    LIWC: Linguistic Inquiry and Word Count
    lo-fi: low-fidelity

    Edited by J Torous; submitted 07.11.19; peer-reviewed by F Delahunty, L McCann; comments to author 01.02.20; revised version received 27.06.20; accepted 09.07.20; published 12.08.20

    ©Dong Whi Yoo, Michael L Birnbaum, Anna R Van Meter, Asra F Ali, Elizabeth Arenare, Gregory D Abowd, Munmun De Choudhury. Originally published in JMIR Mental Health (, 12.08.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.