This study adhered to PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) guidelines for systematic review conduct and reporting [19,20]. We aimed to address the question, “What are the data visualization preferences and perceptions of people living with chronic neurological and mental health conditions when using RMT to manage health?” The project’s patient advisory board, which comprises patient advocates for depression, epilepsy, and multiple sclerosis, was consulted during the study design and data analysis.

This protocol was registered on PROSPERO (International Prospective Register of Systematic Reviews, CRD42019139319) while the review was in its pilot phase [21]. We searched PubMed, IEEE Xplore, EMBASE, Web of Science, proceedings from the Association for Computing Machinery Conference on Human Factors in Computing Systems, and the Cochrane Library for original, peer-reviewed, or gray literature published in English between January 2007 and September 2021. Searches included combinations of terms such as mHealth, along with terms related to data visualization and neurological disease. The search strings are provided in Multimedia Appendix 1. Relevant papers were also identified from manual searches of included studies’ reference lists. Studies were screened in 2 stages: abstract screening and full-text review. Eligibility criteria and screening forms were piloted on a set of 50 abstracts and 15 full-text reviews, and criteria were clarified or amended as needed. Two reviewers (AP, JN, SM, or DM) independently screened each abstract. In the case of disagreement, the abstract automatically proceeded to the full-text stage. Two reviewers then independently assessed each full-text paper for eligibility, and disagreements were resolved through discussion. If no consensus could be reached, a third member of the review team reviewed the paper and made a final determination. Agreement between reviewer pairs was determined through Cohen kappa [22]. Deduplication, record management, and screening were conducted in CADIMA, an open-access systematic review software [23]. Data extraction and coding were conducted using custom forms developed in Microsoft Word (Multimedia Appendix 1).

We included studies if they met the following criteria:

The following studies were ineligible for inclusion:

We adhered to Davis et al’s [24] definition of RMT, which includes “any technology that enables monitoring of a person’s health status through a remote interface,” which can then either be transmitted to a health care provider or as a tool for self-management of one’s health. We purposefully remained broad in our definition of the term “data visualization” because it is understood differently by different people. Therefore, we included any format through which RMT displayed data to service users. We defined chronic neurological or mental health conditions as any long-term, progressive, relapsing, or recurrent conditions related to mental health or dysfunction of the nervous system. Neurodegenerative diseases, depression, anxiety, or bipolar disorder, pain disorders, and sleep disorders, among others were eligible. Mental health conditions were also eligible if they were symptoms or comorbidities of a nonincluded condition. This heterogeneous set of conditions allowed us to identify themes that may be generalizable across conditions and others that are disease-specific.

Two authors independently reread each included study and extracted quotes related to data visualization preferences. When available, screenshots of data visualizations were also extracted. To ensure that analysis remained grounded in the context of the original studies, data extraction forms included a detailed description of each study’s objectives and methods, and annotated PDFs were preserved. Studies were critically appraised with the Mixed Methods Appraisal Tool [25,26]. We then categorized studies as conceptually rich “key papers,” “satisfactory papers,” which are methodologically acceptable but provide only moderate value to the synthesis, and “fatally flawed papers,” which contain major methodological flaws [27,28]. We also noted “minimal impact papers,” which provided minimal contribution to the synthesis.

We employed Thomas and Harden’s [29] inductive approach to thematic synthesis. Following a reading of the extracted text and its context, 2 authors (AP and JN) independently coded data line by line, producing a draft coding frame. The coding frame was iteratively amended, refined, restructured and the data recoded until no additional codes or disagreements were identified, and categorized into “descriptive themes,” which described the structure and content of the codes. Analytical themes, which interpreted the coded data, were developed through iterative rereading and discussion of the codes and thematically organized data (Table 1). An experienced qualitative researcher (SS) oversaw and provided input on this process. Following thematic synthesis, we conducted sensitivity analyses to identify potential differences between conditions. We analyzed 4 subgroups (mental health, neurological conditions, sleep, and pain) that arose from the patient populations and lines of inquiry addressed by included studies. As a form of member-checking, we consulted members of the patient advisory board (authors PB and SB) who reviewed our analysis and interpretations for face validity within the context of their own experiences of RMT and condition self-management.

Searches returned 2928 unique records. Of these, 177 papers were included in full-text review and 35 were eligible for qualitative synthesis (Figure 1). Reviewer agreement was moderate during abstract screening (weighted κ=0.45) and substantial during full-text review (weighted κ=0.79) [22]. Our relatively low agreement during abstract screening was expected and mitigated by reviewing the full texts of all abstracts, which were judged eligible by at least one reviewer. Multiple papers were identified for 3 research projects: the SPIRIT study (n=3) [32-34], the MoodRhythm app (n=2) [35,36], and the MONARCA (MONitoring, treAtment and pRediCtion of bipolAr Disorder Episodes) project (n=2) [37,38]. For these projects, all identified papers were analyzed as 1 study. Thus, 31 unique studies were included. The characteristics of the included studies are summarized in Table 2 and provided in detail in Multimedia Appendix 2. Most studies (19/31, 61%) addressed mental health conditions. Only 2 studies specifically investigated user perspectives on data visualizations [39,40]. All others reported on data visualization preferences within the broader context of RMT design, evaluation, or usability. We categorized 9 publications as key papers, 16 as satisfactory papers, 6 as minimal impact, and 4 as fatally flawed. We identified 3 themes through content analysis: desire for data visualization, impact of visualizations on condition management, and visualization design considerations (Textbox 1). The final coding frame and illustrative quotes are provided in Multimedia Appendix 2.

Users explicitly expressed a desire for data visualizations in 21 of the 31 included studies. Discussed or desired data included mood and disease-specific symptom scores, physical activity, sleep patterns and quality, and rhythm of daily activities. In 6 studies, RMT did not originally include visualizations of users’ historic data. Participants expressed their dissatisfaction, prompting authors to add visualizations to their designs [32-35,38,41,42].

Participants often reflected on aspects of visualization design, such as format, the need for contextual information, timeliness, and customization (Table 4). Health status, data literacy, and previous experience with RMT appeared to moderate individuals’ needs and design preferences.

When designed appropriately, data visualizations are perceived as valuable, engaging components of RMT. They can be validating and empowering, allowing users to become more active, responsible participants in their own care. Individual experiences of health conditions were perceived as highly personal, leading to requests for contextual aids, flexibility, and personalization. Factors such as health status, data literacy, and past experiences affected user needs, thereby moderating design preferences. Figure 2 describes the themes and relationships identified in this study.

Ample literature discusses scientifically derived best practices for app interface design, engagement strategies, and intervention design in various medical conditions and contexts [65-67]. Many of these modern practices emphasize user engagement, co-design, and human-centric design methods with an emphasis on scientific rigor [5,66,68]. In fact, many included studies reported on app development processes that adhered to these principles. Unfortunately, specific literature on visualization design within the context of RMT-enabled health management is sparse. This review aimed to address this gap. Based on our synthesis alone, we cannot suggest concrete examples of good design or universal design preferences. On the contrary, this review highlighted the diversity of preferences both across and within conditions and study samples. However, with this in mind, we pose several recommendations for RMT visualization design.

First, visualizations of personal data were consistently considered valuable, engaging, and even necessary components of RMT. When designed appropriately, they can be an integral part of condition self-management. Therefore, incorporation of data visualizations should be considered when designing RMT. However, designs should be sensitive to the experiences and health statuses of their target audiences. Participants were regularly concerned that visualizations that inadvertently emphasized slow progress, poor health, or unattained goals could prompt disengagement with RMT during periods of relapse or perceptible disease progression [32-34,42]. These warnings are consistent with the findings of Lee et al [69] who suggest that negative emotions and subsequent reductions in health self-efficacy reduce health information–seeking behavior [69].

Designs should be informed by a deep understanding of the service users’ needs and experiences. For example, study participants with rapid-cycling bipolar disorder suggested that tracking apps and visualizations were not useful to them since most apps allowed only 1 data entry per day [42,46]. Such mismatches between user needs and design features are avoidable if sufficient time is taken early in the design process to truly empathize with service users [5]. Designers should work with members of the target audience to understand their lived experiences, needs, and challenges. It is critical to engage with potential users during the design phase to understand when, how, and with whom data visualizations will be used. Fit-for-purpose visualizations can then be designed with these factors in mind.

Included studies repeatedly suggested that visualizations are not “one-size-fits-all.” Symptoms, triggers, contextual factors, and tracking needs differ from person to person, context to context, and timepoint to timepoint. As such, individual data visualization preferences are highly variable and often dynamic. This is consistent with Paterson’s [70] “shifting perspectives model of chronic illness,” which describes how individuals’ perceptions of their conditions, symptoms, and health fluctuate between predominant feelings of wellness and illness over time. Sometimes, individuals may predominantly self-identify as well. In a health-tracking context, this may lead service users to track symptoms less frequently or focus on wellness data (eg, physical activity, sleep) [31]. During such times, reminders of feelings of illness prompted by symptom tracking may even be detrimental to the service user’s self-image. However, when feelings of control over health status is threatened, such as during a relapse, individuals tend to shift their perspectives and more readily identify as ill [70]. During these times, individuals may be more inclined to self-track their symptoms to identify trends and regain control [46,50]. Owing to this interpersonal and intrapersonal variability, flexible design is highly desirable when it is feasible to implement. Designers should remain mindful of sources of heterogeneity in their target audience’s needs and subsequently identify RMT features that are most likely to require flexibility or personalization.

We also identified areas where tensions may emerge between user preferences and those of other stakeholders. Although this study focused on RMT designed for individual use, participants, especially those with mental health conditions, perceived data visualizations as tools to help communicate experiences to their care partners. However, clinicians often find it difficult to interpret and manage the visualizations derived from RMT [71], especially if visualizations are not standardized across the various apps used by patients. If visualizations are to be used as communication tools, they must be interpretable, meaningful, and actionable by those involved in patient care [72]. Otherwise, confusion or even miscommunication may ensue, potentially wasting valuable face-to-face time during short appointments with health care professionals [73]. Therefore, engagement with other possible users such as caregivers or health care professionals may also be valuable during visualization design.

As with all qualitative metasyntheses, the themes identified here are influenced by the topics, context, and limitations of the included studies. All results reported here must be interpreted as third-order constructs [74] or interpretations of interpretations of an individual’s original feedback. Although this is the aim of a meta-synthesis, it does pose limitations to remaining grounded in the authors’ and participants’ original meanings and contexts. As reviewers, we attempted to minimize our assumptions during coding and synthesis by continually questioning each other’s interpretations of the data. However, we acknowledge that is impossible to fully isolate our synthesis from our own experiences and worldviews. Study methodology was heterogeneous and the authors’ place in the data was not always clear. Most comments on visualizations were brief and conducted as part of a larger studies, yielding relatively shallow data. Few studies compared preferences between multiple visualization designs, limiting our ability to compare the merits or relative preferences between design options. Most studies used user experience and co-design methods, which generally did not meet traditional expectations of qualitative research rigor. Although we opted for broad search strategies and sensitive review methods to identify all relevant studies, data visualization was almost always a secondary topic in the included papers and was not always clearly discussed. It is therefore a possibility that our strategies did not identify all eligible papers. Those papers that were included predominantly reflected the perspectives of individuals with mental health conditions, and it is unclear how these results may translate to other populations.

Few studies directly addressed research questions related to data visualization preferences. Rather, data visualization was usually approached as peripheral piece of a larger project. Therefore, this review should serve as a starting point for targeted work on health data visualization design. Additional qualitative research on service-user data visualization preferences is warranted, especially within the context of neurological condition self-management. This research should involve individuals directly through interviews, focus groups, or other qualitative methods and should focus on generating “thick” qualitative data that describe more specific uses and contexts. This work should verify and build upon the concepts developed here. Ideally, this work should span multiple conditions and generate diverse perspectives. These perspectives should then be used to generate more detailed design considerations and recommendations for RMT visualizations.

When designed appropriately, data visualizations are valuable engaging components of RMT. They can provide structure and insight, allowing individuals, especially those with mental health conditions, to manage their own health more effectively. However, visualizations are not “one-size-fits-all,” and it is important to engage with potential users during visualization design to understand when, how, and with whom visualizations will be used to manage health. The considerations presented here should serve as the basis for future designs and discussions to ensure that visualizations address users’ needs and preferences.