This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited. The complete bibliographic information, a link to the original publication on http://mental.jmir.org/, as well as this copyright and license information must be included.
Background
Remote patient monitoring is increasingly integrated into health care delivery to expand access and increase effectiveness. Automation can add efficiency to remote monitoring, but patient acceptance of automated tools is critical for success. From 2010 to 2013, the Diabetes-Depression Care-management Adoption Trial (DCAT)–a quasi-experimental comparative effectiveness research trial aimed at accelerating the adoption of collaborative depression care in a safety-net health care system–tested a fully automated telephonic assessment (ATA) depression monitoring system serving low-income patients with diabetes.
Objective
The aim of this study was to determine patient acceptance of ATA calls over time, and to identify factors predicting long-term patient acceptance of ATA calls.
Methods
We conducted two analyses using data from the DCAT technology-facilitated care arm, in which for 12 months the ATA system periodically assessed depression symptoms, monitored treatment adherence, prompted self-care behaviors, and inquired about patients’ needs for provider contact. Patients received assessments at 6, 12, and 18 months using Likert-scale measures of willingness to use ATA calls, preferred mode of reach, perceived ease of use, usefulness, nonintrusiveness, privacy/security, and long-term usefulness. For the first analysis (patient acceptance over time), we computed descriptive statistics of these measures. In the second analysis (predictive factors), we collapsed patients into two groups: those reporting “high” versus “low” willingness to use ATA calls. To compare them, we used independent t tests for continuous variables and Pearson chi-square tests for categorical variables. Next, we jointly entered independent factors found to be significantly associated with 18-month willingness to use ATA calls at the univariate level into a logistic regression model with backward selection to identify predictive factors. We performed a final logistic regression model with the identified significant predictive factors and reported the odds ratio estimates and 95% confidence intervals.
Results
At 6 and 12 months, respectively, 89.6% (69/77) and 63.7% (49/77) of patients “agreed” or “strongly agreed” that they would be willing to use ATA calls in the future. At 18 months, 51.0% (64/125) of patients perceived ATA calls as useful and 59.7% (46/77) were willing to use the technology. Moreover, in the first 6 months, most patients reported that ATA calls felt private/secure (75.9%, 82/108) and were easy to use (86.2%, 94/109), useful (65.1%, 71/109), and nonintrusive (87.2%, 95/109). Perceived usefulness, however, decreased to 54.1% (59/109) in the second 6 months of the trial. Factors predicting willingness to use ATA calls at the 18-month follow-up were perceived privacy/security and long-term perceived usefulness of ATA calls. No patient characteristics were significant predictors of long-term acceptance.
Conclusions
In the short term, patients are generally accepting of ATA calls for depression monitoring, with ATA call design and the care management intervention being primary factors influencing patient acceptance. Acceptance over the long term requires that the system be perceived as private/secure, and that it be constantly useful for patients’ needs of awareness of feelings, self-care reminders, and connectivity with health care providers.
Trial Registration
ClinicalTrials.gov NCT01781013; https://clinicaltrials.gov/ct2/show/NCT01781013 (Archived by WebCite at http://www.webcitation.org/6e7NGku56)
technology assessmenttelecommunicationstelemedicinepatient care managementclinical decision support systemsdepressiondiabetes mellitussafety-net clinicsIntroduction
In late 2014, the Centers for Medicare and Medicaid Services (CMS) issued new rules that expanded provider reimbursements beginning in 2015 for remote monitoring of Medicare beneficiaries [1]. Telemedicine and telehealth—or, what Kvedar and colleagues refer to collectively as “connected health”—capitalize on advances in health information technology (HIT) to remotely provide health care services, information, health education, and self-management support [2]. A number of studies have demonstrated the potential of these technologies to increase access and quality of care while decreasing health care costs [3-8].
Given the mounting evidence for the clinical and cost effectiveness of connected health, the CMS ruling is likely to boost interest in its adoption. Attempts to improve patient care with connected health, however, will be futile unless patients accept these technologies. Prior studies suggest that individuals who do not accept technologies simply will not use them [9,10]. Indeed, so critical is user acceptance that it has been regarded as “the pivotal factor in determining the success or failure of an information system” [11]. In an editorial review of connected health technologies to support behavior changes for self-management, Piette [12] remarks that patients’ discontinued use, which results from a lack of acceptance, has largely hindered large-scale implementation. Therefore, it is clear that patient acceptance has important implications for the broader domain of connected health, since patients who do not accept (and thus do not use) these technologies will not realize the full benefits of them, resulting in a loss for both patients and payers.
This study investigates patient acceptance of an automated telecommunications system designed to facilitate depression care management of low-income patients with diabetes in a safety-net care system [13,14]. There is evidence of significant disparities in receipt of depression treatment in low-income, uninsured, and minority populations. These groups are less likely to receive depression care [15-19], show greater treatment discontinuation [20], and experience higher rates of clinically significant depression. Patient barriers to depression care influence detection and treatment processes. For example, minority patients are less likely to voluntarily report depressive symptoms, may view depression as a moral weakness or character flaw instead of an illness, may be more likely to ascribe symptoms of depression to a physical illness [21,22], and may refuse or discontinue treatment due to stigma [23]. Nonadherence to depression treatment in minority groups with diabetes is common, due in part to side effects of diabetes medications [24-26]. Further exacerbating the challenges are cost and complex patient-provider interactions inherent in caring for patients with comorbid chronic illnesses. For instance, prioritizing among competing demands may negatively affect initiation and long-term follow-up of depression management in primary care [27-34].
To address these issues in order to accelerate the adoption of evidence-based depression care [24,35], we designed an advanced automated telephonic assessment (ATA) system. It had the capability to inquire—via periodic telephone calls to patients—about important aspects of depression care using a combination of the following six modules: monitoring for depression, assessing pain, assessing adherence to antidepressant medications, assessing psychotherapy practice, prompting depression self-care activities, and allowing patients to request contact from a clinician [14]. The ATA system was fully integrated into an existing disease management registry (DMR), which allowed it to automatically select these modules and the frequency of calls depending on individual patient clinical data: results from previous ATA calls or clinical assessments (depressed patients were called monthly, nondepressed patients quarterly), whether patients had an active antidepressant medication prescription, and whether patients were receiving psychotherapy. It also allowed patients to indicate their preferences for language (English or Spanish), call days and times, and receiving human calls instead of machine calls. If a call was not answered, the ATA system attempted again three times per day for 7 days (morning, afternoon, and evening). As a whole, the design allowed the ATA system to individually customize calls to focus on patients’ specific needs and preferences rather than having patients adapt to standard comprehensive assessments, in essence illustrating the philosophy of patient-centered care.
Moreover, the ATA system facilitated timely, proactive follow-up by clinicians and staff. Data captured on the ATA calls were automatically assessed and the results sent to the DMR for clinician and staff review [14]. Notifications, tasks, and alerts were triggered in response to specific issues identified from the ATA calls: patient requests for contact, high depression scores, nonadherence to antidepressant medications, or suicidal ideation.
The ATA system was tested in the Diabetes-Depression Care-management Adoption Trial (DCAT) [13]. DCAT was a 12-month, quasi-experimental comparative effectiveness research trial conducted in collaboration with the Los Angeles County Department of Health Services (LACDHS) with the aim of comparing different approaches for accelerating the adoption of collaborative team depression care in routine safety-net primary care practice. The study was conducted in its ambulatory care clinics serving low-income, racially/ethnically diverse (but primarily Hispanic/Latino) patients. It tested three depression care delivery models: usual care (UC), supported care (SC), and technology-facilitated care (TC). UC represented the status quo, whereby primary care providers (PCPs) and their staff initiate the translation and adoption of depression care evidence. Both SC and TC included care teams of the LACDHS disease management program (DMP) for the first 6 months of the trial to support diabetes care as well as depression care using evidence-based protocols [36]. After 6 months, patients returned to their PCPs for care. The difference between SC and TC was that the latter utilized the ATA system for 12 months to facilitate automated depression screening and monitoring, and timely follow-up by clinicians and staff. The provider notifications, tasks, and alerts generated by the ATA system were sent to DMP teams during the first 6 months of DCAT and to PCPs and their staff during the second 6 months.
If such automated remote screening and monitoring of depression—and more broadly, connected health—is to be integrated into mainstream health care delivery to help reach the important policy goal of expanding access to high-quality, effective, and efficient care, an understanding of patient technology acceptance is urgently needed. Studies on remote assessment and monitoring via connected health, however, continue to overlook this important research area [35,37]. Those that do touch upon elements of patient acceptance tend to be cross-sectional and operationalize the construct using measures of patient satisfaction with care, which in itself reveals little about technology acceptance or how to design the system to improve patient acceptance.
The present study echoes the information technology literature [38-40] by measuring technology acceptance as patients’ willingness to use ATA calls as part of their depression care. Moreover, this study is longitudinal, which allows for an understanding of how patient acceptance may change over time. Finally, to inform future design choices for automated remote depression monitoring technology, the evaluation includes several system characteristics that may explain why patients accept or reject the technology. Thus, in sum, the study was undertaken to (1) determine patient acceptance of ATA calls for remote depression screening and monitoring over time, and (2) identify what factors predict long-term patient acceptance of ATA calls.
MethodsStudy Design and Participants
To answer the research questions, we analyzed data collected from patients in the TC arm of DCAT. English-Spanish bilingual interviewers administered assessments of technology acceptance at 6, 12, and 18 months. Thus, patients received two assessments during the study and one assessment 6 months after the study had ended.
Survey-Based Measures of ATA Call Acceptance
DCAT defined technology acceptance as patients’ reported willingness to use ATA calls in the future as part of their depression care. The measurement was administered at 6, 12, and 18 months. DCAT also assessed additional measures of ATA call design characteristics: perceived ease of use (7 items), perceived usefulness (6 items), perceived nonintrusiveness (3 items), and perceived privacy/security (1 item). DCAT administered these assessments at 6 and 12 months. Moreover, patients’ preference for mode of reach (1 item) was assessed at 6, 12, and 18 months. Finally, at 18 months, patients were asked about their long-term perceived usefulness of ATA calls (3 items). All measures were assessed on a 5-point Likert scale. Table 1 provides the exact wording used in the DCAT assessments.
Measures of patient ATA call acceptance.
Domain of measurement
Items
Administration
Willingness to use ATA callsa
To what extent do you agree or disagree with the following statement?
6, 12, and 18 months
You would not mind receiving automated calls as part of your depression care in the future.
Perceived ease of useb,c
“How often would you say...”
6 and 12 months
The language used by Amyc in the calls was easy for you to understand?
Amy’s voice on the call was loud enough to hear without straining?
Amy was speaking too fast on the automated call?
You were clear on how to respond to Amy’s questions?
You had difficulty answering the questions when asked to press buttons on your phone?
Giving answers to a real person would have been easier than giving answers to the automated operator Amy?
Amy had difficulty understanding you when you responded verbally?
Perceived usefulnessb,c
“How often would you say...”
6 and 12 months
The call made you feel confident that your nurse or social worker knew how you were doing?
The calls made you feel like your nurse of social worker was more accessible?
The calls by Amy were just as effectiveness in reporting your feelings as an in-person visit with your care provider?
The antidepressant medication questions asked by Amy reminded you to take your medications?
The problem-solving skills questions asked by Amy reminded you to use these skills?
The calls reminded you to do things like a physical activity or a fun activity?
Perceived nonintrusivenessb
“How often would you say...”
6 and 12 months
You enjoyed receiving the calls?
You felt the calls were a bother?
The length of the calls seemed about right?
Perceived privacy/securitya
To what extent do you agree or disagree with the following statement?
6 and 12 months
You feel automated calls are private and/or secure.
Preferred mode of reach
To what extent do you agree or disagree with the following statement?
6, 12, and 18 months
Instead of receiving automated calls, you would prefer to call the automated service at your convenience.
Long-term perceived usefulnessa
To what extent do you agree or disagree with the following statements?
18 months
The automated calls helped you be more aware of how you are feeling.
The automated calls reminded you to take care of your health, such as doing exercise.
The automated calls helped you stay better connected with your doctors, nurses or social worker.
aPatients responded using a 5-point Likert scale of agreement (1=strongly disagree, 2=disagree, 3=neutral, 4=agree, and 5=strongly agree).
bPatients responded using a 5-point Likert scale of frequency (1=never, 2=rarely, 3=about half the time, 4=usually, and 5=always).
c“Amy” was the persona of the ATA calls
ATA Call Completion Rates
We assessed the rate of completed ATA calls for three periods: 0-6 months, 7-12 months, and 0-12 months. An ATA call was defined as complete if it reached the patient and recorded answers to the depression assessment questions: PHQ-2 or PHQ-9, whichever was asked.
Statistical Analysis
We conducted two analyses: one to determine patient acceptance of ATA calls for remote depression screening and monitoring over time, and the other to identify what factors predict long-term patient acceptance of ATA calls. Sample characteristics and sample sizes for each analysis are shown in the Results section (Table 2).
For the first analysis (patient acceptance over time), we included the DCAT TC arm patients who provided responses for a given survey-based measure at each of the measurement periods. By excluding patients who did not meet this criterion, we were able to estimate changes more accurately for each measure over time. We computed descriptive statistics of all measures. For those measures consisting of multiple items, we computed the average points across items and rounded the average to the nearest integer. Furthermore, we conducted a paired t test to determine if there was a significant difference between the ATA call completion rates from 0 to 6 months and from 6 to 12 months. We also used Spearman rank correlation to test the association between the ATA call completion rate of months 0 to 12 and the survey-based measures of ATA call acceptance.
In the second analysis (predictive factors), we used a different criterion to select patients from among the pool of TC arm patients: patients who responded to the question of willingness to use ATA calls at 18 months and at least once at 6 or 12 months or both. If patients answered the question at both 6 and 12 months, we computed the average for use in the analysis. The 125 patients in this sample were collapsed into two groups: those reporting “high” willingness to use ATA calls at 18 months (Likert scale response of 4 or higher) and those reporting “low” willingness to use ATA calls at 18 months (all other response categories). We compared the descriptive statistics for the two groups: patient sociodemographic characteristics, health conditions, health care utilization, and ATA call completion rate. We also compared their responses for perceived ease of use, perceived usefulness, perceived nonintrusiveness, perceived privacy/security, preference of ATA call mode, and long-term perceived usefulness. If patients completed assessments of these measures at both 6 and 12 months, we computed the average of the two for use in the analysis. To compare the two groups of patients, we used independent t tests for continuous variables and Pearson chi-square tests for categorical variables. Next, we jointly entered independent factors found to be significantly associated with 18-month willingness to use ATA calls at the univariate level into a logistic regression model with backward selection to identify predictive factors. Then, we performed a final logistic regression model with the identified significant predictive factors and reported the odds ratio estimates and 95% confidence intervals. All analyses were conducted at 0.05 significance level (2-tailed) using IBM SPSS software, version 22.0.
ResultsSample Characteristics
Table 2 provides the characteristics of patients in the two samples used in the two analyses. The majority of patients were female, Hispanic/Latino, and preferred Spanish as their primary language. The characteristics of the two samples were not significantly different from one another. A comparison of these samples with the rest of the patients in DCAT TC excluded from the analyses did reveal significant differences in characteristics (see Tables A-1 and A-2 in Multimedia Appendix 1). Compared to the rest of DCAT TC, the two samples had a greater proportion of Hispanics/Latinos, reported a higher willingness to use ATA calls at 6 and 12 months, and had a higher ATA call completion rate. The sample for the second analysis also had lower blood sugar values, better diabetes self-care, and reported higher perceived ease of use and perceived nonintrusiveness at 6 and 12 months compared to the rest of patients in the TC arm of DCAT.
Patient characteristics for samples in the two analyses (no statistically significant difference between the two samples).
Characteristic
Sample for first analysis
Sample for second analysis
N
Statisticsa
N
Statisticsa
Female
109
72 (66.1%)
125
80 (64.0%)
Age
109
51.94 (9.01)
125
51.31 (8.81)
Hispanic/Latino
109
105 (96.3%)
125
116 (92.8%)
Spanish as preferred language
109
93 (85.0%)
125
104 (83.2%)
Married
109
49 (45.0%)
125
55 (44.0%)
PHQ-9 (range 0-27, higher=more severe depression)b,c
Sheehan disability scale (range 0-10, 0=none to 10=extremely)c
109
2.21 (2.34)
125
2.14 (2.26)
Number of ICD-9 diagnosisc,h
108
8.60 (4.50)
124
8.46 (4.46)
Number of clinic visitsc
107
10.44 (5.61)
124
10.56 (5.64)
Number of emergency room visitsc
41
1.33 (0.61)
44
1.33 (0.60)
Number of hospitalizationsc
15
1.47 (0.83)
18
1.39 (0.78)
Willingness to usec
109
4.02 (0.93)
125
4.00 (1.08)
Perceived ease of usec
109
4.05 (0.56)
125
4.12 (0.50)
Perceived usefulnessc
109
3.63 (0.89)
125
3.69 (0.90)
Perceived nonintrusivenessc
109
4.20 (0.87)
125
4.29 (0.84)
Perceived privacy/securityc
109
4.10 (1.11)
125
4.17 (1.08)
Preference of ATA call modec
109
3.82 (1.06)
125
3.58 (1.32)
Long-term perceived usefulness
76
3.71 (0.92)
125
3.74 (0.99)
ATA call completion ratec
108
0.70 (0.26)
123
0.74 (0.24)
aValues are numbers (column percentages) for categorical variables and mean (SD) for continuous variables.
bPatient Health Questionnaire, 9 items
cAssessment at 6 or 12 months. If both were available, then the average was taken.
dSymptoms CheckList, 20 items
eShort-Form Health Survey, 12 items
fBody mass index
gGlycated hemoglobin test
hInternational Classification of Diseases, 9th revision
First Analysis: Patient Acceptance of ATA Calls Over Time
Figure 1 illustrates patient acceptance of ATA calls over time. In the first 6 months of the trial, 90% (69/77) of patients reported a high willingness to use ATA calls. At 12 and 18 months, however, the proportion of patients reporting a high willingness to use ATA calls decreased to 64% (49/77) and 60% (46/77), respectively. After 6 months in the trial, 83% (62/75) of patients agreed or strongly agreed that they would prefer to receive automated calls rather than calling the ATA system at their convenience. The proportion of patients reporting this decreased to 40% (30/75) and 29% (22/75) at 12 and 18 months, respectively. Throughout the trial, most patients agreed or strongly agreed that ATA calls felt private/secure (82/108 at 6 months, 89/108 at 12 months). At 6 months, 86.2% (94/109) of patients reported that ATA calls were usually or always easy to use. This number decreased to 78.0% (85/109) at 12 months. The proportion of patients reporting that ATA calls were usually or always useful decreased from 65.1% (71/109) at 6 months to 54.1% (59/109) at 12 months. At the 18-month follow-up, 51.0% (64/125) of patients agreed or strongly agreed that the ATA calls were useful. At 6 months, most patients 87% (95/109) perceived that ATA calls were usually or always nonintrusive. More patients perceived the calls to be intrusive after 12 months in the trial as is evident from a decrease in the proportion of patients who reported otherwise 76% (83/109).
The ATA average call completion rate was 72.6% and 67.8% at 6 and 12 months, respectively—the difference between the two was not statistically significant (P=.10). In investigating the associations between the ATA call completion rate of months 0 to 12 (70.2%) and the various survey-based acceptance measures, only two measures were statistically significant: (1) perceived ease of use (Spearman correlation coefficient=0.25, P=.008) and (2) perceived nonintrusiveness (Spearman correlation coefficient=0.27, P=.004).
Patient acceptance of ATA calls over time.
Second Analysis: Factors Predicting Patient Acceptance of ATA Calls
We compared patients who reported, at 18 months, a high willingness to use ATA with patients reporting low willingness to use ATA calls to determine how the two groups differed in terms of the various sociodemographic characteristics, health conditions, health care utilization, and ATA-related measures listed in Table 2. Table 3 provides results for characteristics where there was a statistically significant difference between the two groups. See Table A-3 in Appendix for full results.
Characteristics of patients reporting high versus low willingness to use ATA calls at 18 months.
Characteristic
High willingness to use ATA calls at 18 months
Low willingness to use ATA calls at 18 months
Pb
N
Statisticsa
N
Statisticsa
Toolbert diabetes self-care in the past 7 days (range 0-7)c
74
4.81 (0.95)
51
4.43 (1.05)
.03
Willingness to usec
74
4.17 (1.00)
51
3.75 (1.16)
.04
Perceived usefulnessc
74
3.84 (0.82)
51
3.49 (0.97)
.03
Perceived nonintrusivenessc
74
4.42 (0.65)
51
4.09 (1.03)
.05
Perceived privacy/securityc
74
4.42 (0.91)
51
3.81 (1.22)
.003
Long-term perceived usefulness
74
4.07 (0.91)
51
3.25 (0.91)
<.001
aValues are numbers (column percentages) for categorical variables and mean (SD) for continuous variables.
bTwo-sample t test
cPatients’ response at 6 or 12 months. If patients provided responses at 6 and 12 months, then the average of these was used.
When we compared patients who reported a high versus a low willingness to use ATA calls at 18 months, we found six factors to be significantly associated with patients’ reported willingness to use ATA calls. Patients with a high willingness to use ATA calls at 18 months (1) had better diabetes self-care (P=.03) and (2) reported a higher willingness to use ATA calls while in the study (P=.04); they also reported (3) higher perceived usefulness (P=.03), (4) nonintrusiveness (P=.05), and (5) privacy/security (P=.003) while in the study. Moreover, patients who reported a high willingness to use ATA calls at 18 months also reported (6) higher long-term perceived usefulness (P<.001). We jointly entered the six factors into a logistic regression model with backward selection to identify predictive factors. The results revealed that two factors jointly predicted willingness to use ATA calls at 18 months: perceived privacy/security (odds ratio OR=1.59, P=.017, 95% CI [1.09, 2.33]) and long-term perceived usefulness (OR=2.77, P<.001, 95% CI [1.65, 4.63]).
DiscussionPrincipal Findings
The promises of connected health to efficiently improve access and quality of care [2], rest upon the assumption that patients will readily accept the technologies. Our study on safety-net patient acceptance of automated depression screening and monitoring using ATA calls has important findings suggesting that assumption may be questionable. In the first 6 months of the trial, most patients were accepting of ATA calls and perceived the calls to be private/secure, easy to use, useful, and nonintrusive. Over time, however, patients’ acceptance and their positive perception of ATA call characteristics decreased—although call completion rates remained high and steady. One explanation may be that since ATA call results and prompts for follow-up were sent to DMP care teams during the first 6 months of the trial and to PCPs thereafter, timely follow-up by the latter might have been challenging due to their busy practice loads. Thus, although patients continued to complete ATA calls in the second half of the trial, their PCPs may not have responded to their needs in a timely manner thereby leading them to doubt the value of ATA calls. Furthermore, patients’ acceptance and their perception of ATA call characteristics may also reflect an improvement in their depressive symptoms over time. That is, patients with improved depressive symptoms—due, possibly, to the intervention itself—may no longer perceive the benefits of the ATA calls. We investigated this hypothesis and found that there was no statistically significant difference in the percentage of patients reporting high usefulness and high willingness to use ATA calls among those with improved symptoms, no change in symptoms, or worse symptoms. It may be, however, that our sample size was not large enough to detect these differences.
Another important finding in our study was the identification of two factors that significantly predicted patients’ long-term acceptance of ATA calls: the perception that ATA calls are private/secure and the long-term perceived usefulness of ATA calls. These two factors could be potentially modified to improve patients’ willingness to use ATA calls as part of their depression care.
Limitations
This study has limitations worth noting. First, we used two different samples for the analyses. For the sample used to determine patient acceptance of ATA calls over time, we included only those patients in the DCAT TC arm who responded to ATA-related measures at each of the corresponding measurement periods. For the sample used to identify factors that predict long-term patient acceptance of ATA calls, we included only those patients in the TC arm who answered the question on willingness to use ATA calls at 18 months and at least once at 6 or 12 months. We chose to accommodate two sample sizes for our study in order to maximize the sample sizes for both analyses, although this may have introduced additional bias.
Second, although the two different samples for the analyses were not significantly different from each other, they were both somewhat different from the rest of TC patients who were excluded from the analyses because they did not answer any of the ATA-related questions. Samples used in the analyses reported a slightly higher willingness to use ATA calls at 6 and 12 months than TC patients excluded from the analyses. However, it is not likely that this limitation affected our findings because only a small percentage (about 10%) of patients excluded from the analyses refused to engage with ATA calls. Nearly 90% of them reported that they could not answer the ATA-related questions because they did not receive or did not remember receiving ATA calls, or they received calls but did not answer because they were unavailable.
Furthermore, the small sample size of 125 patients reporting on willingness to use ATA calls limits the robustness of our findings of factors predicting long-term patient acceptance of ATA calls. Future studies should validate the generalizability of our findings.
A final limitation is that in the analysis of factors predicting long-term acceptance, we defined acceptance as patients’ self-reported willingness to use ATA calls at 18 months instead of using a more objective measure such as ATA call completion rate. This may seem to be a better indicator of patient acceptance, but since we were interested in learning about patients’ long-term acceptance, we did not have the ATA call completion rate at 18 months (the intervention was only for 12 months). Moreover, in our qualitative study of DCAT TC patients with incomplete ATA calls, we discovered that patients were actually willing to take the ATA calls, but were unable to do so mainly because of nonintervention related reasons, including not being available, the ATA system having the wrong phone number, or experiencing connection issues [41]. For this reason, we assumed that if patients did not complete ATA calls, it was not due to a lack of acceptance. Therefore, given the DCAT study design and the practical reasons for patients not answering ATA calls, we chose to follow the Patient Technology Acceptance Model (PTAM) [39] and define acceptance as self-reported willingness (ie, intention) to use the technology.
Comparison With Prior Work
The finding that patients are generally accepting of ATA calls, albeit in the short term, is a promising start to our understanding of patients’ perception of such technologies. Because automated depression screening and monitoring technology is emerging, little is known about patients’ acceptance of it. Related studies of connected health technologies [42], including those focused on depression care [37,43-54], uncritically regard acceptance as patient satisfaction with care, which tells us little about why patients accept or reject the technology or how system design features affect patient acceptance. This study is significant in the connected health literature for depression care in that it utilizes measures from the literature of user acceptance of new technologies [11,55,56]. These user acceptance measures allow us to derive new knowledge that helps not only to explain why the ATA system is acceptable or not to patients, but also to understand how we may improve patient acceptance through the design of the system.
Numerous studies on connected health applications have reported a drop in technology usage over time [57-67]. Unlike these studies, we found that patients’ completion of ATA calls was high and constant throughout the trial. As mentioned above, the main reasons patients reported for incomplete ATA calls were not related to the intervention [41]. In fact, we found in an analysis not included in this paper that the survey-based measures of acceptance were not statistically significant predictors of ATA call completion rates. Nonetheless, as reported in the Results section, the ATA call completion rate was positively correlated with perceived ease of use and perceived nonintrusiveness. The significance of the former factor is in agreement with the PTAM. However, the finding that patients continued to complete ATA calls over time despite a general decrease in acceptance is surprising. Future research is needed to determine whether it was the special characteristics of the study population (ie, predominantly urban, low-income Hispanics/Latinos) or the technology design (ie, outbound calls to patients) that resulted in this finding.
The PTAM sheds light on factors that increase the likelihood that patients will be willing to use connected health technologies. Among a myriad of potential factors, the main ones predicting patient acceptance are perceived usefulness, perceived ease of use, subjective norm, and health care knowledge. Others have similarly reported that perceived usefulness and perceived ease of use are the main driving forces of patient technology acceptance [11,68,69]. Likewise, we found that long-term perceived usefulness of ATA calls significantly predicted patient acceptance of automated depression screening and monitoring. A new predictor of acceptance suggested in our analysis was patients’ perception that calls were private/secure. Future patient technology acceptance research should consider this factor in the technology design and should validate the finding.
Conclusions
In the short term, safety-net ambulatory care patients with diabetes are generally accepting of ATA calls for depression screening and monitoring. How patient acceptance can be sustained over time is an important topic for future investigation. In order to increase the odds that patients will accept ATA calls over the long term, especially for sensitive mental health conditions, the system should gauge patient perception of its privacy/security. Moreover, it is critically important that the technology not only be aligned with patients’ needs, but also be perceived as useful for them over the long term. Based on the items measuring long-term usefulness, future research should focus on designing and testing technologies that help patients be more aware of how they are feeling, remind them to take care of their health, and help them stay better connected with their health care providers.
International Classification of Diseases, 9th Revision
LACDHS
Los Angeles County Department of Health Services
PCP
primary care provider
PHQ
Patient Health Questionnaire
SC
supported care
SCL-20
Symptom Checklist, 20 items
SF-12
Short-form Health Survey, 12 items
TC
technology-facilitated care
UC
usual care
This project was supported with a grant from the US Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, grant number 1R18AE000054-01. Dr Shinyi Wu was the Principal Investigator. Ms Magaly Ramirez received support through the National Science Foundation Graduate Research Fellowship Program. We acknowledge the LACDHS clinics, providers, and patients who participated in DCAT; the research team for their contribution to this study; and 4PatientCare for assisting with the development of the ATA calls.
Jeffrey Guterman, reports a proprietary or commercial interest in the automated telephonic assessment system discussed in this article. For the remaining authors, none were declared.
Medicare ProgramOffice of the Federal Register20142015-12-30Revisions to Payment Policies under the Physician Fee Schedule, Clinical Laboratory Fee Schedule, Access to Identifiable Data for the Center for Medicare and Medicaid Innovation Models & Other Revisions to Part B for CY 2015. (CMS-1612-P)https://www.federalregister.gov/articles/2014/11/13/2014-26183/medicare-program-revisions-to-payment-policies-under-the-physician-fee-schedule-clinical-laboratoryKvedarJCoyeMJEverettWConnected health: a review of technologies and strategies to improve patient care with telemedicine and telehealthHealth Aff (Millwood)201402332194910.1377/hlthaff.2013.09922449376033/2/194DangSDimmickSKelkarGEvaluating the evidence base for the use of home telehealth remote monitoring in elderly with heart failureTelemed J E Health2009101587839610.1089/tmj.2009.002819831704AntonicelliRTestarmataPSpazzafumoLGagliardiCBiloGValentiniMOlivieriFParatiGImpact of telemonitoring at home on the management of elderly patients with congestive heart failureJ Telemed Telecare2008146300510.1258/jtt.2008.0712131877607514/6/300PolisenaJTranKCimonKHuttonBMcGillSPalmerKScottREHome telemonitoring for congestive heart failure: a systematic review and meta-analysisJ Telemed Telecare2010162687610.1258/jtt.2009.09040620008054jtt.2009.090406ClarkRAInglisSCMcAlisterFACleland John G FStewartSTelemonitoring or structured telephone support programmes for patients with chronic heart failure: systematic review and meta-analysisBMJ2007055334760094210.1136/bmj.39156.536968.5517426062bmj.39156.536968.55PMC1865411KulshreshthaAmbarKvedarJoseph CGoyalAbhinavHalpernElkan FWatsonAlice JUse of Remote Monitoring to Improve Outcomes in Patients with Heart Failure: A Pilot TrialInternational Journal of Telemedicine and Applications2010201010.1155/2010/870959DarkinsARyanPKobbRFosterLEdmonsonEWakefieldBLancasterAECare Coordination/Home Telehealth: the systematic implementation of health informatics, home telehealth, and disease management to support the care of veteran patients with chronic conditionsTelemed J E Health200812141011182610.1089/tmj.2008.002119119835VenkateshVMorrisMDavisGDavisFMIS Quart20032015-11-26User acceptance of information technology: toward a unified viewhttps://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=web&cd=7&ved=0ahUKEwiT4pK_4a7JAhXEqB4KHXjiALIQFghLMAY&url=https%3A%2F%2Fsvn.win.tue.nl%2Frepos%2Fprom%2FPapers%2FSimulationDigitalDentistry%2Fliterature%2Fuser_acceptance_of_information_technology_toward_a_unified_view_venkatesh.pdf&usg=AFQjCNE2vQibvv-UfU8eCI7It3NBXsqLWgTaylorSToddPAUnderstanding Information Technology Usage: A Test of Competing ModelsInformation Systems Research1995066214417610.1287/isre.6.2.144DavisFDUser acceptance of information technology: system characteristics, user perceptions and behavioral impactsInternational Journal of Man-Machine Studies1993338347548710.1006/imms.1993.1022PietteJDInteractive behavior change technology to support diabetes self-management: where do we stand?Diabetes Care200710301024253210.2337/dc07-104617586735dc07-1046WuSEllKGross-SchulmanSGSklaroffLMKatonWJNezuAMLeePVidyantiIChouCGutermanJJTechnology-facilitated depression care management among predominantly Latino diabetes patients within a public safety net care system: comparative effectiveness trial designContemp Clin Trials2014033723425410.1016/j.cct.2013.11.00224215775S1551-7144(13)00173-0WuSVidyantiILiuPHawkinsCRamirezMGutermanJGross-SchulmanSSklaroffLMEllKPatient-centered technological assessment and monitoring of depression for low-income patientsJ Ambul Care Manage20143721384710.1097/JAC.000000000000002724525531PMC4193786AlegríaMargaritaChatterjiPinkaWellsKennethCaoZhunChenChih-nanTakeuchiDavidJacksonJamesMengXiao-LiDisparity in depression treatment among racial and ethnic minority populations in the United StatesPsychiatr Serv200811591112647210.1176/appi.ps.59.11.12641897140259/11/1264PMC2668139SclarDARobisonLMSkaerTLEthnicity/race and the diagnosis of depression and use of antidepressants by adults in the United StatesInt Clin Psychopharmacol200803232106910.1097/YIC.0b013e3282f2b3dd1830112500004850-200803000-00006SimpsonSMKrishnanLLKunikMERuizPRacial disparities in diagnosis and treatment of depression: a literature reviewPsychiatr Q20070378131410.1007/s11126-006-9022-y17102936AreánPAUnützerJInequities in depression management in low-income, minority, and old-old adults: a matter of access to preferred treatments?J Am Geriatr Soc2003125112180891468736351569StockdaleSELagomasinoITSiddiqueJMcGuireTMirandaJRacial and ethnic disparities in detection and treatment of depression and anxiety among psychiatric and primary health care visits, 1995-2005Med Care2008074676687710.1097/MLR.0b013e31817894961858038500005650-200807000-00005PMC3956700Sánchez-LacayJALewis-FernándezRGoetzDBlancoCSalmánEDaviesSLiebowitzMOpen trial of nefazodone among Hispanics with major depression: efficacy, tolerability, and adherence issuesDepress Anxiety2001133118241138773110.1002/da.1027HeithoffKDoes the ECA underestimate the prevalence of late-life depression?J Am Geriatr Soc199501431267806734LynessJMCoxCCurryJConwellYKingDACaineEDOlder age and the underreporting of depressive symptomsJ Am Geriatr Soc199503433216217884106SireyJABruceMLAlexopoulosGSPerlickDARauePFriedmanSJMeyersBSPerceived stigma as a predictor of treatment discontinuation in young and older outpatients with depressionAm J Psychiatry20010315834798111229992EllKKatonWXieBLeePKapetanovicSGutermanJChouCCollaborative care management of major depression among low-income, predominantly Hispanic subjects with diabetes: a randomized controlled trialDiabetes Care2010043347061310.2337/dc09-171120097780dc09-1711PMC2845010CabassaLJHansenMCPalinkasLAEllKAzúcar y nervios: explanatory models and treatment experiences of Hispanics with diabetes and depressionSoc Sci Med200806661224132410.1016/j.socscimed.2008.01.05418339466S0277-9536(08)00082-8PMC2475593HuangESBrown Sydney E SZhangJXKirchhoffACSchaeferCTCasalinoLPChinMHThe cost consequences of improving diabetes care: the community health center experienceJt Comm J Qual Patient Saf2008033431384618419043PMC2366091Dwight-JohnsonMUnutzerJSherbourneCTangLWellsKBCan quality improvement programs for depression in primary care address patient preferences for treatment?Med Care2001093999344411502951MirandaJDuanNSherbourneCSchoenbaumMLagomasinoIJackson-TricheMWellsKBImproving care for minorities: can quality improvement interventions improve care and outcomes for depressed minorities? Results of a randomized, controlled trialHealth Serv Res2003043826133012785564PMC1360906OxmanTEDietrichAJWilliamsJWKroenkeKA three-component model for reengineering systems for the treatment of depression in primary carePsychosomatics20024364415012444226OxmanTEDietrichAJSchulbergHCThe depression care manager and mental health specialist as collaborators within primary careAm J Geriatr Psychiatry20031155071614506084BadamgaravEWeingartenSRHenningJMKnightKHasselbladVGanoAOfmanJJEffectiveness of disease management programs in depression: a systematic reviewAm J Psychiatry2003121601220809014638573HenkeRMMcGuireTGZaslavskyAMFordDEMeredithLSArbelaezJJClinician- and organization-level factors in the adoption of evidence-based care for depression in primary careHealth Care Manage Rev20083342899910.1097/01.HCM.0000318766.29277.491881549400004010-200810000-00003PostEPKilbourneAMBremerRWSolanoFXPincusHAReynoldsCFOrganizational factors and depression management in community-based primary care settingsImplement Sci200948410.1186/1748-5908-4-84200438381748-5908-4-84PMC2813228NuttingPAGallagherKRileyKWhiteSDickinsonWPKorsenNDietrichACare management for depression in primary care practice: findings from the RESPECT-Depression trialAnn Fam Med20086130710.1370/afm.742181953126/1/30PMC2203406U.S. Preventive Services Task ForceScreening for depression in adults: U.S. preventive services task force recommendation statementAnn Intern Med2009121151117849210.7326/0003-4819-151-11-200912010-0000619949144151/11/784BodenheimerTWagnerEHGrumbachKImproving primary care for patients with chronic illness: the chronic care model, Part 2JAMA200210162881519091412377092jip21005KroenkeKTheobaldDWuJNortonKMorrisonGCarpenterJTuWEffect of telecare management on pain and depression in patients with cancer: a randomized trialJAMA2010071430421637110.1001/jama.2010.94420628129304/2/163PMC3010214HuPChauPShengOTamKExamining the Technology Acceptance Model Using Physician Acceptance of Telemedicine TechnologyJournal of Management Information Systems19991629111210.1080/07421222.1999.11518247Or Calvin K LKarshBSevertsonDJBurkeLJBrownRLBrennanPFFactors affecting home care patients' acceptance of a web-based interactive self-management technologyJ Am Med Inform Assoc20110218151910.1136/jamia.2010.00733621131605jamia.2010.007336PMC3005875LegrisPInghamJCollerettePWhy do people use information technology? A critical review of the technology acceptance modelInformation & Management2003140319120410.1016/S0378-7206(01)00143-4VidyantiIWuBWuSLow-income minority patient engagement with automated telephonic depression assessment and impact on health outcomesQual Life Res20150524511192910.1007/s11136-014-0900-825543270PMC4412647WilliamsTLMayCREsmailALimitations of patient satisfaction studies in telehealthcare: a systematic review of the literatureTelemed J E Health20017429331610.1089/1530562015281470011886667ClarkeGReidEEubanksDO'ConnorEDeBarLLKelleherCLynchFNunleySOvercoming depression on the Internet (ODIN): a randomized controlled trial of an Internet depression skills intervention programJ Med Internet Res20021243E1410.2196/jmir.4.3.e1412554545PMC1761939ChristensenHGriffithsKMJormAFDelivering interventions for depression by using the internet: randomised controlled trialBMJ20040131328743426510.1136/bmj.37945.566632.EE14742346bmj.37945.566632.EEPMC324455AnderssonGBergströmJHolländareFCarlbringPKaldoVEkseliusLInternet-based self-help for depression: randomised controlled trialBr J Psychiatry2005111874566110.1192/bjp.187.5.45616260822187/5/456LynchDJTamburrinoMBNagelRTelephone counseling for patients with minor depression: preliminary findings in a family practice settingJ Fam Pract19970344329389071250HunkelerEMMeresmanJFHargreavesWAFiremanBBermanWHKirschAJGroebeJHurtSWBradenPGetzellMFeigenbaumPAPengTSalzerMEfficacy of nurse telehealth care and peer support in augmenting treatment of depression in primary careArch Fam Med20000898700810927707foc9085MohrDCLikoskyWBertagnolliAGoodkinDEVan Der Wende JDwyerPDickLPTelephone-administered cognitive-behavioral therapy for the treatment of depressive symptoms in multiple sclerosisJ Consult Clin Psychol2000046823566110780138MillerLWeissmanMInterpersonal psychotherapy delivered over the telephone to recurrent depressives. A pilot studyDepress Anxiety2002163114710.1002/da.1004712415535LynchDTamburrinoMNagelRSmithMKTelephone-based treatment for family practice patients with mild depressionPsychol Rep200406943 Pt 17859210.2466/pr0.94.3.785-79215217028LiebermanMAWinzelbergAGolantMWakahiroMDiMinnoMAminoffMChristineCOnline support groups for Parkinson's patients: a pilot study of effectivenessSoc Work Health Care2005422233810.1300/J010v42n02_0216390834SpekVNyklícekISmitsNCuijpersPRiperHKeyzerJPopVInternet-based cognitive behavioural therapy for subthreshold depression in people over 50 years old: a randomized controlled clinical trialPsychol Med2007123712179780610.1017/S003329170700054217466110S0033291707000542WarmerdamLvanSATwiskJRiperHCuijpersPInternet-based treatment for adults with depressive symptoms: randomized controlled trialJ Med Internet Res2008104e4410.2196/jmir.109419033149v10i4e44PMC2629364MeyerBBergerTCasparFBeeversCGAnderssonGWeissMEffectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trialJ Med Internet Res2009112e1510.2196/jmir.115119632969v11i2e15PMC2762808DavisFDPerceived Usefulness, Perceived Ease of Use, and User Acceptance of Information TechnologyMIS Quarterly19890913331910.2307/249008DillonAMorrisMUser acceptance of new information technology: theories and modelsAnnu Rev Inform Sci Technol199631332GlasgowREBolesSMMcKayHGFeilEGBarreraMThe D-Net diabetes self-management program: long-term implementation, outcomes, and generalization resultsPrev Med200304364410912649049S0091743502000567LorigKRRitterPLLaurentDDPlantKInternet-based chronic disease self-management: a randomized trialMed Care20061144119647110.1097/01.mlr.0000233678.80203.c11706312700005650-200611000-00002HeislerMPietteJD“I help you, and you help me”: facilitated telephone peer support among patients with diabetesDiabetes Educ20053168697910.1177/01457217052832471628809431/6/869FisherLCheslaCASkaffMMGillissCMullanJTBartzRJKanterRALutzCPThe family and disease management in Hispanic and European-American patients with type 2 diabetesDiabetes Care2000032332677210868849BalasEAKrishnaSKretschmerRACheekTRLobachDFBorenSAComputerized knowledge management in diabetes careMed Care200406426610211516732900005650-200406000-00013MurrayEBurnsJSeeTSLaiRNazarethIInteractive Health Communication Applications for people with chronic diseaseCochrane Database Syst Rev20054CD00427410.1002/14651858.CD004274.pub416235356YoungASChaneyEShoaiRBonnerLCohenANDoebbelingBDorrDGoldsteinMKKerrENicholPPerrinRInformation technology to support improved care for chronic illnessJ Gen Intern Med20071222 Suppl 34253010.1007/s11606-007-0303-418026812PMC2150622DorrDBonnerLMCohenANShoaiRSPerrinRChaneyEYoungASInformatics systems to promote improved care for chronic illness: a literature reviewJ Am Med Inform Assoc20071421566310.1197/jamia.M225517213491M2255PMC2213468TateDFJackvonyEHWingRREffects of Internet behavioral counseling on weight loss in adults at risk for type 2 diabetes: a randomized trialJAMA2003049289141833610.1001/jama.289.14.183312684363289/14/1833TateDFWingRRWinettRAUsing Internet technology to deliver a behavioral weight loss programJAMA200103728591172711231746joc01569GlasgowRELa Chance P AToobertDJBrownJHampsonSERiddleMCLong-term effects and costs of brief behavioural dietary intervention for patients with diabetes delivered from the medical officePatient Educ Couns199711323175849423499S0738399197000396JimisonHGormanPWoodsSNygrenPWalkerMNorrisSHershWBarriers and drivers of health information technology use for the elderly, chronically ill, and underservedEvid Rep Technol Assess (Full Rep)2008111751142219408968DavisFDBagozziRPWarshawPRUser Acceptance of Computer Technology: A Comparison of Two Theoretical ModelsManagement Science198908358982100310.1287/mnsc.35.8.982Multimedia Appendix 1