Mental well-being is fundamental to the functioning of communities and nations. However, the World Health Organization states that “[…] many people with mental health problems do not receive the treatment and care they need, despite the development of effective interventions” [1]. Matching the level of provided care to the client’s need for care is a difficult task because many factors have to be balanced simultaneously. Clients want access to the best care, but working hours of practitioners and clinicians are limited, and the interest of society is to keep care affordable. To reconcile these conflicting interests, various models of care have been proposed.

In the Netherlands, the structure of mental health care most closely resembles a stratified model, where “the initial treatment is selected based on the client’s treatment needs” [2]. The lowest level of mental health care is provided in general practices. Dutch general practitioners (GPs) are supported by mental health assistants (MHAs) who have a background in psychology, psychiatric care, or social work. MHAs are capable of treating light and or stable mental problems, and they can help to link clients to social care agencies for housing, employment, and or debt counseling. To get access to either generalist or specialist mental health care providers, clients need a referral from their GP. MHAs advise GPs on whether clients should be treated in general practices or whether they should be referred to either generalist or specialist mental health care providers. We use the term triage here to label the decision process just described.

Psychological tests and questionnaires have long been used to provide valuable information to guide mental health care interventions. In this paper, a Web-based computerized adaptive test (CAT) battery (named CATja) is described that was specifically designed to screen clients in general practices for various forms of psychopathology, thereby facilitating triage. The construction of items banks [3-5] and the derivation of parameter estimates for these item banks have been described elsewhere [6-9]. In this study, we describe the development of CATja and report first results of a pilot study where MHAs implement the tool in daily practice.

Autonomy plays a crucial role in a person’s motivation, especially for those who are mainly intrinsically motivated [10]. Adopting CATja would require the MHAs to change their working routine in many ways, and because the best way to promote change is to provide those who are supposed to change with feelings of ownership of the new situation [11], we included MHAs in the developmental process. In addition, their expertise was highly valued. We organized regular meetings where we inventoried the opinions and ideas of MHAs and where we gave specific recommendations (such as testing adaptively to tap a broad range of constructs efficiently or how to safeguard clients’ privacy). Furthermore, these meetings enabled us to judge whether our plans would be supported. An important contribution by the MHAs was that we should not focus solely on deficiencies (eg, psychopathology), but should also pay attention to clients’ strengths (eg, positive psychological constructs). In addition, MHAs had a strong preference for blended care (ie, a combination of e-assessment and face-to-face interview). Besides the scores on various dimensions, each client is uniquely characterized by a specific combination of situational and environmental factors (eg, life events, motivation to change). Information on all these characteristics that make individuals unique was preferred to be obtained in face-to-face interviews. Furthermore, because a significant proportion of clients are treated by MHAs and the relationship between therapist and client is crucial for successful treatment [12], time spent on getting this auxiliary information during personal sessions is still spent in a valuable way, because these conversations probably strengthen the relationship between client and MHA.

In CAT, items that are presented to respondents are tailored to responses given to previous items. With each consecutive item, an updated person score is derived, and the item that increases measurement precision maximally for this score is used next. This process usually continues until a predefined measurement precision is reached. In CATs, much less items are needed to derive reliable scores compared with assessments with traditional questionnaires. For an introduction to CAT, see the study by Meijer and Nering [13].

The domains of psychopathology available in the alpha version were chosen based on (1) high prevalence in the target population (anxiety and depression), (2) the explicit wish of the envisioned end users (distress), and (3) severity of functional impairment (positive and negative symptoms of psychosis). Five psychopathology domains are currently available: anxiety and depression using the Patient Reported Outcome Measures Information System (PROMIS) item pools [9], positive and negative symptoms of psychosis based on the Prodromal Questionnaire [6], and the distress scale of the Four-Dimensional Symptom Questionnaire [7]. In addition, MHAs can assess the domains companionship and emotional support, using PROMIS item pools [8]. Thus, contrary to many existing eHealth screening tools [14], CATja incorporates domains of positive psychology as well as more severe symptoms of psychopathology (eg, hallucinations), and only uses items that are appropriate for a given client because of its adaptive testing routine.

We recruited MHAs by contacting Primary Care Consultants Northern Netherlands (ELAN), an organization that advises GPs in the north of the Netherlands on eHealth advancements. Four MHAs participated in the pilot study, and they assessed 31 MHAs’ clients in total (23 females). Clients were informed that their responses would be stored anonymously for research purposes, and they provided informed consent for this by selecting the hyperlink provided in the email that was sent to them by their MHAs. On average, clients were aged 30 years and 6 months (SD 12.2). All clients had achieved a high school degree, 3 graduated in applied sciences, and 3 graduated from university. With respect to relationship status, 9 clients chose the response option “living apart together,” 11 were living together, and another 11 clients reported to be single. Moreover, 12 clients reported to be still following education, 6 were looking for work, 4 were working part-time, and 9 were working full-time.

To get a first impression on how implementing CATja would change the information available to MHAs and how their decisions concerning clients’ triage would be affected, we did the following. For each domain on which clients were to be tested, we asked MHAs to estimate clients’quartile scores before administering CATja. These estimates were compared with the quartile scores computed by CATja. In addition, we requested MHAs to appraise expected treatment levels before testing their clients with CATja and to report final treatment levels advised after testing. We compared these initial and final treatment levels. The questionnaire used can be found in Multimedia Appendix 1. For all domains and treatment level advised, we computed coefficients of agreement.