Web-Based Cognitive Remediation Improves Supported Employment Outcomes in Severe Mental Illness: Randomized Controlled Trial

Background Finding work is a top priority for most people; however, this goal remains out of reach for the majority of individuals with a severe mental illness (SMI) who remain on benefits or are unemployed. Supported employment (SE) programs aimed at returning people with a severe mental illness to work are successful; however, they still leave a significant number of people with severe mental illness unemployed. Cognitive deficits are commonly found in SMI and are a powerful predictor of poor outcome. Fortunately, these deficits are amenable to treatment with cognitive remediation therapy (CRT) that significantly improves cognition in SMI. CRT combined with SE significantly increases the likelihood of individuals with severe mental illness obtaining and staying in work. However, the availability of CRT is limited in many settings. Objective The aim of this study was to examine whether Web-based CRT combined with a SE program can improve the rate return to work of people with severe mental illness. Methods A total of 86 people with severe mental illness (mean age 39.6 years; male: n=55) who were unemployed and who had joined a SE program were randomized to either a Web-based CRT program (CogRem) or an Internet-based control condition (WebInfo). Primary outcome measured was hours worked over 6 months post treatment. Results At 6 months, those participants randomized to CogRem had worked significantly more hours (P=.01) and had earned significantly more money (P=.03) than those participants randomized to the WebInfo control condition. No change was observed in cognition. Conclusions This study corroborates other work that has found a synergistic effect of combining CRT with a SE program and extends this to the use of Web-based CRT. The lack of any improvement in cognition obscures the mechanism by which an improved wage outcome for participants randomized to the active treatment was achieved. However, the study substantially lowers the barrier to the deployment of CRT with other psychosocial interventions for severe mental illness. Trial Registration Australian and New Zealand Clinical Trials Registry (ANZCTR) 12611000849998; http://www.anzctr.org.au/TrialSearch.aspx?searchTxt=12611000849998&isBasic=True (Archived by WebCite at http://www.webcitation.org/6sMKwpeos)


1a-i) Identify the mode of delivery in the title
Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computerbased" or "electronic" only if ooine products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms. 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Web-Based Cognitive Remediation"

1a-ii) Non-web-based components or important co-interventions in title
Mention non-web-based components or important co-interventions in title, if any (e.g., "with telephone support").

Does your paper address subitem 1b-iii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Recruitment -"...who were unemployed and who had joined a SE program...". Assessment was done in three different ways, via the web with self assessment, by the DES worker face to face and by the DES worker, totaling salaries and hours worked. This is detailed in the fuull methodology.

1b-iv) RESULTS section in abstract must contain use data
Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-iv?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "A total of 86 people with severe mental illness (mean age 39.6 years; male: n=55) who were unemployed and who had joined a SE program were randomized to either a Web-based CRT program (CogRem) or an Internet-based control condition (WebInfo)." Specific information about allocation, treatment uptake not included in abstract. Is in main body of text.

1b-v) CONCLUSIONS/DISCUSSION in abstract for negative trials
Conclusions/Discussions in abstract for negative trials: Discuss the primary outcome -if the trial is negative (primary outcome not changed), and the intervention was not used, discuss whether negative results are attributable to lack of uptake and discuss reasons.

2a-i) Problem and the type of system/solution
Describe the problem and the type of system/solution that is object of the study: intended as stand-alone intervention vs. incorporated in broader health care program? Intended for a particular patient population? Goals of the intervention, e.g., being more cost-effective to other interventions, replace or complement other solutions? (Note: Details about the intervention are provided in "Methods" under 5) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 2a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Functional recovery in people with a severe mental illness such as schizophrenia remains poor with high rates of dependence upon government benefits and significant difficulty with social isolation. An important index of a good recovery is a return to employment as this requires the individual to be able to combine motivation, cognitive performance, and the ability to relate to others. However, in the developed world few people with schizophrenia are employed."...

2 3 4 5
subitem not at all important essential Does your paper address subitem 2a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Covered in answer to 2a-i, and 5-viii, also "A number of trials have examined the combination of cognitive remediation with employment-based interventions [18][19][20][21][22][23][24][25]. Bell and colleagues combined CRT with a transitional employment program to significantly increase hours of work over 6 months [20]. In a second study over 24 months, individuals who received both CRT and a vocational intervention worked more hours and were more likely to stay in work than those who received the

2b) In INTRODUCTION: SpeciUc objectives or hypotheses
Does your paper address CONSORT subitem 2b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "The aim of this study was to combine a supported employment (SE) program with Internet-based CRT in a randomized controlled trial (RCT) to test if this could improve the employment outcomes for people with a severe mental illness in frontline services." METHODS 3a) Description of trial design (such as parallel, factorial) including allocation ratio Does your paper address CONSORT subitem 3a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "This study is an RCT of Internet-based cognitive remediation plus supported employment (CogRem) versus Internet-based information plus supported employment (WebInfo)." "Patients were randomly allocated to either CogRem or WebInfo in blocks of 8 with a 1:1 allocation ratio. This method ensures that the treatment sample sizes were equal after every batch of 8 enrollments." 3b) Important changes to methods after trial commencement (such as eligibility criteria), with reasons Does your paper address CONSORT subitem 3b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study

3b-i) Bug Uxes, Downtimes, Content Changes
Bug Uxes, Downtimes, Content Changes: ehealth systems are often dynamic systems. A description of changes to methods therefore also includes important changes made on the intervention or comparator during the trial (e.g., major bug Uxes or changes in the functionality or content) (5-iii) and other "unexpected events" that may have inpuenced study design such as staff changes, system failures/downtimes, etc. [2].

Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Participants in the CogRem group were requested to use 4 commercially available cognitive training packages-Lumosity, Brain HQ, MyBrainSolutions, and Scientific Brain Training Pro. Participants were not directed to any one exercise or website but suggested to sample and use as many as they liked. All access and costs for this were paid over a 4-month period. Access to BrainHQ was introduced after that product became available; access to Scientific Brain Training Pro ceased after the shutting down of that service. The 4a) Eligibility criteria for participants Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "All participants who were unemployed and actively seeking work via a SE program (the Disability Employment Service [DES]) were invited to take part in the study. Participants were in the age range of 17-65 years, had English language skills adequate for understanding written instructions and completing assessments, and had a diagnosis of a severe mental illness (schizophrenia, schizophreniform disorder, schizoaffective disorder, bipolar disorder, or psychotic depression). Exclusion criteria were limited to having an intellectual disability or a

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clariUed.

Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study No -earlier work (not cited) had examined levels of computer literacy in this population.

4a-ii) Open vs. closed, web-based vs. face-to-face assessments:
Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. ooine), e.g., from an open access website or from a clinic, and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email conUrmation, phone calls) were used to detect/prevent these.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-ii? * CONSORT-EHEALTH (V 1.6.1) -Submission/Publication Form https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_... 8/9/17, 7:07 PM Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Prospective participants were invited to take part in the study by their DES case manager. After reading a participant information sheet and agreeing to it, they logged on to a purpose built website and were asked to complete demographic and baseline measures." "Assessment of cognition was carried out before randomization and at 6 months using the WebNeuro, which is an Internet-based neuropsychological battery"

4a-iii) Information giving during recruitment
Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.

Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Prospective participants were invited to take part in the study by their DES case manager. After reading a participant information sheet and agreeing to it, they logged on to a purpose built website and were asked to complete demographic and baseline measures" 4b) Settings and locations where the data were collected Does your paper address CONSORT subitem 4b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "After the completion of those measures, they were randomized to one of two Internet-based programs and asked to log on twice weekly to that website either at home, their DES office, or at another rehabilitation support site. All participants had access to computers either at their DES provider or at a Clubhouse at a minimum."

4b-i) Report if outcomes were (self-)assessed through online questionnaires
Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study ""Assessment of cognition was carried out before randomization and at 6 months using the WebNeuro, which is an Internet-based neuropsychological battery" "DES staff collected information on hours worked, wages, number of jobs, and type of jobs at 6 months. This was required by the government agency funding the SE package. Other outcomes included paid work, voluntary work, or education. Function was also measured using the Role Functioning Scale [30], an observer rated

4b-ii) Report how institutional a.liations are displayed
Report how institutional atliations are displayed to potential participants [on ehealth media], as atliations with prestigious hospitals or universities may affect volunteer rates, use, and reactions with regards to an intervention.(Not a required item -describe only if this may bias results) 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study 8/9/17, 7:07 PM This is not addressed in the paper. The purpose built website displayed the logos of the University of Sydney (the academic institution of the author), the Schizophrenia Fellowship of NSW (the non-government organisation that ran the DES) and the Australian Rotary Health Foundation (the funding organisation).
"After reading a participant information sheet and agreeing to it, they logged on to a purpose built website..." 5) The interventions for each group with sutcient details to allow replication, including how and when they were actually administered

5-i) Mention names, credential, a.liations of the developers, sponsors, and owners
Mention names, credential, atliations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Conpict of interest" section or mentioned elsewhere in the manuscript).

Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Participants accessed all material via a purpose built website. This website was used to centralize all assessments, except for the cognitive testing that required a separate log-on to another website. Once randomized, participants were provided with password access to either the treatment (CogRem) or control group (WebInfo) and were sent an email with instructions. This information was also sent to the DES workers so they could assist participants if required. Participants in the CogRem group were requested to use 4 commercially available cognitive training packages-Lumosity, Brain

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.

Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study The paper did not address this.

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial. Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).

Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Access to BrainHQ was introduced after that product became available; access to Scientific Brain Training Pro ceased after the shutting down of that service." Specific dates were not given in the paper.

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.

Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Not all QA measures are described. Aspects that were covered were relevant to the primary outcomes.
"DES staff collected information on hours worked, wages, number of jobs, and type of jobs at 6 months. This was required by the government agency funding the SE package. Other outcomes included paid work, voluntary work, or education." "Participation in work is the basis for reimbursement by the 5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing ^owcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing powcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scientiUc reporting.

Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study The paper did not cover this.

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.

Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study http://www.cogrem.org.au/

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of speciUc group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-vii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "...and asked to log on twice weekly to that website either at home, their DES office, or at another rehabilitation support site. All participants had access to computers either at their DES provider or at a Clubhouse at a minimum." subitem not at all important essential Does your paper address subitem 5-viii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like 8/9/17, 7:07 PM this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Participants in the CogRem group were requested to use 4 commercially available cognitivetraining packages-Lumosity, Brain HQ, MyBrainSolutions, and Scientific Brain Training Pro. Participants were not directed to any one exercise or website but suggested to sample and use as many as they liked. All access and costs for this were paid over a 4-month period."

5-viii) Mode of delivery, features/functionalities/components of the intervention and comparator, and the theoretical framework
"The control participants (WebInfo) were able to log on through the

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.

Does your paper address subitem 5-ix?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "All participants were asked to log on through the study portal twice a week for a total of at least ten hours over a 4-month period. Adherence was monitored by the research team who also provided regular reminders to participants to log on via email. Time on the study website and choice of website was recorded; however, time on proprietary sites beyond the entry onto the study portal, that is, time on a particular game, could not be monitored."

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).

Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Participants were welcomed into the study and contacted by both telephone and email to encourage them to continue in the study. Participants also received birthday and Christmas cards. There was no direct face-to-face contact with study staff; however, the participants continued to have regular visits with their SE worker."

5-xi) Report any prompts/reminders used
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21generalizability).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Participants were welcomed into the study and contacted by both telephone and email to encourage them to continue in the study. Participants also received birthday and Christmas cards. There was no direct face-to-face contact with study staff; however, the participants continued to have regular visits with their SE worker."

5-xii) Describe any co-interventions (incl. training/support)
Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-xii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study All participants attended a supported employment style program.
"All DES workers were given a brief orientation to the trial and were encouraged to refer their clients to the study." 6a) Completely deUned pre-speciUed primary and secondary outcome measures, including how and when they were assessed Does your paper address CONSORT subitem 6a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Assessment of cognition was carried out before randomization and at 6 months using the WebNeuro, which is an Internet-based neuropsychological battery [28]. The cognitive domains tested included attention and concentration (continuing performance task-reaction time, omission, and commission error rates), working memory (digit span forward and trials correct), memory recognition (word list recognition and learning rate), information processing speed (verbal interference, choice reaction time, and switching of attention), response speed (motor tapping), and 6a-i) Online questionnaires: describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed If outcomes were obtained through online questionnaires, describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed [9].

Does your paper address subitem 6a-i?
Copy and paste relevant sections from manuscript text "Symptomatology was measured using the Behavior and Symptom Identification Scale-24 (BASIS-24) [31], a self-rated scale measuring a broad range of psychopathology and substance use developed and validated for Web use" The WHOQoL Brev has not been validated for webuse as far as I am aware

6a-ii) Describe whether and how "use" (including intensity of use/dosage) was deUned/measured/monitored
Describe whether and how "use" (including intensity of use/dosage) was deUned/measured /monitored (logins, logUle analysis, etc.). Use/adoption metrics are important process outcomes that should be reported in any ehealth trial.

Does your paper address subitem 6a-ii?
Copy and paste relevant sections from manuscript text "Adherence was monitored by the research team who also provided regular reminders to participants to log on via email. Time on the study website and choice of website was recorded"

6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained
Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).

Does your paper address subitem 6a-iii?
Copy and paste relevant sections from manuscript text No formal qualitative feedback was recorded.
6b) Any changes to trial outcomes after the trial commenced, with reasons Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study 7a) How sample size was determined NPT: When applicable, details of whether and how the clustering by care provides or centers was addressed

7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size
Describe whether and how expected attrition was taken into account when calculating the sample size.

Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "The sample size calculation (n=150) was based upon published effect sizes of combined cognitive remediation and psychosocial interventions [16]. This effect size was then significantly discounted by half to better reflect the Web-based delivery of the package."

7b) When applicable, explanation of any interim analyses and stopping guidelines
Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study CONSORT-EHEALTH (V 1.6.1) -Submission/Publication 8b) Type of randomisation; details of any restriction (such as blocking and block size) Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Patients were randomly allocated to either CogRem or WebInfo in blocks of 8 with a 1:1 allocation ratio." 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned CONSORT-EHEALTH (V 1.6.1) -Submission/Publication Form https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_... 22 of 41 8/9/17, 7:07 PM Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "The sequence generated was placed in order in sealed opaque envelopes by an independent person and were opened at the time of allocation by the research assistant. Allocation was in order of contact via the website and was concealed from the participants and their case workers who could only access the study via the website. The study coordinator had no prior knowledge of any participant at the time of allocation and was blind as to the allocation until the envelope was opened." 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Generated sequence -"An independent statistician generated the randomization sequence..." Enrolled -"Prospective participants were invited to take part in the study by their DES case manager." Assigned -"Allocation was in order of contact via the website and was concealed from the participants and their case workers who 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment

11a-i) Specify who was blinded, and who wasn't
Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 11a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Participants and DES workers were not blind to allocation nor was the research assistant coordinating the project." 11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".

Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study It is likely that many of the participants knew which was the active treatment, however the information was presented in equipoise.

11b) If relevant, description of the similarity of interventions
(this item is usually not relevant for ehealth trials as it refers to similarity of a placebo or sham intervention to a active medication/intervention) Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Participants in the CogRem group were requested to use 4 commercially available cognitive training packages-Lumosity, Brain HQ, MyBrainSolutions, and Scientific Brain Training Pro. Participants were not directed to any one exercise or website but suggested to sample and use as many as they liked. All access and costs for this were paid over a 4-month period. Access to BrainHQ was introduced after that product became available; access to Scientific Brain Training Pro ceased after the shutting down of that service. The control 12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed Does your paper address CONSORT subitem 12a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Demographic characteristics were tested using t-tests and chisquare tests. The primary outcome for the study was wages earned. Secondary outcomes included number of hours worked, number of days worked, number of hours paid, and number of jobs. These outcomes were tested using nonparametric statistics due to the distribution of results. Secondary outcomes were corrected for multiple comparisons using a Bonferroni correction. Neuropsychological test results were compared using an analysis of variance (ANOVA) design. All

12a-i) Imputation techniques to deal with attrition / missing values
Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 12a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study 8/9/17, 7:07 PM Data was not imputed.
12b) Methods for additional analyses, such as subgroup analyses and adjusted analyses Does your paper address CONSORT subitem 12b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study No further analysis subgroup analyses was done.

X26) REB/IRB Approval and Ethical Considerations
[recommended as subheading under "Methods"] (not a CONSORT item)

X26-i) Comment on ethics committee approval
1 2 3 4 5 subitem not at all important essential

Does your paper address subitem X26-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "All participants in this study received written information about the project, and written consent was obtained. The study was approved by the University of Sydney Human Research Ethics Committee (project no. 2012/1350)."

x26-ii) Outline informed consent procedures
Outline informed consent procedures e.g., if consent was obtained ooine or online (how? Checkbox, etc.?), and what information was provided (see 4a-ii). See [6] for some items to be included in informed consent documents.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem X26-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Prospective participants were invited to take part in the study by their DES case manager. After reading a participant information sheet and agreeing to it, they logged on to a purpose built website..."

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Participants were in regular contact with their DES worker and had occasional contact with study staff.

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Participants were recruited from 8 different sites. The number of participants from each site varied from 2 to 32 with the majority coming from 2 sites. Randomization remained satisfactory across all sites." 13b) For each group, losses and exclusions after randomisation, together with reasons

Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT ^ow diagram) *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study This is covered in the CONSORT diagram and also in the following text "Subjects who dropped out usually never logged on to the website and were not exposed to either the CogRem (15/43, 35%) or to the control WebInfo (19/43; 44%) site."

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other Ugures or tables demonstrating usage/dose/engagement. 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the Ugure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the 8/9/17, 7:07 PM item is not applicable/relevant for your study This was not done.
14a) Dates deUning the periods of recruitment and follow-up Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study This was not stated in the paper.

14a-i) Indicate if critical "secular events" fell into the study period
Indicate if critical "secular events" fell into the study period, e.g., signiUcant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Access to BrainHQ was introduced after that product became available; access to Scientific Brain Training Pro ceased after the shutting down of that service." Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study The trial did not reach its recruitment target.
"The trial was significantly affected by operational issues in the supported employment provider which reduced recruitment."

15) A table showing baseline demographic and clinical characteristics for each group
NPT: When applicable, a description of care providers (case volume, qualiUcation, expertise, etc.) and centers (volume) in each group Does your paper address CONSORT subitem 15? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Table 1 in the manuscript covers this area

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 15-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study eLiteracy was not specifically addressed. Availability of resources was addressed.
"All participants had access to computers either at their DES provider or at a Clubhouse at a minimum." 16) For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups

16-i) Report multiple "denominators" and provide deUnitions
Report multiple "denominators" and provide deUnitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at speciUc pre-deUned time points of interest (in absolute and relative numbers per group). Always clearly deUne "use" of the intervention.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study This is reported in part in the CONSORT diagram and in the participation section of the paper.
"Dropout was high across the trial with only 49 subjects (57.0%, 49/86) being followed up at 6 months. This did not differ significantly between arms of the study. Subjects who dropped out usually never logged on to the website and were not exposed to either the CogRem (15/43, 35%) or to the control WebInfo (19/43; 44%) site."

16-ii) Primary analysis should be intent-to-treat
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).

Does your paper address subitem 16-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Statistical approach adopted non-parametric statistics to better account for subject drop-out.
17a) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% conUdence interval) Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "A total of 23 participants returned to some work during the 6-month follow-up, though for the majority it was infrequent. On the primary outcome for the study (see Table 2) of work place involvement, the CogRem group (Md=168, n=27) worked a greater number of hours than the WebInfo group (Md=50, n=19) (U=143.5, P=.01), more of which were paid (CogRem: Md=100, n=29; WebInfo: Md=0, n=21) (U=202.5, P=.04) and earned significantly more wages. Of wages earned, there was a significant difference between the groups with the CogRem group (Md=Aus $1950, n=29) earning significantly

17a-i) Presentation of process outcomes such as metrics of use and intensity of use
In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational deUnitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is deUned (e.g., timeout after idle time) [1] (report under item 6a).

Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Those who did continue with the study were exposed to an equivalent amount of content from the study website (CogRem: median [Md]=5.5 hours, WebInfo: Md=6.9 hours; Mann Whitney U=331.5, P=.93).
Those who remained in the study were engaged with their SE case managers seeing them a median of 25 times over the 6-month period." 17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Not done.
18) Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-speciUed from exploratory Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Only specified analyses of the whole group were presented.

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).

Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Only specified analyses of the whole group were presented.

19) All important harms or unintended effects in each group
(for speciUc guidance see CONSORT for harms)

Does your paper address CONSORT subitem 19? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study No harms were reported.

19-i) Include privacy breaches, technical problems
Include privacy breaches, technical problems. This does not only include physical "harm" to participants, but also incidents such as perceived or real privacy breaches [1], technical problems, and other unexpected/unintended incidents. "Unintended effects" also includes unintended positive effects [2].

Does your paper address subitem 19-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study No privacy breaches occured.

19-ii) Include qualitative feedback from participants or observations from staff/researchers
Include qualitative feedback from participants or observations from staff/researchers, if available, on strengths and shortcomings of the application, especially if they point to unintended/unexpected effects or uses. This includes (if available) reasons for why people did or did not use the application as intended by the developers.

Does your paper address subitem 19-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study No qualitative feedback was systematically elicited in this protocol.

22)
Interpretation consistent with results, balancing beneUts and harms, and considering other relevant evidence NPT: In addition, take into account the choice of the comparator, lack of or partial blinding, and unequal expertise of care providers or centers in each group

22-i) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use)
Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 22-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "This trial supports the advantages of combining cognitive remediation therapy with supported employment in people with a severe mental illness who wish to return to work." "..participants experienced relatively small doses of CRT. Whereas participants were asked to complete 10 hours on the Web, the median exposure was 5.5 hours."

22-ii) Highlight unanswered new questions, suggest future research
Highlight unanswered new questions, suggest future research.

Does your paper address subitem 22-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "It is unclear what the necessary "dose" of CRT is, and it is possible that at least some of the benefits of CRT in engaging participants in thinking skills may be found after short courses of CRT." "The lack of any significant change in cognition might be as a result of the Web-based neurocognitive measures used. These tests may not have the same sensitivity to change as observed in face-to-face testing nor have been performed in a consistent or rigorous fashion resulting in a considerable variation of results. Motivation to engage 20) Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events.

Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study This was not discussed.

23) Registration number and name of trial registry
Does your paper address CONSORT subitem 23? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Australian and New Zealand Clinical Trial Registry no: 12611000849998 24) Where the full trial protocol can be accessed, if available Does your paper address CONSORT subitem 24? * Cite a Multimedia Appendix, other reference, or copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study On application to the corresponding author CONSORT-EHEALTH (V 1.6.1) -Submission/Publication Form https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_...