Drug Treatment Court Funding Program Evaluation

3. Methodology

The evaluation was guided by the evaluation matrix in Appendix B and included three main lines of evidence.

3.1. Document and Data Review

The document and data review included the following documents:

Profiles of each DTC were developed using the available documentation. To verify the content, each DTC was provided with its profile and changes were made, as requested.

In addition to these documents, the evaluation gathered publicly available information from DTC websites, Departmental Performance Reports, Reports on Plans and Priorities, Budget Speeches, and Speeches from the Throne.

A targeted literature review focused on the most recent research studies related to DTCs with an emphasis on meta-analyses.

3.2. Survey of DTC Stakeholders and Staff

To assess the opinions of DTC stakeholders and staff, the evaluation used a bilingual web-based survey. The survey questionnaire is in Appendix C.

The survey sample was developed by Justice Canada in collaboration with the DTCs. The total sample included 138 contacts representing the six DTCs. The survey was offered online between August 8 and September 2, 2014. Reminder emails were sent to encourage participation. A total of 65 individuals completed the survey, for a response rate of 47%.

Table 3: Survey Respondents by DTC
DTC Number of respondents Number in sample Percentage of response
Toronto 21 41 51%
Regina 11 17 65%
Edmonton 10 29 35%
Ottawa 10 21 48%
Winnipeg 8 15 53%
Vancouver 5 15 33%
Total 65 138 47%

Respondents were primarily members of the DTC team and/or members of the DTC governance or advisory committees. See Table 4 for further details.

Table 4: Types of Survey Respondents (n=65)
  Number Percentage
Members of dedicated DTC team
Treatment provider 11 17%
Judge 7 11%
DTC director 5 8%
Federal Crown/paralegal 5 8%
Provincial Crown 3 5%
Probation or police service 2 3%
Administrative 2 3%
Case manager 1 2%
Housing assistance/manager 1 2%
Duty counsel 1 2%
No response 4 6%
External service providers
Housing services 4 6%
Addictions treatment 2 3%
Other health services 2 3%
Provincial social assistance 2 3%
Employment services 1 2%
Intake 1 2%
Members of DTC governance or advisory committee 22 34%
Defence counsel 3 5%
Other 1 2%

Note: Respondents could provide more than one answer. Totals do not sum to n=65 or to 100%.

3.3. Case Studies and DTC Observations

The case studies focused on the experiences of individual DTC participants at four sites: Ottawa, Regina, Toronto and Winnipeg. The DTCs assisted with recruiting individuals for the case study interviews, which were conducted in person at the DTC offices. A copy of the case study interview guide appears in Appendix C. Each interview took approximately 30 to 45 minutes to complete. Participants had the option of ending the interview at any time.

The evaluation planned to conduct at least 12 case studies with participants at each DTC site for a total of at least 48 interviews. The intention was to divide the interviews evenly between program graduates, current participants who have substantial experience with the program (i.e., not recent entrants), and participants who did not complete the program. Current participants were included because they have more immediate reactions to the programs and can address any challenges or barriers they are presently experiencing.

Overall, 48 individuals were interviewed across the four case study sites, including 22 graduates, 18 current participants, and eight who did not complete the program (“non-completers”). The evaluation targets were generally met, although the non-completers proved to be a challenging group to reach, since they were no longer connected to the DTC and did not have a positive outcome (graduation). Demographic information of case study participants is presented in Table 5.

Table 5: Characteristics of Case Study Participants
  Total
Ethnicity
Caucasian 36
Aboriginal/Métis/Inuit 10
Other visible minorities 2
Gender
Male 32
Female 16
Age
18–24 2
25–34 14
35–44 9
45–54 18
55 and over 5
Age when began using drugsTable note *
10 and under 2
11–19 36
20–29 4
30–39 2
40–49 0
50 and over 1
Drug of choiceTable note **
Cocaine/crack cocaine 30
Opiates 12
Methamphetamines 10
Talwin & Ritalin (Ts & Rs) 1
Table note 1

Note: Three case study participants did not provide the age when they began using drugs.

Return to table note * referrer

Table note 2

Note: Some case study participants provided more than one drug of choice.

Return to table note ** referrer

Most case study participants were charged with multiple crimes before entering the program. Criminal activity self-reported by participants involved, for the most part, property-related crimes such as shoplifting, breaking and entering, theft and/or drug-related offences, mainly trafficking and possession.

The following scale has been applied to report on case study interviews.

The site visits at the four DTCs for the case study interviews also provided an opportunity to observe a pre-court and court session of the DTCs, which showed an understanding of how the DTC team worked together during court sessions. In addition, the DTC directors presented an overview of the DTC services to the evaluation team.

3.4. Recidivism Study

A study was conducted to compare the rates of re-offending of individuals who participated in a DTC program from 2006 to 2010 with two separate comparison groups: a) individuals (N=151) who met the DTC eligibility criteria but who were arrested in a jurisdiction that does not have a DTC; and b) individuals who were eligible and referred to a DTC but did not participate in the program (N=45). The DTC group was comprised of those individuals who successfully completed the program (graduates N=104) and those who were terminated from the program (released N=290).

The four recently established DTC sites (Ottawa, Winnipeg, Regina and Edmonton) provided information (DTCIS) on the adult offenders referred to the DTC, including offenders’ names, date of birth, program status and dates, gender and charges at arrest.

Evaluators reviewed court files (Halifax and Moncton) to identify the individuals for the comparison group. The criteria to select adult offenders who were charged under the CDSA were similar to the DTC eligibility criteria (i.e., offenders had committed a non-violent offense; did not use a weapon in the commission of their offense; there was no indication the individual’s offence was gang or organized crime related and there was a clear reference to the individuals drug and/or alcohol abuse/addiction/dependence).

Individuals who participated in the DTC program between 2006 and 2010, and those who went to the court between 2004 and 2010 (comparison) were included in the analysis. This timeframe allowed for a post-program follow-up time of at least three years after program participation or completion. The names of all individuals (DTC and comparison group) were submitted to the Royal Canadian Mounted Police’s Canadian Police Information Centre (CPIC) to obtain criminal history information. The CPIC records were analyzed up to September 2013. For the analysis, the time elapsed after participation in the program until receiving a criminal conviction or, in the absence of a conviction, the time to the end of the observation period, was statistically modeled. Recidivism is defined, for the purpose of this study, as a new criminal conviction after completion or participation in the DTC program.

Cox regression model was performed to analyze the data. The analysis controlled for gender, age at arrest, total number and types (i.e., offences against a person, administration of justice and drug) of prior convictions, participant’s age of first conviction, years of criminal record and whether or not individuals participated in a DTC program. Even though, the study controlled for some measures, there are however other unmeasured differences in the underlying characteristics of the DTC and comparison group members (i.e., race, employment, health issues, education, drug of choice, and housing) that the study did not account for. The major limitation of the analysis was the lack of true experimental design, as practical and ethical constraints precluded the random assignment of persons to participate.

3.5. Drug Urine Tests Analysis

Part of the requirement of the DTC program is random testing of the participants for illicit drugs. Participants must willingly submit to random and frequent urine drug testing by program staff during participation in the DTC program. The dates and the results of these urine drug tests (UDTs) were recorded in the DTCIS database regularly.

For the present study, UDTs results were extracted from the DTCIS for five sites (Edmonton, Ottawa, Toronto, Winnipeg and Regina) for the period between April 1, 2009 and March 31. 2014. The results of the UDTs were analyzed over a period of 15 months at three-month intervals. Individuals who were not accepted or admitted in the DTC, those with less than 30 days in the program and those who had no UDTs were not included in the study. The total number of the participants included in this study was 672. Only participants who had remained in the program for 15 months were ultimately included in the study (n=98) for the between-subject average effect and interactions. Although this approach introduces selection bias as earlier non-completers are not part of the study, the results align with findings in the literature that indicate that retention and length of stay may lead to better outcomes, as discussed above.

A mixed repeated measures design with one between-subject factor and two within-subject factors was used to analyze the data. The between-subject factor referred to the groups (those who successfully completed the program and those who did not complete the program) and the within-subjects factors were time in months (3, 6, 9, 12 and 15) and UDTs (dirty and clean).

3.6. Limitations and Mitigation Strategies

The evaluation encountered several methodological limitations and undertook steps to mitigate their effects.

Methodological limitations and mitigation strategies