State of the Criminal Justice System Methodology Report

• Previous page
• Table of Contents
• Next page

Monitoring the Performance of the Canadian Criminal Justice System

The monitoring of the Canadian CJS is conducted through a yearly systematic process as demonstrated in Figure 1. This involves a data collection period, followed by the analysis and reporting phase which is showcased via a State of the Criminal Justice System online Dashboard and Report. Finally, JUS ensures the maintenance of the State of the Criminal Justice Framework by conducting ongoing scanning activities and mainting collaborative partnerships with key partners to stay apprised of CJS changes to further improve/refine the Framework.

Data Collection

The information supporting the State of the Criminal Justice System Framework is collected through various administrative databases and surveys administered by federal departments and agencies (e.g., Correctional Services Canada, Justice Canada, Office of the Correctional Investigator, Parole Board of Canada, and Statistics Canada). Data are collected through publicly available reports and in collaboration with data holders at the beginning of every fiscal year. Upon receipt, data are entered in the State of the Criminal Justice System online Dashboard.

There are some data collection challenges, notably:

1. Periodicity: not all indicator data are available at the same frequency (e.g., collection may be yearly or every five years) nor at the same time of year (e.g., fiscal year vs. calendar year). Data collection for the Framework will occur over the course of the year. However, delays in entering the latest data available for some indicators may be possible. In cases where an indicator is not collected on a yearly basis, but is considered essential in measuring the performance of the CJS, the last indicator value collected is used as a proxy until the next collection cycle data becomes available (i.e., no forecasting is conducted). The collection date is clearly indicated.
2. Coverage: Although the State of the Criminal Justice System Framework is national in scope, not all jurisdictions collect the same data. Certain indicators only provides a subset of the national data. The Framework clearly identifies the data coverage for each indicator.

Reporting Vehicles: An Online Dashboard and Report

Each year, the data collected through the Framework are used to monitor the performance of the CJS and are then reported via a State of the Criminal Justice System online Dashboard and Report. The following sections further describe these products.

State of the Criminal Justice System Dashboard

The State of the Criminal Justice System online Dashboard presents data and information from the Framework, where data are available, in one easily accessible location. The Dashboard currently shows information and data collected for 41 performance indicators grouped by nine outcomes.

1. Alternate Views of the Dashboard

The default view of the Dashboard presents an overall view of performance, which includes all population data. This view is distinguishable via the identified “All Population” option noted in the “Theme Selector” bar, as well as the blue filter^{Footnote 13} applied to the Dashboard (see Image 1).

Once the user selects an outcome under the “select an outcome” column, a list of performance indicators will populate. Each indicator will present:

• A direction arrow – this presents an indication of directional change in the data using an arrow and a colour-based system. Any increase or decrease in the latest year of data triggers a change in the direction of the arrow beside the indicator title. The arrow colour shows whether performance moved in the identified direction of improvement (i.e., green) or not (i.e., red), relative to the previous cycle. For example, the crime severity indicator in Image 1 and 3 showed a slight increase from 2017 (73.61) to 2018 (75.01). This increase is presented through the direction of the arrow (arrow pointing up). The arrow is red which shows that the value of this indicator did not move in the identified direction of improvement (i.e., a decrease in the crime severity index would demonstrate improvement in community safety). A grey bar will appear in cases where no change is reported. It is important to note that changes may warrant further attention and analysis in the State of the Criminal Justice System Report.
• Year – identifies the latest year of data available for the indicator.

In addition to the overall view of performance presented above, the dashboard allows this information to be viewed by population-based themes (e.g., Indigenous Peoples). This view presents indicators only where data are available on the selected population. This view can be accessed by selecting the theme in the “Theme Selector” bar (e.g., Indigenous Peoples), and is distinguishable by the different colour filter applied to the Dashboard (e.g., purple for Indigenous Peoples; see Image 2).

A population-based theme is currently available for Indigenous Peoples. This theme presents indicators by Indigenous identity (where data are available) and provides contextual information on Indigenous individuals’ interaction with the CJS to avoid common misconceptions, such as misinterpreting higher contact rates among Indigenous individuals as a sign that they are innately more prone to criminal behavior. The population-based theme was developed with SME engagement activities which highlighted the importance of providing qualitative information. The intention is that other population-based themes (e.g., women and youth) will be developed in the future.

2. Performance Monitoring Indicator Card

The Dashboard provides an indicator card for every performance indicator included in the State of the Criminal Justice System Framework (see Image 3). The indicator card presents the data in a number of ways, including:

• The raw value score data at the national level for the five most recent years, where available.
• Different breakdowns for the indicator such as identity factor, age and sex.
• A data trend chart and table to demonstrate any changes in direction (e.g., increase, decrease, neutral), strengths, and areas for improvement.
• The various tabs at the bottom of the indicator card which provides a collection of information such as what the indicator is, why it is important, the desired direction of improvement, available breakdowns, geographic coverage, limitations, the collection source and relevant resource materials.

3. Design, Functionality and Usability

The Dashboard was designed and created in-house by JUS. Online searches were conducted to obtain ideas and inspirations on the design, functionality and usability for the Dashboard.^{Footnote 14} Usability testing was conducted internally.

JUS also contracted an Indigenous graphic designer to design elements of the Dashboard to report on Indigenous individuals’ interaction with the CJS. In addition, a Departmental Advisory Committee was established to advise on the design elements of the Indigenous Peoples theme of the Dashboard.

State of the Criminal Justice System Report

JUS will be releasing a State of the Criminal Justice System Report annually. The coverage of the report will vary based on a five year cycle. A comprehensive report will be published every five years to present key results and performance trends on all indicators and outcomes in the Framework. Thematic interim reports will be released annually between comprehensive reports. Thematic reports could explore:

• a specific outcome;
• a specific population (e.g., women, youth);
• a theme (e.g., impact of COVID-19 on the criminal justice system, advancements in CJS data collection and methodology); or,
• a government priority (Department and/or government-wide priority).

The first edition of the State of the Criminal Justice System Report (2019) presents key results on selected indicators from the Framework that were chosen based on noteworthy findings and available data. The report presents data, where available, for the five most recent years as well as some longer-term trends. The report was written in-house by JUS and peer reviewed by a performance measurement SME on contract, and relevant federal government data holders. The purpose of the report is to make a complicated and varied collection of CJS datasets and indicators easier for the public to access and understand. It is available on the JUS website.^{Footnote 15}

Maintenance and Scanning

JUS provides ongoing maintenance of the State of the Criminal Justice System Framework and Dashboard by conducting regular scanning activities which includes being aware of new data and research that could support indicator development. This allows for regular updates to the Dashboard and further refinement of the Framework.

Following the release of the Report, JUS takes a critical look at the Framework and identifies, where possible, potential research projects for the upcoming year to further develop data gaps which may be in partnership with key CJS partners. This critical review provides an opportunity for an annual post-mortem to identify where improvements can be made to the project and process.

• Previous page
• Table of Contents
• Next page