Youth Risk/Need Assessment: An Overview of Issues and Practices
4. Research Findings (cont'd)
- 4.4 Standardization and consistency
- 4.5 Case management
- 4.6 Blending of risk and need
- 4.7 Gender and diversity
- 4.8 Interpreting and presenting the results
- 4.9 Training
- 4.10 Overrides
- 4.11 Reassessments
- 4.12 Audits
- 4.13 Community resources
4. Research Findings (cont'd)
The use of standard risk and need assessment tools allows everyone to "speak the same language". Many of our respondents favoured the use of structured risk assessment tools like the YLS/CMI or LSI-R. They believed that these tools resulted in better, more consistent decisions because they relied on a standardized set of predictive criteria, which systematically structure professional discretion. They attempt to ensure that probation officers assess the same areas in the same way. A further advantage of this practice is that it is believed to make decisions more transparent and allow for faster and more systematic transitions when someone is taking over a file or when a youth is transferred into or out of custody.
Many provinces favoured standard tools because they were seen to offer more consistency in terms of the criteria used to make an assessment and recommendations. The standardization of criteria is equated to more uniform and consistent decision making within particular offices and across regions. It also ensures that case files and reports are organized in a particular way and include a reliable set of criteria. Many believed that these tools were essential for ensuring consistency in terms of the kind of information collected for reports and used to make decisions.
The use of risk/need assessment to determine the required level of supervision (i.e., reporting) is useful from a managerial perspective because it links the amount of contact to the risk/need level. A standardized approach for determining the frequency of reporting, for example, ensures that youth of comparable risk levels are seen at comparable intervals and that lower risk youth are seen less often than those who are higher risk. This logic is consistent with the risk and need principles underpinning these tools.
According to the respondents, one of the primary strengths of the risk/need assessment is its usefulness in managing the offender. These tools provide a framework for identifying risk factors and assessing both a youth's threat to the community and need for services. They are seen as identifying red flag areas, which can then be looked at in more detail with the youth.
The risk/need assessment is often equated with best practices and seen as enhancing the role of the probation officer. The tools allow youth workers to identify criminogenic need areas where changes are required, to prioritize these criminogenic factors, and to make concrete program recommendations. The linkage of available programs to identified needs is seen as critical in terms of efficient and effective correctional management.
Despite the claims that risk/need assessments can better direct or inform treatment plans, no Canadian research has specifically explored the utility of these tools for youth case management purposes. To the best of our knowledge, no international researchers have explored this issue (Kemshall 1998; Robinson 1996). Existing research on risk/need assessments has focused almost exclusively on the prediction of recidivism, a factor significant for security management. We know very little about how or whether these tools actually inform decisions about how to intervene with a youth. The following present some initial questions that must be explored. How effective are these tools in directing treatment and program decisions? Do risk/need classifications change with appropriate treatment or programming? How does the system respond to imbalances between risk/need classifications and institutional resources? One of the objectives of risk/need classification is to reduce the levels of interventions for low- to moderate-risk offenders - have such interventions decreased, increased or remained the same? Or, are these tools merely used to dictate supervision levels? These tools are used at a number of decision-making points within the youth justice system (e.g., extrajudicial measures, pre-sentence reports, probation, and custody). Can one instrument be used to inform decisions at all stages of the criminal justice system? Or should different tools be developed specifically for use at each level of decision-making?
These questions should be explored specifically in light of the fact that some jurisdictions have considered, or in the case of Ontario, have already redesigned treatment and program services in response to risk/need assessments.
Many respondents had to pause when asked the difference between a risk and a need factor. Some could not distinguish, others claimed that risk pertained to static historic factors that could not be changed, while need referred to those dynamic factors that could be intervened in and ultimately changed to reduce the likelihood of recidivism. Some felt that need reflected something entirely different, equating it to a need for help. Regardless of the distinctions, the end result of the assessment of need was to facilitate intervention.
What most respondents failed to conceptualize are the problems intrinsic to this kind of needs assessment and how a failure to distinguish between risk and need can result in increased surveillance of youth. Most saw risk and need as
"part of the same issue". Within current research and practices of risk assessment, there is a substantial and somewhat intentional (Hannah-Moffat 2002) slippage between the concepts of risk and need. It seems that where there is an unsatisfied need there is a potential risk factor. In some cases the two are indistinguishable. Hannah-Moffat (2000:36) argues as follows:
The blending of risk and need creates an interesting paradox. It combines two quite different elements: traditional security concerns, which are generally associated with danger and the prevention of harm to others, and a more recent emphasis on need, which by contrast implies that a prisoner is lacking something and entitled to resources.
Criminogenic needs, therefore, are explicitly defined as problems, which influence the chances of recidivism, rather than a statement of resource entitlements (Hannah−Moffat 1999). In the Canadian literature, however, not all needs are seen as criminogenic. The hybrid term risk/need is often used and certain offender characteristics are identified as both risks and needs.
The embracing of third- and imminently, fourth-generation risk assessments has given rise to a new politics of need definition. Policy makers and researchers are engaged in a definitional politics that seeks to construct not only an intervenable needthat is a legitimate correctional target, but by default, also categorizes some needs as illegitimate targets or "lacking in criminogenic potential".
Recent correctional research from the British Home Office captures this dilemma. In defining the meaning of needs, Aubrey and Hough (1997) indicate,
"needs and problems are different sides of the same coin … Needs, as distinct from wishes, entitlements or rights, are defined often only implicitly, by reference to function. Basic needs, for example, are those which have to be met to stay alive… In the case of probation work as currently organized, offenders' problems reflect needs only if their resolution reduces the risk of reoffending,or brings some related advantages to the community" (p. 3).Consequently, needs are constructed within narrowly defined parameters. The definition of a need is not necessarily linked to an offender's perception of what the individual requires but rather in terms of risk reduction and intervenability.This report also poses the question,
"Should a probation officer try to address an offender's poverty, for
example, or poor housing if these are unrelated to the probationer's offending or other anti-social behaviour?" (p. 3). Like other correctional researchers, Andrews (1989) is strategic about which needs ought to be intervened in and which are less promising targets. In his earlier work on the subject, Andrews (1989) explicitly states that certain areas of need are not appropriate for intervention. He argues that treatment interventions should not attempt "to turn the client into a 'better person,' when the standards for being a 'better person' do not link with recidivism" (p. 15). In a more refined description of the needs principle, Andrews and Bonta (1998) extend the concern with empirical links to recidivism to include intervenability. Andrews and Bonta note that many offenders, especially high-risk offenders, have a variety of needs. They need places to live and work, and/or they need to stop taking drugs. Some have poor self-esteem, chronic headaches, or cavities in their
teeth. These are all needs. The needs principle draws our attention to the distinction between criminogenic and non-criminogenic needs. Needs are dynamic attributes of an offender that, when changed, are associated with changes in the probability of recidivism. Non-criminogenic needs are also dynamic and changeable, but these changes are not necessarily associated with the probability of recidivism.
Variables that are significant but not related to recidivism, yet require intervention, are deemed non-criminogenic needs (i.e., poverty, health) and considered a low priority in terms of intervention, except for humane consideration. An intervenable need is not an individual's self-perceived need, but rather is a characteristic an individual shares with a population that is statistically correlated with recidivism. An intervenable need is defined not only through the availability of resources and structural arrangements that allow for intervention and possible amelioration, but also through statistical knowledge of it as a variable that is predictive of an undesirable and preventable outcome: recidivism.
The areas of gender and diversity continue to be neglected in research and practice. The methodological limitations of these tools raise questions about their use with male youth populations. These concerns are magnified for female and non-white youth populations. In general, considerably more work is required to determine if existing tools are suitable for females and various ethno-cultural populations or if the use of these tools produces subtle forms of systemic discrimination. The research on gender, Aboriginals, and other minority groups remains insufficient. None of the tools have been adequately tested on female, Aboriginal, and other minority groups. Some studies have examined the ability of the YLS/CMI to predict recidivism with female and Aboriginal youth and reported promising results (Costigan 1999; Costigan and Rawana 1999; Jung and Rawana 1999). However, the research literature and interviews point to the potential for actuarial tools to classify female and Aboriginal offenders as higher risk because of their greater criminogenic needs. The criteria for classifying high-risk females and Aboriginal youth may be significantly different from those identifying high-risk non-native males. Female offenders are more often deemed higher risk because of their risk to themselves, whereas high-risk male offenders are more likely to pose a risk to others (Hannah-Moffat and Shaw 2001). According to a respondent, an advisory Aboriginal group in Manitoba has questioned the construct validity of these scales, particularly with female Aboriginal offenders.
Some have argued,
"achieving equal validity [for females in particular] may require the use of additional or different risk factors that are objectively and statistically demonstrated to be salient" (Brennan 1998:198). Others have noted the difficulties associated with sample sizes indicating that low base rates are a problem. Studies of adult women reveal that women have different criminal histories and that practitioners use overrides excessively to reduce risk scores, (Acoca and Austin 1996; Austin, Chan, and Elms 1993; Brennan and Austin 1997). Some have argued that the rigid or mechanical application of risk assessment tools could lead to unjust classifications and that they may in fact institutionalize the disadvantages experienced by minority groups (Bhui 1999; Daly and Lane 1999; Dawson 1999; Gottfredson 1987; Petersilia and Turner 1987).
These issues are of added significance if one considers the role risk/needs assessments play in case management. There is a significant body of literature identifying the gendered and racialized aspects of offending and differential needs and responses of women and ethno-cultural groups to correctional programs (see Hannah-Moffat and Shaw 2001). Thus far, existing risk/need research has not fully explored the possibility of gender or cultural variations in offending and recidivism. Research on these tools and policy guidelines do not appear to have examined the research on best practices for these groups and the research debates pertaining to risk/need assessment for females, Aboriginals, and other minority groups. There may be some unresolved legal questions as a result of this neglect, in particular with Aboriginal youth.
The following outlines the issues raised in interviews pertaining to these populations.
Concerning gender, some respondents believed that
"there [was] a classical double standard" in terms of how the behaviours of boys and girls are subjectively assessed by youth workers when completing assessments and in managing their cases. One respondent noted:
"girls we see on probation tend to be higher need (as a rule) - the boys tend to be higher risk (as a rule, judges come down more hard on girls in terms of longer periods of probation)". Some practitioners were concerned about the use of these tools with female youth. They believed the tools tended to over score females and this perception is linked to the random use of overrides for this population. Further, it was stated that the tests do not adequately capture the gender specific needs of females (i.e., doesn't acknowledge histories of physical, mental, or sexual abuse - often reasons for running away, parental responsibilities), which were important for case management or in terms of understanding antecedents to
crime. Some reported (based on their unsystematic observations) that young women are more likely to run away, have substance abuse problems, and come from dysfunctional families, and that young men were more likely to recidivate than females. Others felt there were no significant differences between boys and girls and that gender ought not impact risk/need assessment.
Regarding diversity, many respondents commented on the circumstances of Aboriginal youth. Many respondents indicated that Aboriginal youth were overrepresented in community and institutional populations and that this population has unique concerns. The broader socio-cultural context of Aboriginal youth and the unique issues this group presents are not adequately addressed in risk assessment tools. Some of the ethno-cultural concerns raised included the role of extended families in the lives of Aboriginal youth and the different "ways of life that exist on the reserve". One respondent explicitly indicated that it was important for probation officers and others to "avoid imposing our middle-class moral standards on the families". Some believed that risk assessment criteria included criteria that, when applied to certain Aboriginal youth, would have a discriminatory effect given the marginal social conditions in which they live.
While several provinces indicated that Aboriginal youth were overrepresented in their youth offender populations, little formal attention appears to be directed at the validity or reliability of tools and practices designed for and normed on the general youth correctional population. The failure to understand and integrate racial and ethno-cultural differences into the assessment of youth's needs and level of perceived risk and into training might have discriminatory effects. Since the systemic issues by virtue produce social marginalization, high needs or dynamic risk are magnified for some ethno-cultural youth and Aboriginal youth.
In terms of training and policy guidelines for Aboriginal and female youth, none of the respondents indicated that they had received training pertaining to the use of assessments and the interpretation of criteria for these populations. None of the provinces noted the presence of specific policy guidelines in this area.
Concerns about other ethno-cultural groups were not apparent; however, this does not mean that concerns pertaining to the assessment of such groups are non-existent. Future research ought to explore the cultural relevance and impact of these tools on non-Aboriginal minority populations.
If formal risk/need assessment tools are to be used in the provinces, these tools should be examined to determine if the criteria used in the tools to establish levels of risk and need have a discriminatory effect and if these tools adequately capture the situation of ethno-cultural and Aboriginal youth.
The legal significance of these gaps ought to be explored in terms of equality, systemic discrimination, and recent legal decisions pertaining to the governance of Aboriginal offenders.
The logic of recent risk assessments is premised on an insurance model wherein probabilistic calculations are used to determine the likelihood of an event occurring, in this case recidivism. The predicted risk of reoffending is then used to inform and justify present administrative practices and policies. The risk score does not identify who will actually reoffend; it merely identifies which youth are predisposed to recidivate. Those identified as high risk are more likely to reoffend, but not all of them will. And, those labelled high risk are not at higher risk of committing a serious violent offence. Actuarial tools predict recidivism; they do not differentiate between violent and non-violent offending behaviour. In fact, the actual offence is not used to calculate the YLS/CMI risk score; only the number of past offences is recorded. Risk/needs tools may actually be less effective in predicting serious crimes like escapes and violent behaviour. The rate of such offences is relatively low among young offenders, thereby resulting in sample sizes that are too small to evaluate.
Despite receiving training on these tools and their interpretation, practitioners tend to struggle with the meaning of the risk score and the importance of the items contained in the assessment tools. Most items contained in these tools are included because they are co-related with recidivism (see discussion of research above). Correlation is different from causation. For example, poor academic performance or family problems may be predictors of offending, but they do not cause it (for further discussion on this issue see O'Malley 1999). Rating high in a particular needs area is often tenuously repackaged as a causal factor and thus treated as such through case management planning. Interventions are designed to target the dynamic factor and to manage the factor as if it had been clearly established as an antecedent to crime. Further, the risk score is a reflection of the youth's probability of re-offending. It is calculated by examining the degree to which, or likelihood that, youth with similar traits and behaviours perpetrated crimes. The more likely that those with similar characteristics committed crimes in the past, the higher the risk score of the individual under question. Rather than interpreting the risk score as a mere correlation, individuals are often ascribed the characteristics of the risk category to which they are assigned. O'Malley (1999) points out, risks, and in this case dynamic needs, are statistical artefacts.
Proper training is essential for the proper administration of the tool. Bonta (2002:374) clearly notes:
"those who administer offender risk scales must ensure they are well trained in their administration and knowledgeable of the current issues surrounding offender assessment". He further notes that while
"few staff members will ever find themselves in court to defend their assessment, they should be prepared to do so. This requires each staff member to be ready to explain how the test is used, the research on its predictive validity, and the theory supporting the test" (p. 374). If youth workers apply the tool differently or misrepresent results, this could lead to unequal treatment. According to the developers of the tool, training should consist of at minimum a two-day session for those with knowledge of the treatment and prediction literature, otherwise five days of training was recommended. Most, but not all of those using risk assessments did receive some form of
training which ranged from one to five days.
Gaps between the purpose of these tools and their use in practice are evident. Perhaps it is more significant that practitioners have not been given clear guidelines on when to use and when not to use these tools. Many have received training on the YCJA, but few have considered its relationship to their daily tasks, such as preparing pre-sentence reports. Many practitioners understand things will change but do not know how and have not thought about risk/need assessments from this standpoint. Assessments are seen as part of case management and not situated with the broader framework of the YCJA. Thus, practitioners seem to separate concerns about producing useful and defensible assessments and the preparation from the YCJA. This may change once the act in use.
Most of the actuarial tools examined allow for a professional override such that if the youth worker determines that the classification misrepresents the actual risk posed by an offender, the final risk level can be adjusted or overridden. The authors of the LSI-based tools indicated that approximately eight to ten percent of the cases should result in an override. Data on the frequency of overrides was not readily available, but estimates ranged from none to 15 percent. Overrides appeared to occur less frequently among those using the ORAMS-PRA. Some custodial staff reported that overrides are only used to increase risk classification. Others indicated that their immediate supervisors often denied recommendations for overrides. No overrides, or relatively few, may be an indication that youth workers are not carefully scrutinizing the assessments, or it could point to institutional pressures against such practices. Several youth workers interviewed noted concerns over liability as one reason for the limited use of overrides. Correctional staff can face serious repercussions if they adjust a risk level and a youth subsequently commits a serious offence.
A review of risk classification is not a standard practice in all jurisdictions; however, several regions did indicate that they were in the process of establishing policy directives on reassessments. In an interview, Don Andrews recommended quarterly reviews. In interviews with youth workers, some reservations were raised with respect to risk review. In particular, it was noted that reassessments could potentially be skewed for offenders in custodial facilities since questions relating to, for example, peer relations or current substance abuse would be irrelevant. With respect to the ORAMS-PRA, changes in dynamic factors may not affect the risk score because of the number of static items.
A recommendation that emerged in the interviews with institutional and academic researchers was the need for risk/needs assessment auditing policy. Few jurisdictions had standardized audit practices. Field studies in at least three jurisdictions revealed that missing information and errors in the calculation of the risk score were common. While institutional supervisors are required to ensure the completion of risk assessments, not all of them consistently reviewed the information. Good quality control is essential to ensure accuracy, consistency, proper completion, and the possibility of overrides.
While the assessment of risk and needs can provide practitioners with some guidance on how to proceed with a particular case or how to manage a sentenced youth, there remains a disjuncture between the availability of resources and assessed risk/need areas. This problem is particularly acute in smaller provinces and in non-urban settings where resources for youth are more likely to exist in secure settings rather than in the community. The YCJA's emphasis on community options will likely create some systemic difficulties for practitioners in terms of the availability of suitable resources. These difficulties are likely to be most acute for higher-need youth.
-  Most practitioners accepted the tools at face value - few questioned the ability of the predictive capabilities of the tools. There is, however, some debate in the broader research literature about the predictive capacity of such tools.
-  The following is taken from a forthcoming paper by Hannah-Moffat (presented at the British Criminology Conference 2002).
-  I am not implying that all needs ought to be targets of correctional intervention or those with correctional interventions are unproblematic.
-  However, efforts are underway to develop reliable self-report instruments for the assessment of criminogenic needs (Serin and Mailloux 2001).
-  For detailed information on the statistical determination of variables as needs, see the Forum on Corrections Research September 1998, volume 10, number 3 - special issue on dynamic factors.
-  For a more detailed empirical and theoretical discussion of the some the issues pertaining to gender and diversity in risk assessment, see Hannah-Moffat and Shaw 2001 - Status of Women Canada Policy Research Report. While this report focuses on issues pertaining to federally sentenced women some of the issues will also pertain to youth. Also see the recent (2003) report of the Auditor General - Chapter 4, Women Offenders.
-  Over the past ten years, Canadian federal women's corrections (as well as other international correctional agencies) have restructured based on the premise that male and female offenders are different and that Aboriginal and non-Aboriginal offenders present different needs.
-  Not all tools can document research demonstrating these co-relations.
- Date modified: