Pilot study of method to review closed organized crime files

5. CONCLUSIONS AND RECOMMENDATIONS

This chapter presents the conclusions reached on the basis of the pilot test, and our recommendations for a possible Phase 2 of this study to extend the tested method to a larger and more representative sample of files.

A. CONCLUSIONS

A key conclusion of the pilot test was that the definition of an organized crime file agreed to at the May 30th workshop could, in fact, be used by prosecutors to identify these files in the FPS caseload. According to our assessment of each file, 67 of 85 (79%) clearly met the definition. An additional 16 (19%) could be reasonably assumed to be organized crime files based on their identification as such by the prosecutors, even though no conclusive information was found in the files to indicate an organized crime link. The types of files which fell most often into this latter category involved drug couriers, marijuana grow operations and proceeds cases in which the nature of the likely links to criminal organizations was not set out in the file. Of the 85 files reviewed during the site visits, only two files (2%) were found to be 'not organized crime.' Information in these files indicated that they involved terrorism.

It is useful to consider why some files identified as organized crime files by the regional offices were not obviously so when reviewed:

Determinative information was not in the file reviewed.
Examples of this situation included files in which the link between the accused named in the file and a criminal organization was not clearly specified.
The determination required background knowledge of the larger case.
Examples of this situation included files in which all of the background information on the investigation was filed separately, for example with another related prosecution or under its own file number.
Drug couriers.
In several files, the accused were drug couriers bringing large amounts of valuable drugs into the country. While these files did not make an explicit connection between the accused and any specific criminal organization, logic would suggest that the drugs were intended for distribution by such an organization and were not for the personal use of the courier. As such, these files would arguably fit the definition.
File structures.
The absence of a standard organizational structure for the files reviewed (even within a single regional office) may have contributed to our failure to locate some types of information within these files. This issue extends to the links to other files which may have contained some of the apparently 'missing' information in the files reviewed.

The second key conclusion supported by our experience with the pilot test is that Phase 2 of the study, to involve collecting data on a larger and more representative sample of organized crime files handled by the FPS is, in fact, feasible.

B. RECOMMENDATIONS FOR PHASE 2

Our general recommendations for the conduct of Phase 2 are as follows. Phase 2 should employ a sampling approach which will provide both an estimate of the proportion of the overall FPS file caseload which meets the definition of 'organized crime file' and reliable descriptive information on these files. Our specific recommendations for the design and conduct of Phase 2 are as follows.

Compile the primary sampling frame.
Use CASEVIEW to compile a sampling frame for all FPS files and winnow the caseload down to those files which may qualify as 'organized crime files,' and which were opened and closed during the period of study which is yet to be determined. As discussed at the May 30th workshop, file-level information in CASEVIEW may be used to set aside a number of files, which are very unlikely to involve organized crime. Subject to further discussion and confirmation with FPS, such files may include those in which charges are laid (only) under statutes identified as 'not organized crime files' at the workshop. Furthermore, it may be feasible to set aside files involving particular and frequent charges under specific sections of, for example, the Income Tax Act (low level tax evasion by individual tax filers) or the Fisheries Act (minor violations of season closing dates or geographic boundaries). It will be necessary to consult with Justice lawyers who work in these areas in order to confirm the application of these 'screens' to the CASEVIEW files.
Draw the sample.
From the reduced CASEVIEW sampling frame, draw a random sample of 2,000 of the qualifying files remaining after the winnowing process. A sample of 2,000 would represent approximately 10% of all litigation files closed by FPS in a year (based on 2001/02 data). It should also yield approximately 200 organized crime files (based on the estimated incidence of 10% from the 1998 Case Complexity Study). In order to be representative by region (Atlantic, Quebec, Ontario, Prairies, B.C.), the sample should be stratified by the proportion of the files remaining in each region after the winnowing rules have been applied. Since this sample will include both organized crime files and other files, analysis of the resulting data will enable FPS to estimate the proportion of its overall caseload which meets the definition of 'organized crime files.'
Focus data collection on those characteristics of closed organized crime files which are readily available and reliable, and are related to describing the nature of organized crime files.
Based on the pilot study, the data collection form has been revised to include characteristics that were sometimes or readily available, and which, for the most part, seemed reliable. The form was also revised to balance the amount of effort and time required to track down the file characteristics with their reliability and the importance. For more detail, see Appendix B for a copy of the revised form.
Focus on the highest yield documents.
The experience of the pilot study revealed that much of the required information could be captured from a few key sources, notably, covering folders, informations/indictments, Crown/court briefs, and file indexes.
Be prepared to locate and examine linked files.
In some sites more than others, it was often necessary to examine linked files in order to capture the required information for the files in the pilot sample. This variation reflects the practice in some offices of creating individual file numbers for individual accused in a larger 'case' while in other offices, all of the accused would be covered by a single file number. In other instances, a police 'project' might have its own file number and contain background information on an investigation, which is needed to understand the sampled file.
Define the work so that it can be done by analyst/coders.
In the interests of cost-efficiency, we recommend that the file reviews in Phase 2 be done by two small teams (a supervisor and two or three analyst/coders) with qualifications needed to do the work accurately and quickly. This suggests that the work might be done by articling students or paralegals familiar with the jargon of criminal law. At a purely practical level, team members will require Secret security clearances, and will (collectively) need to be able to review files in both official languages.
Aim for an average review time of one hour per file.
A number of the foregoing recommendations would, if incorporated into the Phase 2 workplan, enable the average time per review to drop from the two hours or so spent in most of the pilot sites to something more in the range of one hour. When it is not clear from a file whether or not it should be coded as 'organized crime,' we recommend that the involved prosecutor be contacted briefly to clarify this point.
Establish a reasonable standard for productivity.
The actual work of reviewing closed organized crime files is physically and mentally fatiguing. This suggests that the work schedule for the analyst/coders should incorporate completion targets which reflect a balance between speed and accuracy, and the need for breaks during the workday. Assuming a review of 2200 files at 30 files per week per reviewer, with eight reviewers (two teams of three plus two supervisors) the review would span approximately ten weeks. We suggest 30 files per week per reviewer/supervisor on the assumption that many files selected in the random pull will be short and straightforward. Conversely, review of the 200 organized crime files flagged by the prosecutors (see last bullet below) will likely require more time per file due to their anticipated size and complexity.
Plan carefully for the visits to the regional offices.
Our experience with the Phase 1 site visits revealed several logistical matters, which should be taken into account when planning the more extensive visits in Phase 2. Included here are ensuring that the timing of the visits suits the regional office in terms of key staff and workspace availability, and requesting indexes for all sampled files (as available) to limit the boxes/pockets requested to those most likely to contain the required information. Regional offices should be provided a list of files to pull in advance of the actual site visits.
Conduct the review of the sampled files.
Review the files in the sample of 2,000 using the attached coding instrument (Appendix B).
Analysis.
Analyze the coded information to both classify the files reviewed as 'organized crime files' or not, according to the definition agreed to at the workshop. Describe the typical and distinguishing characteristics of these two sets of files.
Use second sample of organized crime files flagged by prosecutors.
As a supplement to the analysis of the data from the random sample of 2,000 files, a parallel coding and analysis process should be applied to closed organized crime files currently being flagged by the regional offices. This flagging process is likely to generate a larger sample of organized crime files than will naturally 'fall out' of the random sample, and will support an assessment of the extent to which the prosecutors' lists both incorporate the files in the random sample, and are similar in terms of their descriptive characteristics. Given the expectation that only about 10% of the 2000 files drawn in the random sample will qualify as organized crime files, a thorough analysis of a sample of files selected by the regional offices as organized crime will allow for a more comprehensive understanding of the nature of these files. The specific number of prosecutor-flagged files to be reviewed will depend on how many are identified, and in which regions they are located. However, for planning purposes, we recommend that a regionally proportionate sample of 200 flagged files be reviewed. Where a prosecutor-flagged file turns up in the random sample, it would be replaced by another file in that regional office.

C. FINAL THOUGHTS

Overall, Phase 1 demonstrated the feasibility of conducting retrospective reviews of closed FPS organized crime files and offers some concrete recommendations for a more comprehensive review, should one be undertaken. That said, it is clear that any review of FPS files, which vary so much among regional offices and, in many cases, within a given office, must be seen as a challenging endeavour that can only hope to provide partial insight into the nature of the entire caseload. Improved file management practices, as well as wider and more consistent use of CASEVIEW, particularly in flagging organized crime files would greatly assist the Department in its future efforts to develop policy and evaluate programs concerned with combating organized crime.