Victims of Crime Research Digest, Issue No. 4

The 2009 General Social Survey on Victimization in the Territories: Lessons Learned

This article is adapted from the report 2009 General Social Survey on Victimization in Yukon, Northwest Territories and Nunavut, Collection and Evaluation Report.

The Statistics Canada General Social Survey (GSS) program, established in 1985, conducts telephone surveys across the ten provinces. The GSS is recognized for its regular collection of cross-sectional data that allows for trend analysis and for its capacity to provide information on specific social policy issues of current or emerging interest.

In 2009, Statistics Canada conducted the victimization cycle of the GSS for the fifth time. The purpose of the survey is to collect information on the nature and extent of criminal victimization as provided by Canadians. In addition, it examines risk factors associated with victimization, reporting rates to the police, and how Canadians perceive crime and the criminal justice system.

The main sample of the 2009 survey was distributed over the ten provinces and a supplementary survey was conducted in the three territories during the fall of 2009. This article provides an overview of the collection strategy in the North, the results of the data quality evaluation, and recommendations on the uses and limitations of the data.

Introduction

For many years, territorial governments have emphasized the importance of including their populations in the General Social Survey (GSS) cycles on self-reported victimization. The territories have historically been limited to official police and court statistics to inform policy decisions related to justice issues. Beyond these sources, there has been little information about the scope and nature of victimization in the North.

Due to the challenges associated with conducting surveys in the North, previous efforts to collect victimization data have yielded modest results. In order to improve the quality of the 2009 GSS data for the territories, innovative methods were developed which included using multiple surveys for the sample files and some face-to-face interviews where telephone coverage was poor.

The development and implementation of the strategy involved input from a wide variety of internal and external partners who provided advice and guidance on methodological issues, content development, collection planning, and monitoring.

Background

Given the territorial governments' high priority need to understand victimization in the territories, efforts were made in previous victimization cycles to pilot the collection of data in the territories. The results were mixed. Collection in Yukon, Nunavut, and the Northwest Territories poses unique challenges, and as a result, obtaining a representative sample in each of the territories is more difficult than in other parts of Canada. Some of these challenges include

In 1999, the GSS collected its first pilot test data in the territories using the Random-Digit Dialing (RDD) method. This is the method which is used to select the GSS sample in the provinces. Following a detailed evaluation of the results, it was recommended that data from this northern pilot test not be released because of bias caused by substantial undercoverage.Footnote 1

A second pilot test was conducted in 2004. Collection was again done by telephone, but to improve the coverage of the survey, the sample was selected from respondents to the 2003 Canadian Community Health Survey. Although the data were releasable with cautionary notes, the global response rates in the territories were lower in each of the territories than the provincial results. In addition, Aboriginal people and individuals living in more remote communities were underrepresented in the sample.Footnote 2

In preparation for the 2009 GSS on victimization, the topic of data collection in the territories was revisited in consultation with focal pointsFootnote 3 from the territorial statistical agencies, Statistics Canada methodologists, subject matter experts, survey operations specialists, and regional offices. It was decided that, based on the experience of previous attempts in the North, further efforts to improve the quality of the data in the territories were required. This would be done through a multi-source listing strategy and some face-to-face interviews.

Overview of Data Collection Strategy

Data collection efforts in the North in 2004 and 1999 revealed that telephone interviews were not sufficient to produce reliable estimates. Telephone penetration rates were lower than in the provinces, particularly in Nunavut, where at least 20% of households did not have a regular land line in 2005.

As RDD had not worked well in the past, it was felt that an area frameFootnote 4 would be a better approach to obtaining adequate coverage in combination with an effort to reach households with no telephones. Survey results are prone to bias if this is not done. Furthermore, in addition to telephone interviews, some personal interviews would also be conducted. There was a strong recommendation from Inuit associations that personal interviewing was more appropriate than telephone interviewing for the North. This had also been the experience of the post-censal surveys.

The 2009 strategy for data collection in the territories, therefore, consisted of telephone interviews in areas with acceptable telephone coverage and face-to-face or personal interviews in other areas. In order to support the alternative approach for the data collection, funds were obtained from Statistics Canada, the Department of Justice Canada, and the Policy Research Data Group (PRDG).

Target Population

The target population for the GSS Cycle in the North consists of all residents of the territories aged 15 years and over who are not living in institutions. The survey targeted households, and after the completion of the roster, a single eligible member of each sampled household was randomly selected by the application to complete the questionnaire.

Sample Size

The GSS budget normally provides for approximately 25,000 respondents (completed interviews) in the provinces. For the 2009 GSS, resources for 1,500 respondents (completed interviews) were reallocated from the provinces to the territories. The target of 1,500 was based on results from the previous cycles and with the objective of obtaining good estimates of victimization rates for each territory.

GSS methodologists together with the survey team planned for 80% of the interviews to be carried out by telephone and the remaining 20% to be conducted face-to-face. The latter were planned for communities where telephone coverage was poor and where previous attempts had identified undercoverage of certain subpopulation groups.

Sample Selection

The sample for the victimization survey in the territories was drawn from an area frameof households which had completed the 2007-2008 Canadian Community Health Survey (CCHS) or the 2007-2009 Labour Force Survey (LFS). This decision took into consideration the results of the 1999 Victimization pilot survey in the territories, the 2004 GSS on victimization in the North, and the findings in the report A proposed territorial strategy for households prepared by the Statistics Canada Joint Federal-Territorial Working Group. It was noted in the latter that RDD had not yielded acceptable results, but that the results from second phase surveys by telephone had been positive. The sample unit was the dwelling, as it was for the two source surveys.

It should be noted that due to operational difficulties inherent to remote locales, only the ten largest communities in Nunavut are covered by the CCHS. As the GSS sample was drawn in part from the CCHS, it was also limited to the same communities.

Personal and Telephone Interviews

The most significant change in the 2009 GSS in the territories was the inclusion of personal interviews in addition to telephone interviews. The decision to include personal interviews was based on the recommendation in the report referred to above that personal interviews are the preferred mode of collection in the territories. This decision was also based on the 2004 collection experience.

Input from Partners

Due to the new approach that was being developed for the survey in the North, in addition to the GSS victimization survey team, input and guidance was sought from within and outside of Statistics Canada throughout the development, collection, and production phases. Input from outside Statistics Canada was provided by key federal government policy departments, the Territorial Statistical Focal Points, and justice system stakeholders from the Northwest Territories, Yukon and Nunavut. All were involved in consultations on content development throughout the survey development period.

Collection Time Period

The period between September and December 2009 was identified by the Territorial Focal Points as the best time for collection in the North. Due to weather conditions, it was felt that January and February would not be good months for personal interviews, and given that many potential respondents would be out on the land, the April to June period was also not considered optimal. Collection was therefore conducted from August 31 to December 31, 2009.

Recommendations

The major recommendations from the qualitative analysis of the collection strategy include the following:

Quantitative Analysis of the 2009 GSS in the North

Quantitative analysis of the collection approach, data quality measures, and the sample representativity was carried out. The parameters and data quality measures which were examined included the distribution of telephone and personal interviews, response, non-response and slippage rates, adjustment of weights to improve representativity and calibration. The response rates for the 2009 and 2004 surveys are shown in Table 1.

Table 1: 2004 and 2009 GSS North Response Rates
Territory Responses Response rate (%)
2004 2009 2004 2009
Yukon 450 355 66.1 54.4
NWT 595 536 62.8 54.7
Nunavut* 245 203 44.8 38.7
All 1290 1094 59.3 50.7

* Nunavut's ten largest communities

The following are the findings and recommendations with respect to the data quality and on the uses and limitations of the data.

Findings:

Recommendations:

The 2009 GSS on victimization was conducted in Yukon, the Northwest Territories, and Nunavut using a combination of telephone and face-to-face interviews. Compared to other areas in Canada, conducting surveys in the territories poses unique challenges, including incomplete telephone service, language difficulties, high population mobility, and the remoteness of many communities.

Collection in Nunavut's ten largest communities was particularly challenging and resulted in undercoverage of the Inuit population and lower response rates than those in Yukon and the Northwest Territories. As a result, results from Nunavut should be used with caution.

Conclusion

While many of the data collection challenges outlined in this report will continue to exist in Canada's territories, it is hoped that the lessons learned through the experience of the collection of the 2009 GSS will be applied to future victimization surveys and other household surveys in the North.