Evaluation of the International Legal Programs Section

Appendix B: Evaluation Matrix

Issues/Questions Indicators Data Sources Responsibility for Collecting
Relevance

To what extent are the activities of ILPS aligned with the federal priorities and strategic outcomes of the Department of Justice?

  • Comparison of ILPS description of activities to stated federal priorities in the area of international development and strategic outcomes
  • Alignment of ILPS objectives/activities with federal priorities
  • Alignment of ILPS objectives/activities with the Department's strategic outcomes
  • Key informant interviews
  • Document review
  • JUS Evaluation Division
  • ILPS

To what extent are the activities of ILPS aligned with federal roles and responsibilities? Is there a legitimate and necessary role for the federal government in providing international legal assistance?

  • Alignment of ILPS objectives/activities with federal government's roles and responsibilities
  • Continued relevance of federal government involvement in international development activities for the justice sector in other countries
  • Document review
  • Key informant interviews
  • JUS Evaluation Division
  • ILPS

Is there a continued need for ILPS?

  • Continued need for strategic advice and outreach function
  • Continued need for legal technical assistance in countries in transition assisted by ILPS
  • Impact of research, strategic partnerships and advice on international Justice Sector developmental matters
  • Benefits of JUS participation in international meetings
  • Document review
  • Key informant interviews
  • JUS Evaluation Division
  • ILPS
Design

Are the mandate and objectives of ILPS clear?

  • Clarity of stated mandate and objectives of ILPS
  • Awareness of the stated mandate and objectives of ILPS among client agencies
  • Document review
  • Key informant interviews
  • JUS Evaluation Division
  • ILPS

Is the ILPS governance structure appropriate?

Clarity of ILPS’ roles and responsibilities

  • Document review
  • Key informant interviews
  • JUS Evaluation Division
  • ILPS

Does ILPS have in place appropriate methods/systems for monitoring performance and reporting on outcomes?

  • Standardized methods for monitoring performance and reporting on outcomes
  • Appropriateness of these methods
  • Document review
  • Key informant interviews
  • JUS Evaluation Division
  • ILPS
Performance – Achievement of Expected Outcomes

To what extent has ILPS achieved its expected outcomes?

  • Number and nature of projects
  • Extent to which knowledge was enhanced in the Canadian federal system of current and emerging international justice sector development matters
  • Extent to which knowledge of Canadian justice system principles, structures, processes and experiences transferred to recipient countries to assist them in strengthening their justice system
  • Extent to which capacity of Canada was enhanced to participate strategically in foreign policy and international development matters
  • Evidence of enhanced capacity of recipient countries to deliver fair and accessible justice
  • Extent to which ILPS has contributed to strengthen rule of law and improve systems of justice internationally in furtherance of Justice Canada and Government of Canada priorities and foreign policy objectives
  • Document review
  • Project semi-annual reports to GAC
  • Project annual reports to GAC
  • project performance reports
  • Strategic advice
  • Research papers
  • Presentations/briefing material
  • Key informant interviews
  • Case studies
  • JUS Evaluation Division
  • ILPS

What factors are contributing to or constraining the achievement of expected outcomes?

  • Identified best practices and challenges
  • Nature of factors contributing to or constraining success
  • Document review
  • Project semi-annual reports to GAC
  • Project annual reports to GAC
  • Key informant interviews
  • Case studies
  • JUS Evaluation Division
  • ILPS

To what extent have partnerships supported the achievement of ILPS outcomes?

  • Number and nature of partnerships
  • Contribution of partnerships to achieving identified outcomes
  • Perception of the contribution of partnerships to achieving the identified outcomes
  • Document review
  • Key informant interviews
  • Case studies
  • JUS Evaluation Division
  • ILPS

Have ILPS activities led to any unintended or unanticipated impacts?

Instances of unintended impacts and their effects

  • Document review
  • Key informant interviews
  • JUS Evaluation Division
  • ILPS
Efficiency and Economy

Are there more appropriate and efficient means of achieving the expected results of ILPS activities?

  • Alternative management, accountability and reporting methods
  • Identified resource challenges
  • Appropriateness of ILPS organizational structure to support its outcomes
  • Level of funding
  • Key informant interviews
  • Document review
  • JUS Evaluation Division
  • ILPS

How could the efficiency and economy of ILPS be improved?

  • Extent to which ILPS outcomes could be achieved for less money
  • Identified areas for improvement in efficiency and effectiveness of ILPS performance

Key informant interviews

  • JUS Evaluation Division
  • ILPS

Are other organizations within or outside of government better placed to deliver these programs and services?

Other organizations within or outside of government that are engaged in international development activities for the justice sector in other countries. Degree to which JUS activities complement/duplicate these organizations' activities

  • Document review
  • Key informant interviews
  • JUS Evaluation Division
  • ILPS