Initiative in Support of Access to Justice in Both Official Languages Evaluation

Appendix A: Evaluation Framework for the Initiative

Evaluation Framework for the Initiative
Evaluation Questions Indicators Data Sources
Relevance
1. How relevant is the Initiative in terms of the priorities of the Department of Justice and the federal government with respect to access to justice in both official languages?
  • Comparison between the Initiative’s expected outcomes and the Department of Justice’s strategic outcomes;
  • Comparison between the Initiative’s expected outcomes and the federal government’s official language priorities;
  • Opinion of key informants.
  • Review of relevant literature (Speech from the Throne, the Department’s policy documents, etc.);
  • Interviews;
  • Expert panels.
2. Is there a legitimate and necessary role for the federal government in the area of access to justice in both official languages?
  • Sharing of roles and responsibilities in the area of official languages;
  • Opinion of key informants.
  • Review of relevant literature (Speech from the Throne, the Department’s policy documents, etc.);
  • Interviews;
  • Expert panels.
3. To what extent do the activities undertaken through the Initiative meet the needs of justice stakeholders?
  • Change in the legislative context connected with justice in both official languages;
  • Change in the demographic context of official language communities;
  • Opinion of key informants;
  • Results from studies on access to justice in both official languages.
  • Review of writings;
  • Literature review;
  • Interviews;
  • Expert panels;
  • Training requirements study;
  • Survey among recipient organizations.
4. Is there a continued need for the activities funded by the Initiative?
  • The extent to which the training needs remain.
  • Interviews;
  • Results from the training requirements study.
Effectiveness
5. Does implementation of the training-related component align with the objectives of the Initiative and the Roadmap for Canada’s Linguistic Duality? Was it implemented effectively?
  • Nature of the link between the activities undertaken and the objectives set out in the Roadmap;
  • Degree of fit between the implementation plan and the means available to the Department;
  • Progress level in implementing the training component.
  • Literature review;
  • Interviews;
  • Expert panels.
6. How clear and transparent is the process for selecting projects funded through the Support Fund and the training component?
  • Number and format of information materials made available to eligible organizations;
  • Level of clarity in the project selection process;
  • Opinion of key informants.
  • Literature reviews;
  • Interviews;
  • Surveys among recipient organizations;
  • Case studies.
7. Do the coordination structures connected with the Initiative operate effectively?
  • Level of clarity in the roles and responsibilities of the coordination structures;
  • Level of involvement in the coordination structures;
  • Opinion of individuals involved on the coordination structures;
  • Number of meetings of the coordinating structures;
  • Participants’ satisfaction level.
  • Literature review;
  • Interviews.
8. To what extent are the data relating to performance measurement and accountability collected and incorporated into the day-to-day management of the Initiative?
  • Level of fit between the information gathered and the Department’s reporting commitments;
  • Quality level of the data gathered;
  • Level of use of the data in managing the Initiative.
  • Interviews;
  • Data analysis;
  • Literature review.
9. To what extent did the Initiative contribute to the implementation of projects that help ensure increased, ongoing access to justice services in both official languages?
  • Number and types of projects funded;
  • Level of advancement in the supply of services in both official languages within the justice system;
  • Opinion of key informants.
  • Literature review;
  • Interviews;
  • Survey among recipient organizations;
  • Expert panels;
  • Case studies.
10. To what extent has the training component contributed to the development of an academic program, recruiting tools and strategies for ensuring increased, ongoing access to justice services in both official languages?
  • Number and types of projects funded;
  • Level of advancement in developing a curriculum and access to tools and strategies;
  • Opinion of key informants.
  • Literature review;
  • Interviews;
  • Survey among recipient organizations;
  • Expert panels;
  • Case studies.
11. To what extent were the identified training needs met?
  • Percentage of the identified training needs that are addressed by the projects;
  • Nature of the outcomes achieved through the projects funded in the area of training;
  • Opinion of key informants.
  • Literature review;
  • Interviews;
  • Survey among recipient organizations;
  • Expert panels.
12. To what extent was there complementarity among the activities undertaken by the Support Fund and those undertaken by the training component?
  • Nature of the connections between the Support Fund projects and those of the training initiative;
  • Opinion of key informants.
  • Literature review
  • Interviews;
  • Survey among recipient organizations;
  • Expert panels.
13. Did the Initiative have unanticipated impacts (positive or negative)?
  • Evidence of unanticipated outcomes;
  • Opinion of key informants.
  • Literature review;
  • Interviews;
  • Survey among recipient organizations;
  • Expert panels;
  • Case studies.
Efficiency
14. How adequate were the project funding mechanisms?
  • Opinion of representatives from recipient organizations;
  • Level of Departmental consultation with recipient organizations;
  • Opinions of key informants.
  • Literature review;
  • Interviews;
  • Surveys among recipient organizations.
15. Is there a more effective approach for achieving the Initiative’s objectives?
  • Feasibility of alternative approaches;
  • Opinions of key informants.
  • Literature review;
  • Interviews;
  • Survey among recipient organizations;
  • Expert panels.