Planning and Managing Horizontal Evaluations: Lessons Learned from the Horizontal Evaluation of the Canadian Drugs and Substances Strategy

Horizontal evaluations involving multiple organizations can often be challenging and complex, requiring close coordination with partners and careful stakeholder management. This thematic session presentation will focus on experiences from evaluators at Health Canada and the Public Health Agency of Canada in leading the Horizontal Evaluation of the Canadian Drugs and Substances Strategy (CDSS). The CDSS is the Government of Canada’s public health-based approach to addressing substance use related harms and the overdose crisis. The CDSS was also complemented by the federal government’s 2018 “Addressing the Opioid Crisis” initiative”, which received some reallocated funding from the CDSS. The horizontal evaluation examined the impact of investments and activities under both the CDSS and the Opioid initiative between 2017 and 2022, which were delivered by Health Canada and 15 other federal departments and agencies.

In this presentation, members of the evaluation team will share lessons learned on scoping and managing the evaluation of this complex, large-scale initiative implicating 16 federal departments and agencies. The presentation will start with a brief overview of the project context, as well as the considerations that led to the evaluation scope and questions. The team will then dive into the evaluation’s governance, including the implementation of an evaluation terms of reference and engagement strategy. Presenters will then highlight best practices for effective project management, such as the use of trackers and other data collection strategies to streamline the process and reduce the burden on programs and partners. The team will then conclude by presenting some of the limitations and overarching lessons learned, as well as some of the implications in advancing instrumental use in this evaluation.

Robert Tkaczyk, Evaluation Manager, Public Health Agency of Canada
Robert Tkaczyk brings almost 20 years of program evaluation experience with the federal government. He joined the Office of Audit and Evaluation at the Public Health Agency of Canada/Health Canada in 2017 as an Evaluation Manager. Robert completed a M.A. in Public Policy and Administration from Concordia University.

Stephanie Jolette, Senior Program Evaluation Analyst, Public Health Agency of Canada
Stéphanie Jolette joined the Office of Audit and Evaluation at the Public Health Agency of Canada/Health Canada in 2019 as a Program Evaluation Analyst. Prior to joining the public service, she worked for four years as a consultant where she contributed to well over 20 evaluation/research projects in areas related to justice, health, social development, and official languages. Stéphanie holds a M.A. in International Development and Globalization from the University of Ottawa.

Julie Zhang, Senior Program Evaluation Analyst, Public Health Agency of Canada
Julie Zhang is a Senior Program Evaluation Analyst at the Public Health Agency of Canada and Health Canada. She completed a Master of Public Health at the University of British Columbia and has worked on over a dozen evaluations of health programs. Julie is also a member of the CES Equity, Diversity, Inclusion and Environmental Sustainability Committee.