Description

A randomized controlled trial (RCTs) is the most rigorous research design for evaluating the effectiveness of an intervention or policy. Although used extensively in international development, psychology, education, health care, and medicine, RCTs are infrequently conducted by program evaluators in Canada to test the effectiveness of public policy interventions. We believe that they should be used much more. What makes RCTs stand out from other types of evaluation methods is the introduction of a randomly assigned control group, enabling the comparison of the effectiveness of a new intervention against what would have happened if we had changed nothing. Using a control group eliminates numerous threats to internal validity that can greatly complicate and limit the interpretation of outcome findings. RCTs can be much cheaper and simpler to conduct than is often supposed. As well, they should not be ruled out for ethical reasons if we truly don’t know if an intervention or policy is effective. In sum, evaluators need to always consider RCTs as a potential research design because of their superiority in evaluating effectiveness. The workshop is intended to provide training on how to conduct RCTs in the public policy arena. The two presenters will draw on their experience of having conducted multiple RCTs as well as on writings in the field of evaluation on conducting RCTs (e.g., Boruch, 1997; 2005; Haynes, Service, Goldacre, & Torgerson, 2012). They will debunk some of the myths surrounding RCTs in the program evaluation world, describe the steps for designing and implementing successful RCTs, and discuss how to respond to challenges commonly encountered in conducting RCTs.

Facilitator(s)

Bob Flynn is an emeritus professor in the School of Psychology and a senior researcher at the Centre for Research on Educational and Community Services (CRECS) at the University of Ottawa. He has directed the Ontario Looking after Children project since 2000, monitoring each year the service needs and developmental outcomes of some 7,000 children and adolescents in out-of-home care. He has also completed or is currently conducting several randomized controlled trials of the effects of academic tutoring and attention training on the reading and math skills of children in care who are of primary-school age.

Tim Aubry is a Full Professor in the School of Psychology and Senior Researcher at the Centre for Research on Educational and Community Services at the University of Ottawa. He is currently holder of the Faculty of Social Sciences Research Chair in Community Mental Health and Homelessness. Over the course of his career, Tim has collaborated on research projects with community organizations and government at all levels, contributing to the development of effective social programs and policies. Tim is currently a Member of the National Research Team and the Co-Lead of the Moncton site in the multi-site Mental Health and Homelessness Demonstration Project of the Mental Health Commission of Canada. Tim teaches graduate courses in community psychology and program evaluation.