The Use of Continuous Quality Improvement Evaluations to Drive Service Improvement: The Tale of Two Projects

Saturday 8:45 – 10:00 Symposium III

Presentor: Robyn Mildon

Robyn Mildon, Parenting Research Centre; Aron Shlonsky, University of Melbourne; Catherine Wade, Parenting Research Centre; Bianca Albers, Parenting Research Centre; Cheryl Majka, Parenting Research Centre


Continuous quality improvement (CQI) evaluation methods may be one of the more promising ways to rapidly improve implementation and achieve the outcomes evidence-informed service models can help to achieve in real world service delivery settings.

This presentation will report on the results of two evaluations conducted in different areas of Australia, both applying CQI evaluation methodology and both aiming to improve outcomes for children and families currently receiving services aimed at decreasing child maltreatment but in different ways.

One will report the results of a three-year evaluation of an intensive family support service aimed at decreasing child neglect. Here the process allowed for continual use of implementation and outcome data to support joint work of stakeholders to tailor infrastructure required to support implementation. In teams, data were used to determine implementation fidelity (adherence, quality, coverage and uptake) and plan for improvements, tailor implementation support strategies to target improvements, and monitor clinic impact. How this process was done will be described and implementation and outcomes data reported.

The other will report on an evaluation of child protection practice reform framework where only a brief (3 month) CQI evaluation process could be conducted. Here data and methods for collection were applied and implementation and outcome data collected and analyzed however no structure or ongoing process was established to use data in a CQI way.

The method, processes and findings will be compared and contrasted for each project the effectiveness of the data and process to, over time, improve implementation towards achieving outcomes will be discussed.

 Implementation Efficiencies in Context:  The National Implementation of DBT in Ireland

Saturday 8:45 – 10:00 Symposium III

Presentor: Daniel Flynn

Daniel M Flynn,PhD,  Principal Psychologist, Cork Mental Health Services, Cork, Ireland; Andre Ivanoff PhD, Columbia University New York and the
Linehan Institute


The potential for identifying efficiencies in the training and implementation of DBT has grown immensely, particularly in the US and the UK.   System implementations in 17 US states and 6 international entities, along with rapid expansion in international dissemination and program development suggest that ongoing use of existing data will best inform promising implementation practices and build further efficiencies.  The 2012 Linehan Institute (LI) post hoc evaluation of teams intensively trained between 2008-2011, and the Swales et al 2012 report have been supplanted by a larger LI project with two objectives:  1) evaluating the impact of DBT Intensive training on team functioning, attitudes toward evidence-based practices and implementation abilities/barriers (n=220) and 2) the extent of DBT implementation after training completion (n=110). Since 2011, Cork Mental Health Services has been engaged in a systematic implementation of DBT, evaluating client and economic effectiveness, and also including evaluation of the implementation process.  The enhanced multi-site implementation process is described and data on the quality of implementation as measured by the PETQ (Schmidt, et al, 2011) and the experience of implementation by team leads and therapists are compared to that of the U.S. LI data.  Hypotheses informing future work and the contribution the Irish implementation makes to the DBT implementation knowledge based are discussed.


Factors Implicated in Successful Implementation: Evidence to Inform Improved Implementation from High and Low/Middle Income Countries

Saturday 8:45 – 10:00 Symposium III

Presentor: Melanie Barwick

Melanie Barwick, The Hospital for Sick Children, University of Toronto; Raluca Barac, The Hospital for Sick Children; Stanley Zlotkin, THe Hospital for Sick Children, University of Toronto; Marnie Davidson, CARE Canada; Laila Salim, Save the Children Canada


Feasible methods to inform effective and sustainable implementation of evidence in practice require knowledge about which factors are implicated in successful implementation.  Our program of research has conducted two studies to explore these factors in the context of Canadian child and youth mental health service providers (high income country) and maternal newborn child health in Ethiopia and Mali (low and middle income countries, LMIC).  We used the Consolidated Framework for Implementation Research (CFIR) to examine which of CFIR’s 37 implementation factors were most present in relation to changes (1) in exclusive breast feeding (EBF) rates in Ethiopia and Mali, from baseline to endline, and (2) in the adoption of motivational interviewing in 4 CYMH provider organizations in Ontario.  Our comprehensive analysis of CFIR factors is unique in identifying factors that are most implicated in successful implementation of evidence in two distinct contexts, which can support improved efforts moving forward.