Which Treatments Does Our Agency Need and Which Do We Know Already?
Saturday 2:30 – 3:45 Breakout F1
Presentor: Michael A. Southam-Gerow
Michael A. Southam-Gerow, Virginia Commonwealth University; Selamawit Hailu, Virginia Commonwealth University; Adam Bernstein, Practicewise, LLC
With increasingly limited budgets, public mental health agencies are charged with the delivery of high-quality mental health services to a population of clients with significant and diverse impairments. Evidence-based treatments (EBTs) offer one solution to this challenge. However, which EBTs are most appropriate for an agency given their client population (i.e., problems, age, ethnicity) and given what therapists already know?
We conducted a pilot study with two goals: 1) identify some best-fit EBTs for the agency given patient population characteristics and 2) determine which are currently used by therapists.
To accomplish these goals, we analyzed data from more than 300 clients who received services at the agency in the past 5 years, mapping their (a) problem, (b) age, (c) sex, and (d) ethnicity to the literature to identify the most efficient set of programs and practices. Second, we conducted therapist surveys using the Monthly Treatment Progress Summary (e.g., Orimoto et al., 2013) with current therapists, asking them to identify the practices they used with their current clients and what outcomes they had observed.
We present findings descriptively and identify a few possible solutions to the agency’s question of which treatments would make reasonable targets for a training initiative.
Efficient Methods for Monitoring Fidelity in Routine Implementation: Lessons from the Allentown Social Emotional Learning Initiative
Saturday 2:30 – 3:45 Breakout F1
Presentor: B.K. Elizabeth Kim, Ph.D.
B.K. Elizabeth Kim,University of California, Berkeley; Valerie B. Shapiro, University of California; Berkeley, Jennifer L. Fleming, Devereux Center for Resilient Children; Paul LeBuffe, Devereux Center for Resilient Children
Monitoring implementation fidelity is essential to implementation success and intervention effectiveness. Ongoing monitoring, however, is challenging. The Promoting Alternative Thinking Strategies (PATHS) Curriculum is a school-based prevention program. Implementation guidelines suggest hiring PATHS coaches to monitor implementation fidelity of 20% of PATHS lessons (8-10 observations in grades K-2). In addition to self-monitoring of teacher fidelity, the district-wide Social Emotional Learning Initiative hired two Coaches to conduct observations in 170 classrooms across 15 schools. Coaches attempted to observe each classroom 8 times. Observations of teacher fidelity to PATHS were significantly correlated across time (e.g., r=.58-.80), but observations were not uniformly completed. 94% of classrooms were observed at Time 1. Only 35% of classrooms were observed at Time 8. Teacher fidelity to PATHS at Time 1, as observed by Coaches, was unrelated to the number of observations ultimately completed. However, the more the Coach at Time 1 perceived the teacher to be “committed to a high level of implementation of PATHS in their classroom???, the greater number of observations were ultimately completed (ES=.16). Findings suggest 1) fewer/briefer observations and 2) that teacher “buy-in???, detectable in the first month of implementation, is likely a requisite to implementation fidelity monitoring by technical assistance providers.
Characterizing the Delivery of CBT for Youth Anxiety in Community Settings
Saturday 2:30 – 3:45 Breakout F1
Presentor: Bryce D. McLeod
Bryce McLeod, Virginia Commonwealth University; Michael A. Southam-Gerow, Virginia Commonwealth University; Bruce Chorpita, UCLA; Philip Kendall, Temple University; John R. Weisz, Harvard University
Observational measures capable of assessing implementation integrity (the extent to which the components of an evidence-based treatment (EBT) are delivered as intended) of multiple EBTs are needed in implementation research. We evaluated the validity of the score interpretations for the Therapy Process Observational Coding System for Child Psychotherapy – Revised Strategies scale (TPOCS-RS) to assess implementation integrity. The TPOCS-RS includes five theory-based subscales (Cognitive, Behavioral, Psychodynamic, Client-Centered, and Family). Using the TPOCS-RS, coders independently rated 865 therapy sessions conducted with 68 children (M age = 9.93, SD = 1.79; 51.50% Caucasian) diagnosed with a primary anxiety disorder who received three different treatments (standard CBT, modular CBT, usual care) in community based mental health settings. The TPOCS-RS showed promise for assessing implementation integrity of multiple EBTs as the TPOCS-RS subscale scores, as hypothesized, discriminated between CBT and usual care delivered in community based service settings. The findings support the potential of the TPOCS-RS as a measure to assess implementation integrity.