The SIRC Instrument Review Project (IRP):

A Systematic Review and Synthesis of Implementation Science Instruments

Instrumentation issues have slowed the progression of the field of D&I (Martinez, Lewis, & Weiner, 2014). SIRC’s Instrument Review Project (IRP) aims to advance implementation science through measure development and evaluation. As a first step, we are conducting an enhanced systematic review and synthesis of D&I instruments (Lewis et al., 2018). Our review centers on the implementation outcomes framework put forth by Proctor and colleagues (2011) and constructs outlined in the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009). This work is funded by NIMH R01MH106510 (Lewis, et al., 2015).

Our first round of 2014 results are available in published form: Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. 

Current state of the repository:

Since our seminal 2014 publication reporting on measures of implementation outcomes, substantial revisions have been made to our evidence-based assessment criteria. First, the rating criteria has a new name: Psychometric And Pragmatic Evidence Scale (PAPERS). The PAPERS scale can be found in our revised protocol paper (Lewis et al., 2018). Specifically, current criterion anchors were revised to add a (-1) rating to reflect poor performance, three new psychometric properties were integrated into the rating scale, and criteria for assessing pragmatic quality were incorporated.

As available, you will now have access to the following psychometric properties:

  •  1) internal consistency,
  • 2) convergent validity,
  • 3) discriminant validity,
  • 4) known-groups validity,
  • 5) predictive validity,
  • 6) concurrent validity,
  • 7) structural validity,
  • 8) responsiveness, and
  • 9) norms

and pragmatic qualities:

  • 1) cost,
  • 2) accessibility of language,
  • 3) assessor burden (training),
  • 4) assessor burden (interpretation), and
  • 5) length of each measure

included in the repository.

Currently, measure data with the new rating criteria based on our most recent work is available for the Implementation Outcomes (Proctor and colleagues (2011) 

We are in the process of updating the repository with PAPERS data for the Inner Setting constructs , in addition to adding new data for Intervention Characteristics, Outer Setting, Characteristics of Individuals, and Process constructs (Damschroder et al., 2009).

How can you access the repository?

Paid SIRC members can access a user friendly repository of the measures and their associated psychometric quality profile by following the link at the bottom of this page.

An example of the IRP webpage layout.

Our instrument repository, available only for SIRC members, is organized as follows.

There is a navigation page that includes the CFIR diagram, which organizes the constructs according to domains (e.g., outer setting, inner setting). 

Below this diagram, you can select from a list of constructs from the CFIR framework or an outcome from the Implementation Outcomes Framework. 

Selecting a construct/outcome will then take you to a construct page that includes the definition, and the list of measures with a brief summary of each (shown below).

Finally, you can select a measure to access sample items, links to related articles, the scoring guide, any supplemental material, and a graphical display of the PAPERS rating score (see below).

***CLICK HERE to enter the repository***

For more information on this SIRC initiative, please consider viewing Dr. Cara Lewis’ 2016 Research to Reality talk hosted by NCI entitled, “Advancing Implementation Science through Measure Development and Evaluation”.