Validation Studies for Early Learning and Care Quality Rating and Improvement Systems

(QRISs) have advanced and matured, a number of states and localities have undertaken evaluations to validate the systems. Such efforts stem from the desire to ensure that the system is designed and operating in the ways envisioned when the system was established. Given that a central component in a QRIS is the rating system, a key concern is whether the rating process, including the use of particular measures and the manner in which they are combined and cut scores are applied, produces accurate and understandable ratings that capture meaningful differences in program quality across rating levels.

The aim of this paper is to review the set of studies that seek to validate QRIS rating systems in one of several ways: by examining the relationship between program ratings and objective measures of program quality; by determining if program ratings increase over time; and by estimating the relationship between program ratings and child developmental outcomes. Specifically, we review 14 such validation studies that address one or more of these three questions. Together, these 14 studies cover 12 QRISs in 11 states or substate areas: Colorado, Florida (two counties), Indiana, Maine, Minnesota, Missouri, North Carolina, Oklahoma, Pennsylvania, Tennessee, and Virginia. In reviewing the literature, we are interested in the methods and measures they employ, as well as the empirical results.