ASPE Research Brief
By: Tamara G. Halle, Elizabeth C. Hair, Margaret Buchinal, Rachel Anderson, and Martha Zaslow
Printer friendly version in PDF format (8 pages)
Contents
Introduction
What do children need to know and be able to do at the start of school to be “in the running” for long-term success? This is a question for which educators and policy makers would love to have a definitive answer. Yet, the answer remains elusive. There is growing state and national interest in promoting early learning and development, as noted by the expansion of state-funded pre- kindergarten programs and the reauthorization of Head Start which includes explicit requirements for ongoing child assessment and screening. Related initiatives include the development of state early learning guidelines and kindergarten readiness assessments aimed at articulating and evaluating the set of skills and competencies young children need to prepare them for the increased challenges and demands of kindergarten and to succeed in later schooling (Scott-Little, Lesko, Martella, & Milbum, 2007; Stedron & Berger, 2010). Of particular interest is raising low-income children’s school readiness skills to the level that they can optimally benefit from subsequent schooling. Currently, there is little research that identifies specific levels of school readiness children need to achieve in order to put them on a successful trajectory.
The translation of school readiness goals into state standards or core competences implies that there is a level of skill that children need to achieve prior to entering kindergarten in order to thrive within an academic setting, i.e., a point at which they are “ready” (or ready enough). Indeed, many states have attempted to articulate such thresholds for particular skills and abilities within early learning guidelines, albeit without much research evidence to support the particular criteria they adopt. The purpose of this brief is to explore the concept of thresholds of school readiness and to share findings from a recently completed project supported by the Assistant Secretary for Planning and Evaluation within the U.S. Department of Health and Human Services that sought to identify thresholds of readiness within a secondary data analysis project. (See side bar for an overview of the project.)
What are Thresholds?
The In the Running project explored three ways of testing thresholds. The first two approaches examined school-entry language, pre-academic, and social skills individually. First, nonlinear analyses explored whether there was a cut point in the prediction of school outcomes from school readiness skills. If so, that cut point might be regarded as a threshold, especially if children above the cut point showed faster acquisition of academic or social skills over the school-age years (i.e., that achieving a certain level of performance within one or more skill domain serves as a “springboard” for growth once this threshold is reached). Second, other analyses created a cut point based on existing criteria and asked if children above and below the selected cut point differed in terms of their school-age outcomes. The last approach involved asking whether different combinations of school readiness skills might serve as a springboard for acquisition of academic and social skills over the school-age years.
[ Go to Contents ]
Methodology
The data
The data sets utilized for secondary analysis were the National Institute of Child Health and Human Development’s Study of Early Child Care and Youth Development (NICHD SECCYD) and the Early Childhood Longitudinal Study — Kindergarten Class of 1998-99 (ECLS-K).
The NICHD SECCYD is a comprehensive longitudinal study initiated by The National Institute of Child Health and Human Development (NICHD) in 1989 to answer many questions about the relationship between child care experiences and children's developmental outcomes. Conducted by a network of investigators, the study followed a diverse sample of about 1,400 children and their families from birth through adolescence. NICHD SECCYD data includes longitudinal direct assessments of children’s language, attention, and academic skills and parent or teacher ratings of social skills collected from 2 to 15 years of age.
The ECLS-K data provide information on a nationally-representative sample of approximately 20,000 children who were in kindergarten in fall of 1998 and whose development and experiences were followed from kindergarten through eighth grade. ECLS-K data include longitudinal direct child assessments of academic skills and teacher ratings of attention and social skills from fall of the kindergarten year through the spring of eighth grade. For these analyses, the sample was restricted to children who were first-time kindergartners and who had completed the direct assessments in English.
To the extent possible, comparable measures were chosen across the two datasets for school readiness skills, and academic and social outcomes. The school readiness variables included: direct assessments of language or general knowledge, math, and reading skills; teacher ratings of attention, social skills and behavior problems; and parent ratings of health. The NICHD SECCYD also included an assessment of attention. The school-age outcome variables in both datasets included direct assessments of math and reading and teacher ratings of social skills and behavior problems, collected from entry to school through eighth grade for the ECLS-B and through tenth grade for the NICHD SECCYD. Similarly, a consistent set of child and family characteristics (e.g., race/ethnicity, child age, family income or poverty status, maternal education, marital status, etc.) were identified across the two datasets and these were entered as controls in the analyses.
Analytic methods used to examine thresholds
Three sets of longitudinal analyses of each of the four school-age outcomes (i.e., reading, math, social skills and behavior problems) were conducted. In the first analysis, the individual school readiness variables were entered as linear and nonlinear (quadratic) predictors. Using this variable-centered approach, the school readiness skills were first considered separately to look at the predictive power of each school readiness variable to each later school outcome and then secondly considered together to identify which school readiness skills provided the best prediction of which school-age outcomes. Evidence of a nonlinear association between entry-level skills and subsequent rate of acquisition of skills over time would be an indication of a threshold. To the extent that this nonlinear relationship indicated acceleration in rate of growth over time, it would suggest a “springboard” effect.
In the second analysis, a priori cut points in the individual school readiness variables using a piecewise regression analysis explored whether each school readiness variable predicted academic and social development differently for children above and below the cut point on the school readiness variable. The cut point of one standard deviation below the mean was selected a priori to distinguish those in the “low” versus “normal” range on each school readiness variable. One standard deviation was chosen because it is widely used to distinguish children deemed at-risk for problems on many standardized assessments.
Thus, for the piecewise analysis, two groups were created for each school readiness variable:
- Children in the “low range” on a school readiness skill (i.e., those who scored more than one standard deviation below the mean on the school readiness variable), and
- Children in the “normal range” on a school readiness skill (i.e., those who scored at or above one standard deviation below the mean on the school readiness variable).1
Of interest was whether this a priori threshold in school readiness skills predicted to differential outcomes over time.
In the third analysis, a person-centered approach was used to identify different combinations of school readiness skills which might serve as a springboard for acquisition of subsequent academic and social skills. Latent profile analyses created the combinations of school readiness skills that were most prevalent in the samples and these profiles of school readiness skills were used in models of longitudinal prediction of subsequent academic and social school-age outcomes. Of interest was the examination of developmental trajectories over time for children with different school readiness profiles. For example, finding that a particular school readiness profile predicted accelerated growth in certain skills over time might suggest the identification of a threshold.
[ Go to Contents ]
Findings
Is there any evidence for “thresholds” of school readiness?
These analyses revealed limited evidence of thresholds. No evidence emerged indicating cut points above which children showed more rapid gains during the school years — that is, there was no evidence of a springboard effect. In contrast, some evidence emerged suggesting school was compensatory. All of the longitudinal analyses suggested that the children who started school with lower cognitive and academic skills tended to show slightly larger gains on academic skills during the school years. These children did not catch up with their more advantaged peers, but the gap declined slightly over time.
Do school readiness skills matter?
All of the analyses indicated that children who started school with higher levels of cognitive, academic and social skills tended to maintain that advantage over time. The ability of school readiness skills to predict the level of subsequent academic and social skills during the school years was slightly stronger among children in the “normal” range in their school readiness skills. For example, the piecewise analysis of the school readiness variables in the NICHD SECCYD data found reliably stronger prediction in the normal range than low range of overall levels of language, reading, math, and social skills for all school readiness variables except health. While there was no one level which children needed to reach by school entry to benefit from school, starting at a higher level was clearly better for later performance. That is, the findings suggest that improving children’s school readiness skills will benefit them no matter where they may be on the continuum.
[ Go to Contents ]
Summary
The In the Running project’s findings contribute to an emerging picture of school readiness skills and developmental trajectories. While no children appeared to be primed for accelerated growth by being above a particular threshold of skill level at school entry, the findings were informative about relationships between school readiness skills and subsequent academic and behavior skills during the school years.
The results of the secondary data analyses conducted as part of the In the Running project showed limited evidence of these thresholds, but confirmed strongly the importance of school entry skills in predicting later school outcomes. While there was little evidence for a clear “threshold” above which children showed accelerated growth in either academic or social outcomes over time, the skill levels with which children entered school predicted their skills levels during the school years. These findings, albeit not causal, suggest benefits associated with helping all children develop their cognitive, social-emotional, and attention skills as much as possible before school entry even if they are not able to reach the same skill levels as their better-performing peers. Collectively, these findings indicate that efforts to support children’s school readiness skills prior to school entry are critically important, and that school itself may be an important intervention for those children most at risk of poor outcomes.
Implications for policy and practice
The goal of maximizing child growth before school entry rather than achieving specific skill thresholds has important implications for preschool programs, quality rating and improvement systems (QRIS), and school entry assessments. If children do not need to reach specific thresholds on skills before school entry, but do benefit from development of skills prior to school entry as suggested by the In the Running analyses, early care and education programs and the QRIS systems designed to evaluate them should focus on improving quality that leads to continuous development of children’s skills across developmental domains rather than trying to get children to reach a single target or average for a particular skill.
Measures that focus on getting a child to a specific skill threshold may too easily dismiss children at the low end of the scale, even though they could benefit tremendously from continued improvement despite not attaining the national norm. Additionally, threshold-based child assessments may also dismiss the continued growth of children at or above the threshold as of lesser importance, although the better scores children have at school entry, the more likely they are to have better performance later in school. While criterion-based skill measures are critically important to identifying and tracking a child’s development, artificial thresholds of achievement do not appear useful.
A complementary project to In the Running, supported by the Office of Planning, Research and Evaluation within HHS’s Administration for Children and Families, makes a similar point about the place of thresholds within our understanding of children’s early growth and development. The Quality, Dosage, Features and Thresholds (Q-DOT) project found some, though not consistent, evidence of thresholds between the quality of early care and child outcomes. 2 The evidence of thresholds was more likely to be found in analyses considering measures of quality focusing more specifically on teacher-child interactions, and especially instructional interactions, than on measures of global quality. Although evidence of thresholds was found in looking at the associations of quality and child outcomes across domains of development, there was more evidence of thresholds in predicting to measures of children’s academic outcomes. These findings, like those of In the Running, underscore the need to focus on the full range of quality and the full continuum of child outcome scores when seeking to strengthen quality and children’s development. Q-DOT’s findings, again like those of In the Running, also suggest that while efforts to improve early care quality are meaningful and improve outcomes, children may not benefit from policies which set quality requirements with too much focus on a single skill area.
Implications for future research
While the In the Running project represented a more complex and thorough investigation of school readiness thresholds and trajectories than had previously existed in the literature, the analysis was challenged by limitations, including those stemming from the available data.
A major limitation of this study was the type of data available for analysis. Most large-scale national surveys did not use criterion-based measures (i.e., those that compare children to a baseline score and not to other children) which lend themselves better to cutpoint analyses and which are more similar to the types of measures states and school systems are using currently to assess children’s skill levels in kindergarten. Additionally, continuity within skill domains identified by the secondary data analyses may have reflected the types of measures used within versus across domains and may have obscured accurate assessments of children’s skills in the same domain over time.
In addition, currently, there is difficulty in identifying an a priori cut point for continuous variables in existing datasets; there was some evidence that one standard deviation below the mean (the cut point used in these analyses) was a good estimate of children who are doing well versus not. However, a differently operationalized cut point that may have more accurately reflected the skill level at which children were truly at risk for poor later school outcomes and therefore may have yielded more conclusive or clear evidence of thresholds.
Much more work is needed in this largely unexplored area of study of school readiness thresholds, especially with longitudinal datasets that contain more criterion-based measures and with the use of causal analytic methods. The In the Running study, therefore, points to future directions for new data collection efforts as well as for further prospective and retrospective data analyses.
[ Go to Contents ]
ABOUT THIS RESEARCH BRIEF AND ACKNOWLEDGEMENTS
The findings described in this brief are the product of the ASPE-funded project entitled In the Running for Successful Outcomes: Early Education, Care, and Comprehensive Services (Contract # HHSP23320095631WC). The In the Running project examined the evidence for thresholds of school readiness within and across domains (including language, reading, math, attention, socioemotional, and health) which, when attained, predict skill acquisition in later schooling. Research questions included: Are there particular school readiness skills or a level of development that children need to attain in early childhood in order to meet later measures of success? Do outcomes in elementary or later schooling depend on the school readiness skills and competencies in various domains at entry to school? Through secondary data analysis, the project tested the extent to which associations between trajectories of school-age skills can be predicted from school readiness skills (both within and across domains) and whether those associations are linear or nonlinear. It also examined whether identified thresholds replicate across datasets.
Key project tasks included a review of the research literature and a multi-method approach to the secondary analysis of two large-scale, longitudinal datasets: the Early Childhood Longitudinal Survey-Kindergarten 1998-1999 cohort (ECLS-K) and the NICHD Study of Early Child Care and Youth Development (SECCYD). A panel of experts in the field was convened and consulted on technical aspects of the project and on the implications of findings for early childhood programs.
The project team, led by Tamara Halle at Child Trends, also included Beth Hair (NORC; co-Principal Investigator), Peg Burchinal (UNC-Chapel Hill; Expert Advisor) and Marty Zaslow (SRCD and Child Trends; Expert Advisor). The Technical Expert Panel was comprised of Linda Espinosa (University of Missouri-Columbia, Retired), Sue Hegland (Iowa State University), Stephanie Jones (Harvard Graduate School of Education), and Katherine Magnuson (University of Wisconsin-Madison). Amy Madigan and Laura Radel were the Project Officers at ASPE for this project. The contents of this brief are solely the responsibility of the authors and do not necessarily represent the official views of the Office of the Assistant Secretary for Planning and Evaluation, or the U.S. Department of Health and Human Services.
Endnotes
1 One standard deviation above the mean was used for behavior problems.
[Go to Contents]