Title I and Student Achievement: A Meta-Analysis of Federal Evaluation Results

NICHCY’s Structured Abstract 51 describes the following:

Title | Title I and Student Achievement: A Meta-Analysis of Federal Evaluation Results

Author | Borman, G.D., & Agostino, J.V.

Source Educational Evaluation and Policy Analysis, 18(4), 309-326.

Year Published | 1996 (Winter)

Abstract
Despite the seeming wealth of Title I evaluation information, the educational effectiveness of the program has remained debatable. Inconsistent findings can be attributed to variations in evaluation methods. Nevertheless, results from key studies have contributed to a conventional wisdom concerning Title I and student achievement. In contrast to previous reviews, this study employed meta-analytic techniques to assess the overall impact of the program on achievement and to examine the effects of mediating methodological and programmatic factors. The data were derived from 17 federal studies, ranging from 1966 to 1993, from which 657 unique effect sizes were derived. Results indicated a modest overall impact of Title I. However, the mediating factors were significant predictors. After controlling for these effects, Title I effect sizes were more favorable as the program matured. This finding may be attributable to expanded federal oversight and the growing focus on program improvement that has evolved over the years.

Background
Title I of the Elementary and Secondary Education Act (ESEA) provides “financial assistance to local education agencies serving areas with concentrations of children from low-income families to expand and improve their educational programs by various means…which contribute particularly to meeting the special educational needs of educationally deprived children” (Elementary and Secondary Education Act of 1965, 79 Stat. 27). During the last reauthorization of the Elementary and Secondary Education Act, it became known as the “No Child Left Behind” Act or NCLB, so many people associate those names with Title I today.

Seventeen federal summaries of local Title I program evaluations were conducted between 1966 and 1993. In total, the progress of over 41 million children in Title I programs was evaluated in these reports. This meta-analysis examines those federal summaries and what they can tell us about: (a) the effectiveness of Title I programs for reading and math; (b) changes in Title I’s effectiveness over its first 2 decades; and (c) the effect of summer vacation on the educational performance of children in Title I at the beginning and end of each school year.

Research Questions

  1. How effective are Title I programs for reading and math?
  2. Has Title I’s effectiveness changed over its first 2 decades?
  3. What is the effect of summer vacation on the educational performance of children in Title I at the beginning and end of each school year?

Research Design
Meta-Analysis *

  • Number of Studies Included | 17
  • Number of Subjects | 41,706,196
  • Years Spanned | 1966-1993

Research Subjects
Children and adolescents in U.S. public school Title I programs. By definition, Title I programs are intended for schools with high concentrations of children living in poverty.

Age/Grade of Subjects
The vast majority of Title I students—90%—are in grades K through 8. The remaining 10% are in 9th through 12th grade.

Specified Disability
N/A

Intervention
Students participated in Title I programs. Eleven of the 17 studies provided Title I supports in both reading and math, and 6 of the studies only examined Title I programs in reading.

Duration of Intervention
The students in these studies participated in Title I programs for at least one year.

Findings

  1. Title I students participating in math programs tended to post larger gains than students in reading programs.
  2. Summer vacation had a greater negative impact on math skills than reading skills. Students tended to make more progress in math than reading during the school year.  However, they were more likely to retain their reading skills and lose some of their math skills over the summer.
  3. Students who participated in Title I in early elementary school tended to outperform their peers in middle and high school.
  4. “After controlling for other variables, the model suggests that the Title I program has become more effective as it has matured.”

Combined Effects Size

  • Controlled comparisons produced effect sizes of 0.18.
  • Overall mean weighted effect size = 0.11. The researchers suggest this “should not be considered an estimate of the population effect size. Instead, it should be considered as the mean of the effect sizes found across a diverse set of evaluation and program criteria, which influenced the estimates of Title I effectiveness in significant ways” (p. 322).
  • Grade level was negatively correlated to effect size; each grade level increment was associated with a -0.01 effect size decrement.
  • “The model predicted that Title I effect sizes were 0.27 greater (or nearly 6 percentile points greater) during the 1992-1993 academic year than they were during the first year of the program in 1965″ (p. 320).

Conclusion/Recommendations
Title I became an increasingly effective program over its first 2 decades. Research shows that those students who participated in Title I programs in the early grades have maintained the gains they achieved in Title I through middle school and high school, continually outperforming their non-Title I peers. While students make their biggest gains in math during the school year, the likelihood exists that they will lose many of their math skills over the summer. The authors suggest that schools consider using part of their Title I funds to provide math programs over the summer, so that students will not lose the gains they make in math during the school year.

__________________

* Meta-Analysis | A widely-used research method in which (1) a systematic and reproducible search strategy is used to find as many studies as possible that address a given topic; (2) clear criterion are presented for inclusion/exclusion of individual studies into a larger analysis; and (3) results of included studies are statistically combined to determine an overall effect (effect size) of one variable on another.

Back to top