Why it Matters
State education agencies (SEAs) and grantees should allocate at least 1% of their overall programmatic budgets to support ongoing evaluation, including for programs that have never been rigorously studied to show proof of effectiveness.
Ongoing evaluation allows an SEA to understand how well the grant program is working, for whom and under what circumstances.
Evaluation includes both efforts to study how a program is being implemented and, where possible and relevant, impact evaluations.
Appropriate evaluation and learning activities will vary based on the characteristics of the grant program and the projects it funds. That said, all activities should:
Be aligned with the SEA’s strategic priorities, evaluation policy and the grant program’s theory of change.
Enable states to speak to the effectiveness of both the grant program itself and the effectiveness of the programs invested in by grantees.
When should SEAs use impact evaluations?
It is beneficial to conduct an impact evaluation in a policy area with a weak evidence base, to build new knowledge about whether something produces a positive impact.
In cases where there is strong evidence of the core features of an evidence-based practice, ongoing evaluation might focus on whether the practice is being well-implemented in a new context.
Using Grant Funds for Evaluation
SEAs should provide guidance and standard grant application language that makes clear that grantees are authorized to invest funds to build and strengthen their data capacity and infrastructure and conduct evaluations of their grant-funded efforts.
State Example:
The Colorado Department of Education includes, “Evaluation of existing structures, practices, and instructional materials across birth through grade 12 to ensure they are evidence-based…” as an allowable use of funds awarded through their 2021 Comprehensive Literacy State Development grant program.
Federal Example:
At the federal level, the Office of Budget and Management's Uniform Administrative Requirements, Cost Principles and Audit Requirements for Federal Awards (commonly called “Uniform Guidance”) includes guidance that evaluation is an allowable use for all federal grants.
Strategies to Encourage Critical Learning and Evaluation Activities
SEAs with human and financial resources for sponsoring evaluations may form partnerships with their grantees, in which the grantees implement programs and the SEA evaluates the effectiveness of the programs. This strategy for engaging in evaluation is particularly valuable in cases where the grant program funds a limited number of programs across grantees.
SEAs may require:
Applicants to include evaluation plans in their proposals, and assign points based on the quality of the proposed evaluation in rating proposals.
Grantees to conduct evaluations.
Agency review and approval of evaluation plans after award.
This strategy for engaging in evaluation is particularly valuable in cases where the grant program is funding many different programs across grantees.
SEAs may tailor evaluation requirements based on several considerations:
The SEA may require only larger grantees to conduct evaluations, or may require larger grantees to conduct more ambitious evaluations than smaller grantees.
The SEA may choose not to require grantees to conduct evaluations — or specify that they should conduct implementation evaluations rather than impact evaluations — if they are implementing evidence-based models.
The SEA may adjust requirements for evaluation based on the capacity of likely applicants. In the case where likely applicants may have limited evaluation capacity and resources, the SEA may choose primarily to conduct state sponsored evaluations rather than requiring grantees to conduct evaluations. Or, the SEA may invest heavily in technical assistance to help grantees conduct successful evaluations, as described below.
To support grantees in fulfilling this requirement, SEAs may want to consider opportunities to facilitate connections between grantees and external evaluators.
SEAs may provide technical assistance to help grantees develop and carry out evaluations, including by building connections between grantees and external evaluators. SEAs may also choose to enter cooperative agreements with grantees that outline the grantees’ and granting agencies’ respective roles. In addition to providing technical assistance, the state role might include review and approval of key activities, such as selection of outcome measures or other aspects of evaluation design. A February 2022 special issue of Evaluation Review is dedicated to discussing evaluation technical assistance.
SEAs may sponsor annual meetings for grantees of one or several related programs. SEAs may include evaluation among the discussion topics at these meetings. SEAs may also take a more intensive approach. For example, an SEA might sponsor quarterly meetings (virtual or in person) for a subset of grantees carrying out related evaluations. Some grantees may be encouraged or expected to collaborate closely, such as by establishing common research questions or outcome measures, or pooling data.
Rather than having each grantee conduct its own evaluation in isolation, SEAs may take a coordinated approach by designating one grantee as the lead to conduct an evaluation for a group of grantees.