2025-08842NoticeWallet

Federal Researchers Revise the Scorecard for Job Program Studies

Published Date: 5/19/2025

Notice

Summary

The Administration for Children and Families is updating the rules for how they review and rate studies about programs that help people with low incomes find and keep jobs. They want your feedback on these new standards by June 18, 2025. These changes will help states and local agencies pick the best programs to boost employment, making sure taxpayer money is spent wisely.

Analyzed Economic Effects

5 provisions identified: 5 benefits, 0 costs, 0 mixed.

New Standards to Rate Program Cost Studies

If you are a State or local agency deciding which employment programs to fund, the Clearinghouse will use new standards to review three types of cost studies (cost analysis, cost-effectiveness, cost-benefit). Cost studies must meet threshold standards to be rated; those that meet thresholds are scored on 1–3 quality scales that are averaged (rounded to the nearest tenth). Average-score bands are: 1.0–1.5 = "meets standards with low quality," 1.6–2.5 = "meets standards with moderate quality," and 2.6–3.0 = "meets standards with distinction." Studies that fail thresholds receive a rating of "Interpret cost findings with caution."

Clearinghouse Aims to Improve Program Choices

ACF states that Pathways to Work is intended for State and local Temporary Assistance for Needy Families administrators and others to identify services that best help people with low incomes improve employment and earnings, and that the Protocol Version 2.0 revisions (including cost-study standards) are intended to help states and local agencies pick the best programs and make sure taxpayer money is spent wisely.

Baseline Equivalence Via Effect-Size Rules

When a study has a statistically significant baseline difference on a required characteristic, reviewers may compute an effect size. If the effect size is less than 0.05 standard deviations, the baseline equivalence requirement is met; if between 0.05 and 0.25 standard deviations the study must control for the characteristic; if greater than 0.25 standard deviations (or no effect size can be computed), baseline equivalence is not met.

Limited Waiver of 1-Year Baseline Timing Rule

The protocol relaxes the rule that control measures of employment or earnings be taken at least 1 year before an intervention in three limited cases: (1) studies where participants were incarcerated for 1 year or longer before the program began; (2) studies of minor youth who would not have 1 year of prior work history (for example, programs for 16-year-olds); and (3) when authors provide explicit, credible evidence that both groups could not have had a change in earnings in the year prior. Any timing waiver requires approval by the Pathways to Work principal investigator. The exemption applies only to the timing requirement, not to the requirement to control for pre-program differences.

Changes to How Program Effectiveness Is Labeled

The Protocol revises program effectiveness ratings: the "Insufficient Evidence to Assess Support" category now applies only to single-study cases where none of the findings are statistically significant; the "Not Supported" category continues but now explicitly covers two or more studies with a pattern of only null or unfavorable findings; and the prior category "No Evidence to Assess Support" is renamed "Cannot Assess Support."

Personalized for You

How does this regulation affect your finances?

Personalize government policy and PRIA will tell you what this federal register document means for your household, plus every other regulation we track. PRIA reads each provision against your financial profile to show you exactly what matters to your wallet.

Key Dates

Published Date
5/19/2025

Department and Agencies

Department
Independent Agency
Agency
Health and Human Services Department
Children and Families Administration
Source: View HTML

Related Federal Register Documents

Previous / Next Documents

Back to Federal Register