A Bipartisan Case for Government Spending to Devise and Evaluate Smarter Policies

Just before the start of a lengthy government shutdown, evidence-based policymaking quietly achieved a major win. On December 19th, 2018 The Foundations for Evidence-Based Policymaking Act of 2017 passed the Senate with unanimous consent. Few issues facing the United States have widespread agreement, yet evidence-based policy is one area that continues to garner bipartisan support because it is fiscally responsible and effective across many diverse areas of government. Evidence-based policymaking ensures that decisions are driven by proof of what works, not simply by opinion.

Small Steps and Future Work

Although the bipartisan vote on this legislation is a promising step, there is still more to be done to strengthen evidence-based decision making. For example, problems arise when the government must swiftly respond to issues like human trafficking or young people who age out of foster care, challenges not currently addressed by evidence-based policies. Even when the government does have evidence-based programs, additional evidence and evaluation may be needed – for example, to determine whether they can be extended to different populations or whether modifications might be needed to adapt programs to unique local contexts. Evidence-based programs are typically tested first with small populations, and it is challenging to ensure continued effectiveness when they are expanded to larger populations.  For example, the Laura and John Arnold Foundation studied randomized controlled trials (RCTs) of federally-funded programs after they had been scaled-up, and found nearly all (11 of 13) have failed to find significant outcomes overall.

This shows that evaluations are needed both to develop initial evidence and to further test the effectiveness of programs after they are implemented at scale. But current investments in research and development, evaluation, and implementation are fragmented, and government agencies can do better research to inform strategic decisions by coordinating investments in research and development of evidence-based programs, implementing programs with higher-standards of evidence,  and continuing to evaluate programs as they are expanded. This approach is not new, but it has not been consistently used.

Spending on Ongoing Program Evaluations

Evaluation of federal programs is often mandated by legislation, yet little attention has been paid to the realistic funding levels required to support adequate research and evaluation. Spending guidelines cannot simply be set at an arbitrary percentage of any given legislative grant. Funding levels for ongoing research should vary based on the evidence already available about a program.  As the evidence becomes available about specific programs, funding for rigorous early research — such as studies using randomized trials — should be redirected toward continuing evaluation and improvement. Ongoing research and adjustments based on the facts can help promising programs continue to achieve their intended results when they are extended to new populations or locales — that is, when programs are put in place outside of the highly-controlled environments in which they were originally devised and tested.  

Currently, a common recommendation is that agencies set aside from 1% to 3% of their total budgets for evaluation of programs, so that particular offices can develop evaluation plans and allocate funds to programs as they see fit. In contrast, there is no simple recommendation for evaluation spending on individual grants, contracts, or programs. Recommended expenditures range between .33% and 25% depending on needs are for more or less extensive investigations or monitoring of particular programs’ impact or implementation. Spending based on percentages may prove inadequate because it does not take into account the many nuances of program development and implementation. A better set of guidelines should be developed, taking account of the various contexts that may require more or less spending on program evaluation.  

What Should Be Done with Evidence about Program Effectiveness?

When evaluation findings are positive, support for the continuation of the programs at issue tends to be strong. But, when program evaluations fail to show evidence that intended outcomes are happening, the findings can be used in alternative ways — either as a weapon to eliminate programs or cut their funding, or as tools to improve programs.  Whether or not programs should be eliminated based on findings should depend on the evidence base of the program and not political climate. In instances where an evidence-based program registers no effects in an evaluation study, it may be more efficient to improve the existing strategy than to reinvent another program, starting from scratch with little available evidence. Conversely, if a new program  without strong evidence demonstrates no effects, it might be advisable to redirect investments to another program backed by some evidence of effectiveness, or reallocate resources to research and development on possible new solutions.

Frustration can result when communities whose needs are addressed by public policies witness programs coming and going, sometimes scrapped after one failed evaluation. This means that evaluation research must be supported and used in careful, strategic ways, to improve as well as eliminate promising policies and keep practitioners and communities engaged in the search for programs that prove effective.    

The Future of Evidence-Based Policy

Although evidence-based policy has been developed in key areas — such as for the Maternal, Infant, and Early Childhood Home Visiting Program and the Family First Prevention Services Act — there is still much to do. Widespread adoption of evidence-based policy in the federal government will require programs to be launched on the basis of prior rigorous evidence and, once implemented, continuously evaluated. Evidence-based policymaking requires much more  than legislative mandates, and more than setting aside fixed proportions of agency budgets for research. Progress in this area is challenging to achieve, but it should be a bipartisan ideal to ensure that taxpayer dollars are spent efficiently to federal programs. Good research can help agencies discover what might work, and then evaluate programs in an ongoing way to see if they are effectively serving their participants.