Demonstrating How Low-Cost Randomized Controlled Trials
Can Drive Effective Social Spending:
Project Overview and Update
In response to the White House and Office of Management and Budget (OMB) call to action for evidence-based reforms across the federal government, the Coalition for Evidence-Based Policy has launched a competition for low-cost randomized controlled trials (RCTs) that seek to build valid, actionable evidence about “what works” in U.S. social spending. This is designed as a high-visibility, three-year initiative, whose purpose is to demonstrate the feasibility and value of low-cost RCTs to a wide policy and philanthropic audience.
In its first year, the competition selected and funded three low-cost RCTs that met the criteria for policy importance and other factors described in the Request for Proposals – as follows:
- A large, multi-site RCT of Bottom Line, a program that provides one-on-one guidance to help low-income, first-generation students get into and graduate from college. This study will measure college enrollment, persistence, and completion outcomes for a sample of nearly 1,400 students over a seven-year period, using administrative data from the National Student Clearinghouse. The study cost is $159,000. (Link to brief study summary.)
- A large RCT of Durham Connects, a postnatal nurse home visiting program designed to improve child and mother health and well-being. The study will use hospital administrative records to measure program impacts on families’ emergency department use and related healthcare costs through child age 24-months, for a sample of about 1,100 families in Durham County, North Carolina. The study cost is $183,000. (Link to brief study summary.)
- A large, multi-site RCT of workplace health and safety inspections conducted by the federal Occupational Safety and Health Administration (OSHA). For a sample of about 29,000 business establishments eligible for a randomized inspection, the study will test whether being randomly chosen for inspection affects establishments’ subsequent injury rates and business outcomes (e.g., sales, business closures) over a multi-year period – all measured through administrative data from OSHA and other sources. The study cost is $153,000. (Link to brief study summary.)
These and other examples were discussed at a July 28, 2014 conference on low-cost RCTs, co-sponsored by the White House Office of Science and Technology Policy (OSTP) and our organization. The conference explored wider use of such studies with officials from the White House, OMB, federal agencies, Congress, state/local agencies, and philanthropic foundations (see our brief overview, and White House OSTP summary). The conference builds on other recent Executive Branch efforts to advance low-cost, rigorous evaluations, described in the 2014 Economic Report of the President (chapter 7) and the FY 2015 budget (chapter on performance and management, pp. 65-70).
In its second and third years of the competition, we will select and fund a total of about 6 additional low-cost RCTs. The second Request for Proposals will be released in December of 2014, and will be available on this webpage. If you wish to be notified when the RFP is released, please subscribe to our listserv.
The Coalition is a nonprofit, nonpartisan organization that is unaffiliated with any social programs or program models. This initiative is funded through philanthropic grants to the Coalition from the Laura and John Arnold Foundation and the Annie E. Casey Foundation.