Skip to main content

MILLENNIUM CHALLENGE CORPORATION: Vanuatu Compact Overstates Projected Program Impact

GAO-07-909 Published: Jul 26, 2007. Publicly Released: Jul 26, 2007.
Jump To:
Skip to Highlights

Highlights

What GAO Found

MCC projects that the Vanuatu compact’s transportation infrastructure projects will provide direct benefits such as reduced transportation costs and induced benefits from growth in tourism and agriculture. MCC estimated the costs and benefits over 20 years, with benefits beginning in full in 2008 or 2009 and growing each year, and it counted poor, rural beneficiaries by defining the area where benefits were likely to accrue. Using projected benefits and costs, MCC calculated the compact’s economic rate of return (ERR) and its effects on Vanuatu’s gross domestic product (GDP) and per capita income.

MCC’s portrayal of the projected impact does not reflect its underlying data. MCC states that per capita income will increase by approximately $200, or 15 percent, by 2010 and by $488, or 37 percent, by 2015. However, MCC’s underlying data show that these figures represent the sum of individual years’ gains in per capita income relative to 2005 and that actual gains will be $51, or 3.9 percent, in 2010 and $61, or 4.6 percent, in 2015. MCC also states that GDP will increase by an additional 3 percent a year, but its data show that after GDP growth of 6 percent in 2007, the economy’s growth will continue at about 3 percent, as it would without the compact. MCC states that the compact will benefit approximately 65,000 poor, rural inhabitants, but this statement does not identify the financial benefits that accrue to the rural poor or reflect its own analysis that 57 percent of benefits go to others.

We identified five key risks that could affect the compact’s projected impacts. (1) Cost estimate contingencies may not be sufficient to cover project overruns. (2) Compact benefits will likely accrue more slowly than MCC projected. (3) Benefit estimates assume continued maintenance, but MCC’s ability to ensure maintenance will end in 2011, and Vanuatu’s maintenance record is poor. (4) Induced benefits depend on businesses’ and residents’ response to new opportunities. (5) Efficiency gains, such as time saved in transit, may not increase per capita income. Our analysis of these areas of risk illustrates the extent that MCC’s projections are dependent on assumptions of immediate realization of benefits, long-term maintenance, realization of induced benefits, and benefits from efficiency gains.

Why GAO Did This Study

In January 2004, Congress established the Millennium Challenge Corporation (MCC) for foreign assistance. Congress has appropriated almost $6 billion to MCC. As of March 2007, MCC had signed almost $3 billion in compacts with 11 countries, including a 5-year, $65.7 million compact with Vanuatu. MCC states that the Vanuatu compact will have a transformational effect on the country’s economy, increasing per capita income and GDP and benefiting 65,000 poor, rural people. GAO examined (1) MCC’s methods of projecting economic benefits, (2) MCC’s portrayal and analysis of the projected benefits, and (3) risks that may affect the compact’s impact. GAO reviewed MCC's analyses and met with officials and business owners in Vanuatu as well as with other donors.

Recommendations

GAO recommends that the Chief Executive Officer of MCC (1) revise the public reporting of the Vanuatu compact’s projected impact, (2) assess whether similar reporting in other compacts accurately reflects underlying analyses, and (3) improve its economic analyses by more fully accounting for risks to project benefits. MCC did not directly address our recommendations but commented that it had not intended to make misleading statements and that its portrayal of projected results was factual and consistent with underlying data.

Recommendations for Executive Action

Agency Affected Recommendation Status Sort descending
Millennium Challenge Corporation The Chief Executive Officer (CEO) of MCC should revise the public reporting of the Vanuatu compact's projected impact to clearly represent the underlying data and analysis.
Closed – Implemented
MCC has downwardly revised its public statements of the expected benefits of the Vanuatu compact. MCC eliminated 9 of the original 11 transportation subprojects from the compact due to cost concerns, greatly reducing the expected number of beneficiaries. From an initial public statement of 65,000 beneficiaries, MCC reduced the expected number of beneficiaries to approximately 18,000 in mid-2008 and to 14,783 in the public Vanuatu Quarterly Status Report (QSR) of December 2010. MCC has also restated the expected economic rate of return for the overall compact in its publicly available monitoring and evaluation plan for the compact, reducing it from 24.2 to 20.7%. MCC has also updated its expected effect on income. MCC now states in the December 2010 QSR that the $65.6 million compact will result in an increase in household income of $73.8 million over 20 years. This equates to a per capita income increase of approximately $18 per year per person in Vanuatu -- a decrease from MCC?s initial statement to Congress that the compact would increase "average income per capita (in real terms) by approximately $200, or 15 percent of current income per capita, by 2010" in the country. MCC has also publicly stated a new expected poverty impact: reducing the proportion of households in the project catchment area living below the National Basic Needs Poverty Line on the island of Efate from 37.7% to 31.2% and from 14.6% to 7.3% on the island of Santo.
Millennium Challenge Corporation The CEO of MCC should improve economic analysis by phasing the costs and benefits in compact ERR calculations and by more fully accounting for risks such as those related to continuing maintenance, induced benefits, and monetized efficiency gains as part of sensitivity analysis.
Closed – Implemented
MCC's response letter noted that instructions to consultants undertaking economic due diligence work have included more specific requests for information under different scenarios. MCC also provided us with excerpts from instructions to its economists and consultants. These instructions have directed them to take actions such as (1) preparing best and worst case scenarios and Monte Carlo simulations of economic rates of return (ERRs), (2) assessing the likelihood of key assumptions, such as projected farm-gate prices, being realized, (3) identify key assumptions that are difficult to quantify (such as induced benefits or monetized efficiency gains) and present alternative scenarios under different assumptions, (4) state whether maintenance of existing infrastructure has been inadequate and whether it is expected to remain inadequate or institutional strengthening efforts may change the maintenance performance.
Millennium Challenge Corporation The CEO of MCC should assess whether similar statements in other compacts accurately reflect the underlying data and analysis.
Closed – Not Implemented
In its initial response to the GAO recommendation, MCC stated that did not believe that a comprehensive review of all of its spreadsheets comparing them with all past public statements about impact was feasible or justified by the results of the Vanuatu audit. In addition, in July 2011, GAO-11-728 reported that MCC had not continued to update the economic analyses and expected results for its compacts in Cape Verde and Honduras as the scope of these compacts changed. MCC had not updated economic rate of return (ERR) analyses of the largest compact projects in Cape Verde and Honduras and these analyses have not been well documented or linked to revised targets. For example, in Honduras, updated ERR analyses of projects representing over 50 percent of compact funds have not been well documented or supported. While MCC updated the ERR analysis for the watershed management and agricultural support project in Cape Verde, the analysis does not reflect the values and numerical ranges of key revised targets.

Full Report

GAO Contacts

Office of Public Affairs

Topics

Program beneficiariesConstructionCost estimatesCoastlineBridgesPublic roads or highwaysData errorsTransportation infrastructureEconomic growthBudgeting