This is the accessible text file for GAO report number GAO-08-730 
entitled 'Millennium Challenge Corporation: Independent Reviews and 
Consistent Approaches Will Strengthen Projections of Program Impact' 
which was released on June 17, 2008.

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Chairman, Committee on Foreign Affairs, House of 
Representatives: 

United States Government Accountability Office: 
GAO: 

June 2008: 

Millennium Challenge Corporation: 

Independent Reviews and Consistent Approaches Will Strengthen 
Projections of Program Impact: 

GAO-08-730: 

GAO Highlights: 

Highlights of GAO-08-730, a report to the Chairman, Committee on 
Foreign Affairs, House of Representatives. 

Why GAO Did This Study: 

In January 2004, Congress established the Millennium Challenge 
Corporation (MCC) for foreign assistance. Eligible countries submit 
compact proposals for MCC funding for projects aimed at reducing 
poverty through economic growth. To assess the proposed compacts’ 
likely impact, MCC performs economic analyses estimating the compacts’ 
economic rate of return (ERR) and effects on income and poverty as well 
as the number of compact beneficiaries. MCC uses these analyses to 
inform its decisions to fund proposed compacts and to inform Congress 
and the public about its progress in achieving its mission of poverty 
reduction through economic growth. GAO was asked to examine MCC’s 
projections of (1) ERR and (2) compacts’ impact on income and poverty 
as well as numbers of beneficiaries. GAO reviewed MCC’s stated impacts 
and analyses for four MCC compacts that represented 41 percent of MCC’s 
compact assistance and met with MCC officials. 

What GAO Found: 

MCC used different time frames and methods to calculate ERRs for its 
compacts with Armenia, El Salvador, Lesotho, and Mozambique. In 
calculating ERR for 20 projects within the compacts, MCC used a 20-year 
time frame for 9 projects and used different time frames for the other 
11. In 2 of the 11 projects, using a 20-year time frame, as MCC used 
for similar projects, would reduce the ERR below the level MCC set as 
the minimum acceptable ERR. At the compact level, MCC’s use of varying 
time frames did not affect the ERRs significantly. MCC used varying 
methods to account for the costs of same-sector projects, although its 
approaches to determining project benefits were generally similar. MCC 
also used two different methods to calculate compact-level ERR; 
however, the choice of method did not reduce it below the minimum ERR. 
In three of the four compacts that we reviewed, MCC did not retain 
documentation of the economic analyses used to support the investment 
decision, but continued to modify the analyses. MCC has recently begun 
to standardize elements of its economic analyses and centralize its 
records management. 

MCC identified and corrected analytic errors in its projections of 
impact on income and poverty. MCC also used varying methods to project 
these impacts. 

* In responding to GAO’s questions about its published projections of 
impact on income and poverty, MCC identified analytic errors for three 
of the four compacts and, in correcting these errors, generally lowered 
the projected impacts. Correcting these errors raised one projection by 
5 percent but reduced others by 3 percent to 96 percent. According to 
MCC officials, the revised projections would not have affected MCC’s 
decision to recommend signing the compacts. The officials noted that, 
in the future, compact impact projections will undergo peer review. 
However, MCC has not documented procedures for these reviews. 

* MCC used varying methods for its projections of impact on income and 
poverty, limiting comparability and replicability across compacts. To 
project impact on income for Armenia, El Salvador, and Mozambique, MCC 
estimated the compacts’ impact by summing the total benefits of 
individual compact projects and adding them to the income that would 
have prevailed without MCC. However, for Lesotho, MCC estimated the 
impact on income based on the published results of a World Bank model 
based on elements different from those of the MCC compact. In response 
to our questions, MCC revised its initial estimate of the effect of 
income growth on poverty for Mozambique by presenting two estimates, 
based on Mozambique-specific and cross-country data, respectively. 
Although a number of methods for projecting poverty impact are valid, 
the method chosen can affect the results, and MCC’s guidelines do not 
identify preferred methods for these calculations. MCC also used 
varying methods to estimate numbers of beneficiaries for the compacts 
and has not provided specific criteria for defining beneficiaries; 
however, MCC officials reported they are taking steps to provide more 
detailed guidance. 

What GAO Recommends: 

GAO recommends that the Chief Executive Officer of MCC (1) adopt and 
implement written procedures for a secondary independent review of its 
economic analyses and (2) improve MCC’s guidelines by identifying a 
consistent approach with preferred methods for projecting compacts’ 
impact on income and poverty. MCC concurred with GAO’s recommendations. 

To view the full product, including the scope and methodology, click on 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-08-730]. For more 
information, contact David Gootnick at (202) 512-3149 or 
gootnickd@gao.gov. 

[End of section] 

Contents: 

Letter: 

Results in Brief: 

Background: 

MCC Used Different Time Frames and Methods to Calculate ERRs but Is 
Taking Steps to Increase Consistency: 

MCC Made Analytic Errors in Compact Impact Projections and Used Varying 
Methods that Affected the Projections' Results: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: MCC Minimum Acceptable ERRs: 

Appendix III: Compact ERRs: 

Appendix IV: Impact of Alternative Beneficiary Counts on El Salvador 
Compact Impact Projections: 

Appendix V: Comments from the Millennium Challenge Corporation: 

GAO Comments: 

Appendix VI: GAO Contact and Staff Acknowledgments: 

Tables: 

Table 1: Compact Impact Indicators Included in MCC's Public Reporting: 

Table 2: MCC Compact Hurdle Rates and Hurdle Rate Definition: 

Table 3: Comparison of MCC Compact ERRs Stated in Investment Memo with 
ERRs Calculated Using a 20-Year Time Frame: 

Table 4: Compact ERR Using Alternative Methods: 

Table 5: MCC's Compact-Level Impact Estimates for El Salvador, with 
Alternative Assumptions for Estimating Beneficiaries of Education 
Projects: 

Figures: 

Figure 1: MCC Compact Development and Implementation Process: 

Figure 2: Illustrative Examples of MCC Economic Analysis at Compact, 
Project, and Activity Levels: 

Figure 3: Illustration of Net benefits and ERR Calculations: 

Figure 4: Summary of Compacts for Armenia, El Salvador, Lesotho, and 
Mozambique: 

Figure 5: MCC Revisions to Impact Projections for Armenia, Based on Its 
Corrections of Analytic Errors: 

Figure 6: MCC Revisions to Impact Projections for El Salvador, Based on 
Its Corrections of Analytic Errors: 

Figure 7: MCC Revisions to Impact Projections for Mozambique, Based on 
Its Corrections of Analytic Errors: 

Figure 8: MCC's Alternative Methods for Calculating Compact ERR: 

Abbreviations: 

CEO: chief executive officer: 

ERR: economic rate of return: 

GDP: gross domestic product: 

MCA: Millennium Challenge Account: 

MCC: Millennium Challenge Corporation: 

OMB: Office of Management and Budget: 

[End of section] 

United States Government Accountability Office:
Washington, DC 20548: 

June 17, 2008: 

The Honorable Howard L. Berman: 
Chairman: 
Committee on Foreign Affairs: 
House of Representatives: 

Dear Chairman Berman: 

In January 2004, Congress established the Millennium Challenge 
Corporation (MCC) to administer the Millennium Challenge Account (MCA) 
for foreign assistance. MCC's mission is to reduce poverty through 
sustainable economic growth in some of the world's poorest countries 
that create and maintain sound policy environments. MCC has received 
appropriations for fiscal years 2004 to 2008 totaling more than $7.5 
billion and, as of March 2008, has signed $5.5 billion in compacts 
[Footnote 1] with 16 countries. The President has requested $2.225 
billion for fiscal year 2009. 

MCC uses income and country performance criteria to annually select a 
list of countries eligible for MCA assistance. Eligible countries may 
then submit proposals for compacts containing multiple projects for 
MCC's review and approval. On receiving proposals for MCA assistance, 
MCC undertakes a comprehensive review, or due diligence,[Footnote 2] 
which seeks to ensure that the proposed compacts will advance MCC's 
mission. As part of due diligence, MCC assesses the potential economic 
impact of each compact. During due diligence, MCC identifies a minimum 
acceptable economic rate of return[Footnote 3] (ERR) for the compact 
and its projects; compares estimated costs and benefits to determine 
the compact's ERR; and projects the compact's impact, including its 
number of beneficiaries and its impact on income, economic growth, and 
poverty. 

MCC uses its projections, as well as other information gathered during 
due diligence, to inform its internal decisions to fund proposed 
projects and compacts. The results of the due diligence assessment are 
reported in an investment memo--an internal document prepared by MCC's 
transaction team[Footnote 4] that analyzes the compact--submitted to 
MCC's investment committee.[Footnote 5] MCC also publishes its compact 
impact projections during ongoing consultations with Congress. This 
information--found in MCC documents such as compacts, compact 
summaries, annual reports, and congressional notifications and budget 
justifications--sets expectations for the compact and provides 
information to Congress and the public about MCC's progress in 
achieving its mission. 

In July 2007, we reported that MCC's portrayal of the projected impact 
of its Vanuatu compact did not reflect MCC's underlying analyses. 
[Footnote 6] Our recommendations included that MCC revise its public 
reporting of the Vanuatu compact's projected impact and assess whether 
similar reporting for other compacts accurately reflects underlying 
economic analyses. The committee subsequently requested that we examine 
MCC's economic analyses for its compacts with other countries. 

As agreed with your office, for this report, we assessed (1) MCC's 
projections of ERR and (2) MCC's projections of compacts' impact on 
income and poverty as well as numbers of beneficiaries. 

To carry out this review, we reviewed MCC compacts with four countries: 
Armenia, El Salvador, Lesotho, and Mozambique. When we began our work 
in August 2007, these four countries represented about 41 percent of 
the $3.85 billion MCC had set aside for 13 compacts and included the 
most recent compacts signed in Eurasia, Latin America, and Africa. To 
assess MCC's compact projections, we reviewed MCC public and internal 
documents as well as relevant World Bank and International Monetary 
Fund documents. We also reviewed MCC's original and revised spreadsheet 
calculations of its economic projections and interviewed MCC economists 
and other officials in Washington, D.C., regarding these analyses. 
Finally, we consulted Office of Management and Budget (OMB) and GAO 
guidance on establishing and implementing internal controls.[Footnote 
7] We determined that the data we used were sufficiently reliable for 
the purposes of our analysis; however, we did not independently assess 
the reliability of all data and assumptions that affect the projections 
and that MCC used in its underlying economic analyses of compact 
projects and activities. We also did not assess MCC's progress in 
implementing these compacts or toward its projected results. We 
conducted this performance audit from August 2007 to June 2008, in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. (See app. I for additional 
details of our scope and methodology.) 

Results in Brief: 

MCC used different time frames and methods to calculate ERRs for the 
four compacts we reviewed. In calculating ERR for the 20 projects 
within the compacts that we reviewed, MCC used a 20-year time frame for 
9 projects and used different time frames for the other 11 projects. In 
2 of the 11 projects, applying a 20-year time frame, as MCC had done in 
similar projects, would reduce the ERR below the level MCC set as the 
minimum acceptable ERR. At the compact level, MCC's use of varying time 
frames did not significantly affect the results of the ERR 
calculations. MCC used varying methods to account for the costs of same-
sector projects, although its approaches to determining project 
benefits were generally similar. MCC also used two different methods to 
calculate compact-level ERR; however, the choice of method did not 
reduce it below the minimum ERR. In three of the four compacts that we 
reviewed, MCC did not retain documentation of the economic analyses 
used to support the investment decision, but continued to modify the 
analyses. MCC has recently begun to standardize elements of its 
economic analyses and centralize its records management. 

MCC identified and corrected analytic errors in its projections of 
impact on income and poverty. In addition, MCC used varying methods for 
projecting compact impact on income and poverty, which affected the 
projections' results. 

* Analytic errors. In responding to our questions about its published 
impact projections, MCC identified analytic errors for three of the 
four compacts[Footnote 8] and, in correcting these errors, generally 
lowered its projected impacts on poverty and income. Correcting these 
errors raised one projection by 5 percent but reduced others by 3 
percent to 96 percent. For example, for Armenia, MCC corrected an 
erroneous baseline, reducing the projected decrease in rural poverty 
from 6 percentage points to 3 percentage points. For Mozambique, MCC 
used a corrected formula and data, reducing its projection of the 
number of persons to be lifted out of poverty in Mozambique in 2015 
from 270,000 to either 27,000 or 56,000, depending on the approach 
selected. According to MCC officials, the revised projections would not 
have affected MCC's decision to recommend signing the compacts. The 
officials noted that MCC's impact projections had not undergone a final 
check for accuracy and validity but that MCC has begun to implement a 
peer review process for compacts currently under development. However, 
MCC has not documented procedures for these reviews. 

* Varying methods. MCC used varying methods to project the four 
compacts' impact on income and poverty, limiting the projections' 
comparability and replicability across compacts. For example, for 
Armenia, El Salvador, and Mozambique, MCC estimated the impact of 
compact projects on income by summing the total benefits of individual 
compact projects and adding them to the income that would have 
prevailed without MCC. For Lesotho, MCC extrapolated its compact 
results from the published results of a World Bank model based on 
elements different from those of the MCC compact. In response to our 
questions, MCC revised its poverty impact projection for Mozambique by 
presenting two estimates of the relationship between income growth and 
poverty--estimates based on either Mozambique country-specific data or 
its initial ad hoc estimate, which MCC stated was consistent with cross-
country experience. Although a number of methods for projecting impact 
are valid, the method chosen can affect the results, and MCC's 
guidelines do not identify preferred methods for these calculations. 
MCC also used varying methods to estimate numbers of beneficiaries for 
the compacts and has not provided specific criteria for defining 
beneficiaries; however, MCC officials reported they are taking steps to 
provide more detailed guidance for estimating beneficiaries. 

To improve the reliability and comparability of its projected ERR and 
economic impacts, we recommend that the CEO of MCC take the following 
actions: 

* adopt and implement written procedures for a secondary independent 
review of the methods and results of its economic analyses and: 

* improve MCC's guidelines by identifying a consistent approach with 
preferred methods for projecting compacts' impact on income and 
poverty. 

In commenting on a draft of this report, MCC concurred with our 
recommendations and outlined steps it is taking, including developing 
standard practices and templates and initiating a peer review process. 
MCC stated that in many cases the inconsistencies we identified were 
technically appropriate. MCC also stated that it disagreed with what it 
saw as our report's implication that ERRs are disconnected from income 
and poverty effects. However, we assert that ERRs do not provide 
information about MCC's impact on the poor and are not an absolute 
measure of income benefits, but a relative measure of benefits in 
relation to costs. We have reprinted MCC's comments, with our response, 
in appendix V. We have incorporated technical clarifications from MCC 
where appropriate. 

Background: 

MCC conducts its economic analyses during the compact development 
process, the first of three compact phases. These analyses include 
establishing the minimum ERR that the compact should achieve, 
projecting the compact's ERR, and estimating the compact's impact on 
income and growth. MCC did not publish ERR information for three of the 
four compacts we reviewed, but its public documents include statements 
of compact impact on income and poverty. Of the four compacts that we 
reviewed--Armenia, El Salvador, Lesotho, and Mozambique--three include 
funding for road projects, and three include funding for water 
projects, among other projects. 

Compact Development Process: 

If MCC determines that a country is eligible for assistance, the 
country may submit a compact proposal, which generally comprises 
several projects. According to MCC guidelines,[Footnote 9] the proposal 
should include economic analyses of proposed projects to demonstrate 
their likely impact on growth and poverty in the country. MCC also may 
provide assistance and feedback in developing the proposal, including 
making grants to facilitate the development of the compact (see fig. 
1). 

Figure 1: MCC Compact Development and Implementation Process: 

[See PDF for image] 

This figure is an illustration of the MCC Compact Development and 
Implementation Process, as follows: 

Development compact: 
* Eligibility determination: 
- Country proposal development; 
* Opportunity memo: 
- MCC's due diligence review; 
* Investment memo: 
- Compact negotiation and MCC Board approval. 

Finalize supplemental agreements: 
* Compact signing: 
- MCC and country complete entry into force requirements. 

Implement compact: 
* Entry into force: 
- MCC authorizes fund disbursement and oversees country implementation 
of compact. 

Source: GAO analysis of MCC data. 

[End of figure] 

After a country submits its proposal, MCC's transaction team for the 
compact conducts a preliminary assessment of the proposal and reports 
its findings in an internal opportunity memo to the MCC investment 
committee. MCC assembles a different transaction team for each compact. 
If the opportunity memo is approved, the team launches a detailed due 
diligence review that includes economic analyses of the proposed 
projects. As members of MCC country transaction teams, MCC's lead 
economists undertake these analyses while working with other members of 
the team and country officials. According to MCC, as part of this 
effort, MCC economists review the preliminary analyses performed by 
country counterparts and consultants. These due diligence reviews, 
including conducting economic analyses, have lasted, on average, 
slightly more than 10 months for the 16 countries with signed compacts 
as of March 31, 2008. 

At the conclusion of due diligence, the transaction team sends an 
investment memo to the MCC investment committee, with recommendations 
based on its assessment of the proposal. MCC notifies Congress 15 days 
prior to beginning negotiations with the country, sending a formal 
congressional notification.[Footnote 10] For the four compacts we 
examined, the congressional notifications included some of the results 
of MCC's economic analyses. If compact negotiations with the eligible 
country are successful, the investment committee submits the proposed 
compact to the MCC Board for approval.[Footnote 11] With the Board's 
approval, MCC and the country sign the compact before completing 
additional agreements--such as a disbursement agreement and procurement 
agreement--and ultimately implementing the projects funded in the 
compact.[Footnote 12] 

Economic Analyses: 

For each compact proposal, MCC and the eligible country conduct 
economic analyses, which MCC uses to inform its decisions to fund 
compacts and report to Congress and the public.[Footnote 13] MCC 
generally conducts these analyses at the compact and project levels 
(see fig. 2). 

Figure 2: Illustrative Examples of MCC Economic Analysis at Compact, 
Project, and Activity Levels: 

[See PDF for image] 

This illustration offers examples of MCC economic analysis at compact, 
project, and activity levels, as follows: 

Compact-level analysis: 
* Aggregate economic rate of return (ERR); 
* Aggregate income and poverty effects; 
* Aggregate number of beneficiaries. 

Project-level analysis: 
* ERR, number of beneficiaries; 
Illustrative examples: 
- Water and sanitation; 
- Road construction. 

Activity level analysis: 
* ERR; 
Illustrative examples: 
- Individual city systems; 
- Individual road segments. 

Sources: GAO analysis of MCC data; Nova Development (clip art). 

[End of figure] 

The economic analyses include projections of compacts' impact, 
including their ERR, the impact on income and poverty, and the 
projected number of beneficiaries. 

* ERR analysis. To provide a basis for assessing the compact and 
project ERRs, MCC sets the minimum acceptable ERR that compacts and 
projects should achieve to be eligible for funding. (See app. II for a 
discussion of MCC's minimum ERRs for the four countries.) If the 
compact or project does not meet the minimum rate, MCC retains the 
discretion to fund it but requires justification based on the specific 
circumstances. 

The ERR analysis compares costs and benefits, where the costs are the 
MCA grants and the country's future recurrent costs--such as 
maintenance expenses--and the benefits are increases in incomes in 
recipient countries. (See fig. 3.) MCC guidelines state that normal 
practice is to calculate ERRs using 10-, 20-, and 30-year time horizons 
to determine project and compact ERRs' sensitivity to varying time 
frames. If the ERR is sensitive to this time horizon, the guidelines 
require that this be noted explicitly. 

Figure 3: Illustration of Net benefits and ERR Calculations: 

[See PDF for image] 

This illustration depicts the following Net benefits and ERR 
calculations: 

Benefits, minus Costs, equals Net benefits, generates ERR. 

Year: 1; 
Annual benefit to the country: Benefits1; 
Expenditures to implement project(s): Costs1; 
Annual average expected rate of return on each dollar of MCC 
assistance: Net benefit1. 

Year: 2-9; 
Annual benefit to the country: [Empty]; 
Expenditures to implement project(s): [Empty]; 
Annual average expected rate of return on each dollar of MCC 
assistance: [Empty]. 

Year: 10; 
Annual benefit to the country: Benefits10; 
Expenditures to implement project(s): Costs10; 
Annual average expected rate of return on each dollar of MCC 
assistance: Net benefit10; 

Year 1-10 yields 10-year ERR. 

Year: 11-19; 
Annual benefit to the country: [Empty]; 
Expenditures to implement project(s): [Empty]; 
Annual average expected rate of return on each dollar of MCC 
assistance: [Empty]. 

Year: 20; 
Annual benefit to the country: Benefits20; 
Expenditures to implement project(s): Costs20; 
Annual average expected rate of return on each dollar of MCC 
assistance: Net benefit20; 

Year 1-20 yields 20-year ERR. 

Year: 21-29; 
Annual benefit to the country: [Empty]; 
Expenditures to implement project(s): [Empty]; 
Annual average expected rate of return on each dollar of MCC 
assistance: [Empty]. 

Year: 30; 
Annual benefit to the country: Benefits30; 
Expenditures to implement project(s): Costs30; 
Annual average expected rate of return on each dollar of MCC 
assistance: Net benefit30; 

Year 1-30 yields 30-year ERR. 

Source: GAO analysis of MCC data. 

Note: The ERR of a project is the discount rate (interest rate) at 
which the present value of the project's cost stream is equal to the 
present value of its benefits stream. 

[End of figure] 

* Impact analysis. MCC's guidelines state that economic analyses should 
quantify the proposed projects' expected beneficiaries and expected 
impact on incomes and on poverty.[Footnote 14] Based on our analysis, 
MCC does not establish minimum thresholds for compact impact. 

Public Reporting: 

While MCC's transaction team reports both ERR and impact projections to 
the MCC investment committee in its investment memo, MCC's public 
reporting for the four compacts in our review generally included 
information about only its projections of compact impact; for these 
four countries MCC published ERR projections for only Armenia. MCC's 
public reporting included projections of compact impact on income and 
poverty for three countries and projections of gross domestic product 
(GDP) growth for one country. For each compact, MCC also estimated the 
number of beneficiaries. (See table 1.) 

Table 1: Compact Impact Indicators Included in MCC's Public Reporting: 

Increased income in compact areas: 
Armenia: [Check]; 
El Salvador: [Check]; 
Lesotho: [Empty]; 
Mozambique: [Check]. 

Increased GDP growth rate: 
Armenia: [Empty]; 
El Salvador: [Empty]; 
Lesotho: [Check]; 
Mozambique: [Empty]. 

Decreased poverty: 
Armenia: [Check]; 
El Salvador: [Check]; 
Lesotho: [Empty]; 
Mozambique: [Check]. 

Number of beneficiaries: 
Armenia: [Check]; 
El Salvador: [Check]; 
Lesotho: [Check]; 
Mozambique: [Check]. 

Source: GAO analysis of MCC compacts. 

[End of table] 

Compact Projects and Funding: 

The types of projects included in the four compacts vary, although El 
Salvador, Lesotho, and Mozambique include water projects[Footnote 15] 
and Armenia, El Salvador, and Mozambique include road projects. The 
compacts provide a total of approximately $1.56 billion in MCA 
assistance.[Footnote 16] (See fig. 4.) 

Figure 4: Summary of Compacts for Armenia, El Salvador, Lesotho, and 
Mozambique: 

[See PDF for image] 

This figure is a map of the world, depicting the locations of Armenia, 
El Salvador, Lesotho, and Mozambique, and providing the following 
information: 

Country: Armenia; 
Compact size: $235.65 million; 
Projects: 
* Rural road rehabilitation; 
* Irrigation; 
Area of intervention: Rural areas. 

Country: El Salvador; 
Compact size: $460.94 million; 
Projects: 
* Education and community infrastructure; 
* Agricultural productivity; 
* Road construction and rehabilitation; 
Area of intervention: Northern zone. 

Country: Lesotho; 
Compact size: $362.55 million; 
Projects: 
* Water and sanitation; 
* Health; 
* Private sector development; 
Area of intervention: Entire country. 

Country: Mozambique; 
Compact size: $506.92 million; 
Projects: 
* Water and sanitation; 
* Land tenure; 
* Farmer income; 
Area of intervention: Four northern provinces. 

Sources: GAO analysis of MCC data; Map Resources (map). 

[End of figure] 

MCC Used Different Time Frames and Methods to Calculate ERRs but Is 
Taking Steps to Increase Consistency: 

MCC used different time frames to calculate project-level ERRs; these 
differences did not affect the ERR significantly in 18 of the 20 
compact projects[Footnote 17] we examined, but two ERRs would fall 
below the minimum ERR if MCC applied the 20-year time frame it used for 
other compacts and similar projects. At the compact level, the 
different time frames did not change the ERR significantly. In 
addition, we found that MCC used varying methods to account for the 
costs of same-sector projects, although its approaches to determining 
benefits were generally similar. MCC also used two different methods to 
calculate compact-level ERR; however, the choice of method did not 
change the ERRs significantly. In some cases, MCC did not fully retain 
its documentation of its economic analyses. MCC has recently taken 
steps to standardize elements of its economic analyses and improve its 
records management and plans to implement additional measures. 

MCC Used Different Time Frames for Project and Compact ERRs: 

In its ERR calculations for 20 projects included in the four compacts 
we reviewed, MCC used a 20-year time frame for 9 projects and used 
different time frames for 11 projects. Nearly all project ERRs that MCC 
initially calculated met the minimum ERR set for each compact.[Footnote 
18] Our analysis shows that for 9 of the 11 ERRs calculated with other 
time frames, recalculating the ERR with a 20-year time frame does not 
lower the ERR below the minimum ERR. However, for two projects, 
recalculating the ERR with a 20-year time frame produces a result below 
the minimum ERR. 

* The ERR for the El Salvador Community Infrastructure project, which 
MCC calculated as slightly below the minimum ERR of 10.8 percent over 
25 years, drops to 9.0 percent when calculated over 20 years. 
Correcting a calculation error further reduces the El Salvador 
project's ERR to 6.8 percent over 20 years--four percentage points 
below the El Salvador minimum ERR. 

* The ERR for the Mozambique roads project, which MCC calculated at 
10.3 percent over 24 years, drops to 8.1 percent over 20 years, below 
the minimum ERR of 8.76 percent. Three of the four individual 
Mozambique roads MCC analyzed as part of this project also fall below 
the minimum ERR at 20 years.[Footnote 19] 

If MCC had applied a 20-year time frame in calculating the ERR for 
these two projects, the projects might have been restructured to 
increase their ERR or MCC would have had to specifically justify the 
exception. MCC does not currently have a policy addressing what steps 
to take in cases where subsequent analysis results in a project ERR 
below the minimum. MCC stated that it would address changes such as 
this on a case-by-case basis depending on the timing of the change 
within the compact development process, and the magnitude of the 
change. MCC officials also noted that the ERR time frame is only one 
aspect of MCC's analysis of the ERR's sensitivity to various factors. 

In addition, MCC used different time frames in calculating ERRs for 
comparable projects in different compacts. MCC calculated water project 
ERRs for El Salvador, Lesotho, and Mozambique over 25, 20, and 21 
years, respectively, and calculated road project ERRs for Armenia, El 
Salvador, and Mozambique over 20, 25, and 24 years, respectively. MCC 
officials told us they explored the ERRs' sensitivity to varying time 
frames, as MCC's guidelines require, but they did not regard as 
prescriptive the guidelines' statement that normal practice is to 
examine 10-, 20-, and 30-year time horizons.[Footnote 20] MCC officials 
also noted that MCC economists have recently committed to the use of a 
default 20-year time horizon for their analyses. MCC also will alter 
this time frame for specific circumstances or projects whose benefits 
have a longer duration--such as education projects and large-scale 
construction projects--or have a shorter or longer physical life 
expectancy. 

At the compact level, our analysis shows that applying a 20-year time 
frame in place of varying time frames does not significantly affect the 
results of the ERR calculations. (See app. III for a summary of our 
comparison of the compact minimum ERR with the compact ERRs that MCC 
reported and our calculations of the 20-year ERRs.) 

MCC Used Varying Methods in Calculating Project and Compact ERRs: 

In calculating project ERRs, we found that MCC used varying methods to 
account for the investment and recurrent cost components of water and 
road sector projects for the four compacts we reviewed. 

* Investment costs. For water projects, MCC counted the cost of 
construction in Lesotho as one lump sum in the first year of the 
analysis but phased in the cost in Mozambique and El Salvador over 
multiple years. Phasing the costs evenly over the 5-year compact in 
Lesotho would have increased the ERR for both the urban and rural water 
projects.[Footnote 21] For road projects, MCC phased the investment 
cost over time for Armenia, El Salvador, and Mozambique. However, in El 
Salvador, MCC accounted for the salvage value[Footnote 22] of the 
project in the last year of the analysis but did not include a similar 
projection in the analysis for Armenia and Mozambique. 

* Recurrent costs. For water projects, MCC counted Lesotho and 
Mozambique's recurrent costs in the years after the original compact 
investment. For El Salvador, MCC counted recurrent costs only during 
the initial 5-year compact period. Including future recurrent costs 
would have reduced the ERR. For road projects, MCC spread recurrent 
costs over time for Armenia, El Salvador, and Mozambique. 

According to our review, MCC's approaches to determining the benefits 
of water and road projects were generally similar. 

* Water projects. MCC generally counted as water project benefits 
increased time available for work, resulting from less time spent 
fetching water or being ill, and lower spending on health care and 
water. 

* Road projects. MCC generally estimated traffic volumes on the roads 
and determined road project benefits based on the savings from reduced 
travel time and vehicle operating costs for existing and generated 
traffic.[Footnote 23] 

In calculating the compact-level ERRs for the four compacts' investment 
memos, MCC used two different methods.[Footnote 24] 

* For Armenia and Mozambique, MCC first determined the net benefits in 
each year for each project and then calculated the compact ERR based on 
the total net benefits. 

* For El Salvador and Lesotho, MCC first determined each project's net 
benefits and ERR and then determined the compact ERR by averaging the 
project ERRs, weighting each ERR according to the project's budgeted 
size relative to the overall compact. 

According to MCC officials, the choice of method depended on the 
preference of the transaction team's lead economist. Our analysis shows 
that using the second method to recalculate each compact ERR reduces it 
by less than 2 percentage points and does not reduce it below the 
minimum ERR. However, these results demonstrate that the choice of 
method influences the compact-level ERR and may affect the 
comparability of ERRs across compacts. (See app. III for details of our 
analysis of MCC compact ERRs.) 

Insufficient Records Management Led to Discrepancies between Investment 
Memos and Underlying Analyses: 

In three of the four cases we reviewed, the projections of ERR reported 
to the MCC investment committee differ from those in MCC's underlying 
spreadsheets. 

* For Armenia, MCC changed its estimate of project benefits after the 
investment memo. 

* For Lesotho, the MCC spreadsheets contain figures calculated using 
different methods or reflecting additional analysis after the 
investment memo. 

* For Mozambique, the calculation of the figures presented in the 
investment memo relied on spreadsheet formula links that were not 
properly updated at the time. According to MCC, when they provided the 
spreadsheets to us, they updated the formulas and overwrote the 
original calculations. 

For these three compacts, MCC's records management did not fully 
preserve the information and analysis used to support the investment 
decision. OMB guidance and GAO guidelines for internal controls both 
note the importance of controls over the information that U.S. agencies 
use to make decisions.[Footnote 25] 

MCC Has Taken Several Steps to Improve the Consistency of ERR Analysis: 

After completing due diligence for the four compacts we studied, and 
during the course of our audit, MCC told us they took or began taking 
steps to increase the consistency of its analyses and improve its 
records management. For example, MCC: 

* committed to using 20 years as the default time frame for calculating 
ERRs and identified projects where this time frame may be altered; 

* began revising its guidance to clarify that ERR sensitivity should be 
tested by varying the time frame of the analysis, rather than 
specifying 10-, 20-, and 30-year analyses; 

* began developing standards for consistently analyzing certain types 
of projects across compacts; 

* established that summing net benefits and costs across all projects 
is the preferred method for calculating a compact-level ERR; and: 

* planned to implement a data management system to centralize its 
records management. 

MCC Made Analytic Errors in Compact Impact Projections and Used Varying 
Methods that Affected the Projections' Results: 

MCC identified and corrected analytic errors in its projections of 
compact impact for Armenia, El Salvador, and Mozambique, generally 
reducing each compact's estimated impact on income and poverty. 
[Footnote 26] According to MCC, the revised calculations would not have 
affected its approval of the compacts, but it will perform peer reviews 
of future impact projections. In addition, MCC used different methods 
for projecting compact impact on income and poverty, limiting the 
estimates' comparability and replicability; its current guidance does 
not address the choice of method for these projections. MCC also used 
different methods to estimate numbers of compact beneficiaries, but 
stated that it is taking steps to provide more detailed guidance for 
estimating number of beneficiaries. 

MCC Identified and Corrected Analytic Errors for Three Compacts: 

In responding to our questions about its impact analyses for Armenia, 
El Salvador, and Mozambique,[Footnote 27] MCC identified a number of 
analytic errors in its projections of impact on income and poverty. MCC 
subsequently corrected these errors, generally reducing the projected 
impacts on income and poverty for each compact. 

* Armenia. MCC determined that it had included road project benefits in 
estimating the income increase from agriculture for Armenia. MCC also 
used the wrong baseline in projecting poverty effects. Correcting these 
errors affected projections of the compact's effect, reducing the 
estimated increase in rural areas' real income from agriculture after 5 
years from 5 percent to 3 percent and lowering the estimated decline in 
Armenia's poverty rate from 6 percentage points to 3 percentage points. 
Figure 5 summarizes MCC's original impact projections for Armenia and 
its revisions after correcting the errors it identified. 

Figure 5: MCC Revisions to Impact Projections for Armenia, Based on Its 
Corrections of Analytic Errors: 

[See PDF for image] 

This figure is a table depicting the following data: 

Armenia: 

Income/growth: Annual income in rural areas in 2010; 
Original calculation: $36 million increase; 
MCC's revised calculation: No change; 
Absolute change: 0; 
Percent change: 0. 

Income/growth: Annual income in rural areas in 2015; 
Original calculation: $133 million increase; 
MCC's revised calculation: No change; 
Absolute change: 0; 
Percent change: 0. 

Income/growth: Real income from agriculture in rural areas at end of 
compact; 
Original calculation: 5 percent increase; 
MCC's revised calculation: 3 percent increase; 
Absolute change: -2 percentage points; 
Percent change: -40%. 

Income/growth: Real income from agriculture in rural areas in 2013[A]; 
Original calculation: 23 percent increase; 
MCC's revised calculation: 9 percent increase; 
Absolute change: -14 percentage points; 
Percent change: -61%. 

Poverty: Rural poverty rate[B]; 
Original calculation: 6 percentage point decrease; 
MCC's revised calculation: 3 percentage point decrease; 
Absolute change: -3 percentage points; 
Percent change: -50%. 

Beneficiaries: 
Original calculation: 750,000; 
MCC's revised calculation: No change; 
Absolute change: 0; 
Percent change: 0. 

Source: GAO analysis of MCC data. 

[A] MCC stated projections of real income increases in Armenia using an 
index, with the baseline set at 100 in 2005. Thus, MCC's initial 
projection of an increase from 2005 baseline index of 100 to 123 in 
2013 corresponds to a 23 percent increase in real income, which was 
later revised to 109, or a 9 percent increase, for the same year. 

[B] MCC stated projections of poverty rate reductions in Armenia using 
a baseline of 32 percent. Thus, MCC's initial projections of a 
reduction in the poverty rate from 2004 baseline of 32 percent to a 
target of 26 percent in year 2013 corresponds to a decline of 6 
percentage points, which was later revised to a reduction to 29 
percent, or a decline of 3 percentage points, for the same year. 

[End of figure] 

* El Salvador. MCC determined that it had made an error in its formula 
for projecting per capita income increases and overestimated income 
increases in its projections of poverty rate reduction for El Salvador. 
In correcting these errors, MCC lowered its projections of income and 
poverty impact. Figure 6 summarizes MCC's original impact projections 
for El Salvador and its revisions after correcting the errors it 
identified. MCC also presented alternative methods for identifying 
beneficiaries, which further reduces the results of its compact-level 
projections. (See app. IV.) 

Figure 6: MCC Revisions to Impact Projections for El Salvador, Based on 
Its Corrections of Analytic Errors: 

[See PDF for image] 

This figure is a table depicting the following data: 

El Salvador: 

Income/growth: Incomes within the region within 5 years; 
Original calculation: 18 percent increase; 
MCC's revised calculation: No change; 
Absolute change: 0; 
Percent change: 0. 

Income/growth: Incomes within the region within 10 years; 
Original calculation: 26 percent increase; 
MCC's revised calculation: No change; 
Absolute change: 0; 
Percent change: 0. 

Income/growth: Annual per capita income of beneficiaries within 5 
years; 
Original calculation: $148 increase; 
MCC's revised calculation: $123 increase; 
Absolute change: -$25; 
Percent change: -17%. 

Income/growth: Annual per capita income of beneficiaries within 10 
years; 
Original calculation: $230 increase; 
MCC's revised calculation: $189 increase; 
Absolute change: -$41; 
Percent change: -18%. 

Poverty: Number of persons for whom poverty is alleviated[A]; 
Original calculation: 150,000 persons; 
MCC's revised calculation: 145,000 persons; 
Absolute change: -5,000 persons; 
Percent change: -3%. 

Poverty: Poverty rate in the Northern Zone within 5 years; 
Original calculation: 11 percentage point decrease; 
MCC's revised calculation: 10 percentage point decrease; 
Absolute change: -1 percentage point; 
Percent change: -9%. 

Poverty: Poverty rate in the Northern Zone within 10 years; 
Original calculation: 17 percentage point decrease; 
MCC's revised calculation: 15 percentage point decrease; 
Absolute change: -2 percentage point2; 
Percent change: -12%. 

Beneficiaries: 
Original calculation: 850,000; 
MCC's revised calculation: No change; 
Absolute change: 0; 
Percent change: 0. 

Source: GAO analysis of MCC data. 

Notes: For El Salvador, MCC calculated with-and without-project 
scenarios and cited these figures in its public documents. To estimate 
compact impact attributable to MCC, we calculated the differences 
between with-and without-project scenarios. 

MCC also presented alternative methods for identifying beneficiaries of 
two education projects, which affect the results of MCC's compact 
impact analysis. (See app. IV.) 

[A] MCC's spreadsheet calculations show a figure of more than 160,000 
Salvadorans lifted out of poverty in year 10 of the project. However, 
MCC's compact summary stated that "the program is projected to directly 
alleviate the poverty of over 150,000 Salvadorans." The compact 
summary's projection of 150,000 is used here. 

[End of figure] 

* Mozambique. MCC determined that it had not updated formula links in 
its analytic spreadsheets for Mozambique. Correcting this error led MCC 
to revise the projected income increase upward by $4 million for 2015 
and downward by $13 million for 2025. In addition, MCC corrected both 
the formula and the data[Footnote 28] used to calculate the effect of 
this income increase on poverty in Mozambique, presenting two 
alternative approaches for estimating poverty elasticity.[Footnote 29] 
In making these changes, MCC lowered the projected decline in the 
poverty rate as well as the number of persons likely to be lifted out 
of poverty because of the compact. For example, MCC originally 
projected a 7 percent reduction in Mozambique's poverty rate in 2015; 
using the alternative approaches for estimating poverty elasticity, MCC 
projected a poverty rate reduction of either 0.6 percent or 2 percent 
in 2015. Likewise, MCC originally projected that 270,000 people would 
be lifted out of poverty in Mozambique in 2015; using the alternative 
approaches, MCC projected that either 27,000 persons or 56,000 persons 
would be lifted out of poverty in 2015. Figure 7 summarizes MCC's 
original impact projections for Mozambique and its revisions after 
correcting errors and presenting alternative approaches for estimating 
poverty elasticity. 

Figure 7: MCC Revisions to Impact Projections for Mozambique, Based on 
Its Corrections of Analytic Errors: 

[See PDF for image] 

This figure is a table depicting the following data: 

Mozambique: 

Income/growth: Income in 2015; 
Original calculation: $75 million increase; 
MCC's revised calculation: $79 million increase; 
Absolute change: +$4 million; 
Percent change: +5%. 

Income/growth: Income in 2025; 
Original calculation: $180 million increase; 
MCC's revised calculation: $167 million increase; 
Absolute change: -$13 million; 
Percent change: -7%. 

Poverty: Poverty rate in 2015; 
Original calculation: 7 percent decrease; 
MCC's revised calculation: 0.6 decrease (lower estimate); 2 percent 
decrease (higher estimate); 
Absolute change: -6.4 percentage points (lower estimate); -5 percentage 
points (higher estimate); 
Percent change: -91% (lower estimate); -66% (higher estimate). 

Poverty: Poverty rate in 2025; 
Original calculation: 16 percent decrease; 
MCC's revised calculation: 0.7 decrease (lower estimate); 3 percent 
decrease (higher estimate); 
Absolute change: -15.3 percentage points (lower estimate); -13 
percentage points (higher estimate); 
Percent change: -96% (lower estimate); -83% (higher estimate). 

Poverty: Number of persons lifted out of poverty in 2015; 
Original calculation: 270,000 persons; 
MCC's revised calculation: 27,000 persons (lower estimate); 56,000 
persons (higher estimate); 
Absolute change: -243,000 persons (lower estimate); -214,000 persons 
(higher estimate); 
Percent change: -90% (lower estimate); -79% (higher estimate). 

Poverty: Number of persons lifted out of poverty in 2025; 
Original calculation: 440,000 persons; 
MCC's revised calculation: 32,000 persons (lower estimate); 43,000 
persons (higher estimate); 
Absolute change: -408,000 persons (lower estimate); -397,000 persons 
(higher estimate); 
Percent change: -93% (lower estimate); -90% (higher estimate). 

Beneficiaries: 
Original calculation: 5 million by 2015; 
MCC's revised calculation: No change; 
Absolute change: 0; 
Percent change: 0. 

Source: GAO analysis of MCC data. 

Notes: For Mozambique, MCC's public documents stated projections of 
poverty (1) with the MCC compact and (2) without the MCC compact. To 
estimate compact impact attributable to MCC, we calculated the 
differences between with-and without-project figures. 

In addition to correcting its formula used to calculate the effect of 
increased income on poverty in Mozambique, MCC presented alternative 
approaches for estimating poverty elasticity. MCC's poverty impact 
projections using both alternatives are shown here. 

[End of figure] 

Revised Projections Would Not Have Changed Funding Decisions, but 
Future Compact Impact Projections Will Undergo Review: 

According to MCC officials, the revisions to its initial calculations 
of compact impact on income and poverty for Armenia, El Salvador, and 
Mozambique would not have changed MCC's decision to recommend each 
compact to the MCC Board. The officials emphasized that MCC's economic 
impact projections are one of many aspects of the due diligence process 
that inform its compact investment decisions. A senior official stated 
that MCC's compact-level analysis is separate from the decision about 
whether to invest in specific projects; the two levels of analysis are 
connected but play different roles in MCC's decision making. MCC 
officials also told us they used conservative data and assumptions to 
project compacts' impact on income, growth, and poverty. However, 
future public statements would reflect MCC's corrections and revised 
analyses.[Footnote 30] 

In addition, MCC officials said that although lead economists 
independently review the economic projections performed by others on 
the transaction team for each country, calculations and assumptions 
performed by the lead economists for the compacts we reviewed did not 
undergo a final check for accuracy and validity. The officials stated 
that such a review might have caught the errors that MCC later 
identified and corrected.[Footnote 31] They further stated that, with 
fewer compacts undergoing due diligence in the future, MCC will have 
the staff capacity to ensure that such reviews are performed. MCC 
officials told us they have begun to implement this peer review for 
compacts currently under development. However, MCC has not documented 
its procedures for conducting these reviews, so it is unclear what 
criteria and level of detail the peer review includes. 

MCC Used Varying Methods to Project Compact Impact on Income and 
Poverty: 

MCC used different methods to project the four compacts' impact on 
income and poverty, limiting the estimates' comparability and 
replicability. Although the method chosen can affect the results, MCC 
has not provided preferred methods, or guidelines for selecting a 
method, for these calculations. 

MCC Used Varying Methods for Economic Growth Projections: 

MCC used a different method to estimate impact on economic growth for 
Armenia, El Salvador, and Mozambique than it used for Lesotho. As a 
result, the estimates are not comparable. 

* For Armenia, El Salvador, and Mozambique, MCC estimated the impact of 
compact projects on aggregate income or income growth.[Footnote 32] 
This method entailed summing the total benefits of individual compact 
projects and adding them to the income that would have prevailed 
without MCC. 

* For Lesotho, MCC estimated the impact of the compact on the country's 
GDP growth rate using the published results from a World Bank 
simulation model. According to MCC, the World Bank's model showed that 
a public and private investment of about $100 million would lead to an 
increase in GDP growth rate of 3.75 percentage points. Reasoning that 
its projected investment in the Lesotho compact is analogous to the 
investment in the World Bank model, MCC extrapolated from the World 
Bank's estimate to project that the compact, if successfully 
implemented, would nearly double Lesotho's GDP growth rate.[Footnote 
33] 

According to MCC, its use of GDP growth impacts based on the published 
World Bank model's results was appropriate, in that the model was 
consistent with the scale and type of public infrastructure investments 
proposed under the compact. MCC also noted that the World Bank model 
provided more information than did MCC's analyses for the other 
compacts.[Footnote 34] However, our analysis of the World Bank report 
[Footnote 35] shows some differences between MCC's and the World Bank's 
interventions. For example, whereas MCC's compact comprises only 
increased public investment in infrastructure, the World Bank report's 
model projects the combined effect of three elements: increase in 
garment exports, growth in commercial agriculture, and increase in 
public investment in infrastructure.[Footnote 36] Because the model's 
three elements have different and interlinked effects, MCC's 
extrapolation of the effect of one of these elements, public investment 
in infrastructure, to its compact requires a number of assumptions that 
cannot be validated. 

Although MCC guidelines call for projecting compacts' impact on 
economic growth starting from the project level, MCC has not provided 
preferred methods, or guidelines for selecting a method, for estimating 
compacts' impact on GDP growth rate. 

MCC Used Varying Methods to Project Poverty Reduction: 

MCC estimated compacts' impact on the poverty rate for Armenia, El 
Salvador, and Mozambique based on the responsiveness, or elasticity, of 
the poverty rate to changes in income.[Footnote 37] MCC uses several 
income measures in this analysis. For Armenia, MCC estimated poverty 
reduction based on its elasticity with respect to agricultural value 
added to the local economy; for El Salvador, with respect to increased 
annual national GDP; and for Mozambique, with respect to increased 
annual regional GDP. [Footnote 38] 

Each of these measures can be a valid measure of income and 
corresponding elasticity of the poverty rate to income changes. 
However, the types of data used to estimate poverty elasticity can 
significantly affect poverty impact projections. For example, in 
projecting the Mozambique compact's impact on poverty, MCC initially 
used an ad hoc estimate of elasticity rather than an elasticity based 
on historical income and poverty data specific to Mozambique; according 
to MCC officials, the ad hoc elasticity was consistent with 
conservative cross-country estimates. In a response to our questions 
about its original analysis, MCC projected revised poverty impacts for 
Mozambique in two ways, in both cases correcting for analytic errors 
discussed above. For one revised projection, MCC used its initial 
elasticity based on cross-country data. For the second projection, MCC 
used an alternative elasticity calculated from Mozambique data, which 
resulted in lower poverty reduction estimates.[Footnote 39] The 
resulting estimates for the number of persons to be lifted out of 
poverty were 56,000 versus 27,000 persons by 2015 and 43,000 versus 
32,000 persons by 2025, respectively. 

MCC's guidelines call for estimating compacts' impact on reducing the 
poverty rate but do not discuss or prioritize various possible methods 
for estimating poverty reduction.[Footnote 40] Because the methods 
chosen for each compact can vary, the resulting estimates of poverty 
reduction may not be comparable or replicable across countries. 

MCC Used Different Methods to Estimate Beneficiaries and Is Taking 
Steps to Strengthen Guidance: 

MCC used two different methods to estimate the number of beneficiaries 
it reported for the compacts we reviewed: (1) summing beneficiaries 
from the individual projects, with adjustments for double counting and 
timing of benefits, and (2) counting the intervention area's entire 
population as beneficiaries. 

* For Armenia and Mozambique, MCC counted the population of the areas 
where the compact is being implemented as project-specific 
beneficiaries and summed these project totals to produce the reported 
number of beneficiaries. MCC corrected for double counting of 
beneficiaries in areas where more than one project was implemented. For 
Mozambique, MCC projected a gradual increase in the number of 
beneficiaries as benefits accrue over time, whereas for Armenia it did 
not. 

* For El Salvador, MCC reported that the entire population of the 
intervention area would benefit,[Footnote 41] although the project- 
specific beneficiary counts did not sum to the area's population. 

* For Lesotho, assuming that the compact would have economywide impact, 
MCC reported the country's entire population as beneficiaries although 
estimates of project beneficiaries were available. 

MCC guidelines define program beneficiaries as individuals or groups 
that derive economic gains from MCC investments. However, the 
guidelines do not explicitly define economic gain or provide criteria 
for counting beneficiaries based on the amount of accrued benefits or 
the degree of exposure to compact-provided services. As a result, it is 
unclear whether MCC's transaction teams would count as equal 
beneficiaries a person using a compact-related benefit once and a 
person using the same benefit regularly or whether MCC would apply an 
equivalence scale to adjust for varying use of the benefit. For 
example, for El Salvador education programs, MCC first counted as 
beneficiaries all individuals entering the programs--assuming they 
would experience some income increases, and later revised the method to 
include only individuals completing the programs, assuming they would 
experience measurable income increases. The transaction team's choice 
of method for defining beneficiaries, in turn, affects the compact- 
level projections of impact on income and poverty. For example, MCC's 
estimate of the number of persons to be lifted out of poverty by the 
compact as a whole ranges from 145,000 in the first case to 83,000 in 
the second, depending on the assumptions used. (See app. IV for more 
details.) 

According to MCC officials, MCC is in the process of making its 
guidance regarding program beneficiaries more operational by 
establishing criteria for defining beneficiaries. The officials also 
noted that increases in income are considered a starting point for 
measuring economic gains and therefore determining the number of 
compact beneficiaries. 

Conclusions: 

As MCC works in some of the world's poorest countries, it has taken on 
the difficult challenge of projecting its compacts' likely ERR, and 
economic growth and poverty effects. However, our analysis shows that 
analytic errors affected the results, and the choice of analytical 
method can change the results of economic projections. Without a 
consistent approach and preferred methods for its due diligence 
economic analyses, MCC's transaction teams and individual team members 
have used different methods to identify compact results. This 
heterogeneity means that different teams could reach different 
conclusions about the worthiness of individual projects or compacts 
based on the method chosen. MCC has refined its guidance over time, and 
continues to do so. Further refinements that include formal procedures 
for reviewing the results of due diligence economic analyses and 
establishing a consistent approach with preferred methods--which could 
be modified if required by specific country conditions--would help MCC 
reduce the likelihood of errors and provide a common MCC lens for 
projecting compacts' ERRs and impacts. This in turn would enhance the 
reliability and comparability of the information MCC uses internally 
for decision making as well as the information it provides to Congress 
and the public for oversight of MCC's activities. 

Recommendations for Executive Action: 

To improve the reliability and comparability of its projected ERR and 
economic impacts, we recommend that the CEO of MCC take the following 
actions: 

* Adopt and implement written procedures for a secondary independent 
review of the methods and results of its economic analyses. 

* Improve MCC's guidelines by identifying a consistent approach with 
preferred methods for projecting compacts' impact on income and 
poverty. 

Agency Comments and Our Evaluation: 

MCC provided comments regarding a draft of this report, which we have 
reprinted, with our response, in appendix V. MCC also provided 
technical clarifications, which we have incorporated as appropriate. 

In commenting on a draft of this report, MCC concurred with our 
recommendations and outlined related steps that it is taking--including 
developing standard practices and templates, initiating an independent 
peer review process, and posting its economic analyses on the Internet. 
MCC commented that there is not a single cost-benefit standard practice 
and that in many cases the inconsistencies we identified were 
technically appropriate. We acknowledge that there is not a single 
standard practice; however, we maintain that consistent analytic 
approaches are needed to ensure the reliability of MCC's analyses. 
According to MCC, the choice of analytic time frames was in some cases 
based on the judgment of the individual lead economist rather than on 
established criteria. The steps that MCC has stated it is taking--using 
a default time frame for analyses and subjecting the time frame 
decision to peer review--will help to enhance the comparability across 
compacts of the time frame of ERR calculations. MCC also stated that it 
disagreed with what it saw as our report's implication that ERRs are 
disconnected from income and poverty effects. However, we assert that 
ERRs do not provide information about MCC's impact on any specific 
population group, including the poor. Also, ERRs are a relative measure 
of benefits in relation to costs that can fluctuate owing to changes in 
costs alone--and therefore cannot be considered absolute measures of 
impact on income. 

We are sending copies of this report to interested congressional 
committees as well as the Chief Executive Officer of MCC. We will also 
make copies available to others upon request. In addition, this report 
will be available at no charge on the GAO Web site at [hyperlink 
http://www.gao.gov]. 

If you or your staff have any questions about this report, please 
contact David Gootnick at (202) 512-3149 or gootnickd@gao.gov. Contact 
points for our Offices of Congressional Relations and Public Affairs 
may be found on the last page of this report. GAO staff who made major 
contributions to this report are listed in appendix VI. 

Sincerely yours, 

Signed by: 

David Gootnick: 
Director: 
International Affairs and Trade: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

At the request of the chairman of the House Committee on Foreign 
Affairs, we assessed the Millennium Challenge Corporation's (MCC) 
economic analyses, including its projections of the compact's economic 
rate of return (ERR), changes in income and poverty, and the number of 
beneficiaries. To carry out this review, we reviewed MCC compacts with 
four countries: Armenia, El Salvador, Lesotho, and Mozambique. We 
selected these countries based on their percentage of total compact 
funding, the recentness of the compact, and geographical 
representation. These four compacts make up a total of about $1.56 
billion in compact assistance. When we began our work in August 2007, 
this represented about 41 percent of the $3.85 billion MCC had set 
aside for 13 compacts[Footnote 42] and included the most recent 
compacts signed in Eurasia, Latin America, and Africa. We chose to 
include two African countries in acknowledgement of MCC's focus on 
Africa, which accounted for 7 of MCC's 13 compact countries and more 
than half of the total compact value. 

MCC's compact-level projections are influenced by the specific data 
points and assumptions used in project-and activity-level analysis. We 
determined that the data we used were sufficiently reliable for the 
purposes of our analysis; however, within our scope of work, we did not 
independently evaluate the thousands of data points and assumptions at 
the project and activity levels or the sensitivity analyses used in 
MCC's economic analyses and therefore are not assessing all aspects of 
the validity of MCC's ERR or impact projections. We also did not assess 
MCC's progress in implementing these compacts, and therefore its 
progress toward achieving projected compact results. 

To assess MCC's compact ERR projections, we reviewed MCC's public and 
internal documents for statements of compact and project ERR. When we 
began our review, MCC had published the compact and project ERRs for 
only Armenia.[Footnote 43] However, for all four countries, MCC stated 
ERRs in the internal investment memo. Next, we consulted MCC's guidance 
for ERR analyses as well as the spreadsheets provided by MCC to support 
its calculations of ERR and document its work. We reviewed MCC guidance 
on calculating ERR and minimum ERRs and compared MCC's internal 
documentation and spreadsheets to elements of this guidance. We 
examined the spreadsheets to determine how MCC aggregated project-level 
ERRs into one compact-level ERR for the four countries in our review 
and identified the two approaches---summing net benefits and using 
weighted averages--that MCC used. To determine the effects of MCC's 
alternative approach on the compact ERR, we used MCC's cost and benefit 
data to calculate the alternative ERRs. We also used these data to 
calculate compact-and project-level ERRs over different time horizons 
to explore the sensitivity of the ERRs to differing time horizons. To 
determine how MCC approached same-sector projects in different 
compacts, we studied road and water projects because these were each in 
three of the four compacts. We then compared the broad approaches MCC 
used in assessing the costs and benefits associated with these 
projects. We interviewed MCC economists and other officials regarding 
MCC's ERR analysis to further discuss MCC's approaches and clarify 
aspects of MCC's analysis. We also consulted Office of Management and 
Budget (OMB) guidance and GAO guidance[Footnote 44] on establishing and 
implementing internal controls to inform our assessment of MCC's 
processes for developing and maintaining information for the purpose of 
management decision making. 

To assess MCC's projections of the number of compact beneficiaries, and 
changes in income and poverty, we first compiled and analyzed MCC's 
public documents for statements of compact impact. These documents 
included MCC's compacts, compact summaries, congressional notifications 
and budget justifications, and MCC's annual reports. MCC makes all of 
these documents available online.[Footnote 45] We identified statements 
that fell into four categories of economic projections: impact on 
national or regional income, impact on national or regional gross 
domestic product (GDP) growth rate, impact on national or regional 
poverty, and number of compact beneficiaries. MCC reviewed and 
concurred with our compilation and summary of these statements of 
compact impact. We also reviewed MCC's guidance on projecting these 
economic impacts. We reviewed the spreadsheets for each compact that 
MCC used to conduct and document its economic analyses. After an 
initial examination of these spreadsheets, we met with MCC economists 
and other officials to discuss MCC's methods and calculations for 
projecting compact impact. MCC officials also provided responses to our 
questions in written form. In these responses, MCC revised its initial 
calculations of its impacts in Armenia, El Salvador, and Mozambique and 
also presented alternative methods. We then reviewed the updated 
information that MCC submitted to us and calculated the magnitude of 
the difference in MCC's original statements and those supported by its 
revised analyses. In the case of Lesotho, we reviewed the World Bank 
Country Economic Memo regarding the economic growth model that MCC used 
to estimate compact impact in Lesotho. We also reviewed World Bank 
country assistance strategies and International Monetary Fund Article 
IV consultation reports and Poverty Reduction Strategy Papers to 
improve our contextual understanding of each country's compact program. 
Finally, we consulted OMB guidance and GAO guidance on establishing and 
implementing internal controls to inform our assessment of MCC's 
internal processes. 

We conducted this performance audit from August 2007 to June 2008, in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 

[End of section] 

Appendix II: MCC Minimum Acceptable ERRs: 

MCC has issued two definitions of the minimum acceptable ERR, which MCC 
refers to as the hurdle rate, for compacts and projects. In its initial 
guidelines, issued in April 2005, MCC defined the hurdle rate as the 
average of the country's real growth rates for the previous 3 years. 
[Footnote 46] The January 2006 guidelines did not set a hurdle rate. 
The November 2006 guidelines defined the hurdle rate as the greater of 
(1) two times the average real growth rate of GDP for the country for 
the most recent 3 years for which data are available or (2) two times 
the average real growth rate of GDP for all of the MCC eligible 
countries for each country for the most recent 3 years for which data 
are available.[Footnote 47] The November 2006 guidelines also state 
that the hurdle rate may not be higher than 15 percent. 

In setting the hurdle rate for each compact that we reviewed, MCC 
applied the definition of the rate from the guidelines current at the 
time. For Armenia and Mozambique, MCC used its April 2005 guidance. For 
El Salvador, according to MCC officials, and for Lesotho, MCC used the 
definition in the November 2006 guidelines. (See table 2.) 

Table 2: MCC Compact Hurdle Rates and Hurdle Rate Definition: 

Compact: Armenia; 
MCC hurdle rate[A]: 12.5 percent; 
Hurdle rate definition: April 2005 guidance: The country's average real 
growth rate for the past 3 years. 

Compact: Mozambique; 
MCC hurdle rate[A]: 8.76 percent; 
Hurdle rate definition: April 2005 guidance: The country's average real 
growth rate for the past 3 years. 

Compact: El Salvador; 
MCC hurdle rate[A]: 10.8 percent; 
Hurdle rate definition: November 2006 guidance: Based on two times the 
average real growth rate of GDP for all of the MCC eligible countries 
for each country for the most recent 3 years for which data are 
available. 

Compact: Lesotho; 
MCC hurdle rate[A]: 10.8 percent; 
Hurdle rate definition: November 2006 guidance: Based on two times the 
average real growth rate of GDP for all of the MCC eligible countries 
for each country for the most recent 3 years for which data are 
available. 

Source: MCC documents. 

[A] For Armenia, Mozambique, and Lesotho, the hurdle rate was stated in 
the investment memo. The investment memo for El Salvador did not state 
the hurdle rate, but MCC officials reported that a hurdle rate of 10.8 
percent, based on the November 2006 definition, was applied. 

[End of table] 

In two cases, the ERRs that MCC initially calculated did not meet the 
compact hurdle rates. MCC originally calculated the El Salvador 
Community Infrastructure Project ERR as slightly below the hurdle rate 
at 10.4 percent over 25 years. MCC's investment memo for Lesotho stated 
that the ERR for the Rural Water Project was under 6 percent. 
Subsequent revisions to the Lesotho analysis reduced the projected ERR 
to less than 1 percent. The Lesotho investment memo discusses the 6 
percent ERR but notes a different view within the transaction team that 
other benefits are not captured by the economic analyses. 

[End of section] 

Appendix III: Compact ERRs: 

We assessed the sensitivity of MCC's compact-level ERRs to the use of 
varying time frames and varying methods for calculating the compact- 
level ERR. We found in each case that the variance does affect the 
results, but does not change the ERRs to below the applicable hurdle 
rate. 

Time Frames: 

MCC used different time frames to calculate and report compact ERRs. 

* For Armenia, MCC calculated and reported a 20-year time frame for the 
compact and both of its projects. 

* For El Salvador, MCC calculated a 25-year ERR for most projects and 
for some projects reported the ERR as a 25-year ERR in its investment 
memo. MCC officials explained that their El Salvador country 
counterparts originally performed the analysis over 25 years and that 
MCC's lead economist determined this to be a reasonable approach. 

* For Lesotho, MCC used a 20-year ERR for all but one ERR calculation 
but did not state the time frame of the calculations in the investment 
memo. 

* For Mozambique, MCC used a 24-year ERR for most compact projects. 
According to MCC officials, MCC intended to present a 25-year ERR, but 
a delay in the beginning of compact implementation pushed the compact's 
time horizon 1 year into the future and reduced the time frame for the 
analysis to 24 years. The Mozambique investment memo does not state the 
number of years used for the time frame of the ERR calculation. 

Table 3 summarizes our comparison of the compact hurdle rates with the 
ERRs that MCC reported and our calculations of the 20-year ERRs. Our 
analysis shows that applying a 20-year time frame in place of varying 
time frames does not significantly affect the results of the ERR 
calculations by lowering it below the hurdle rate. 

Table 3: Comparison of MCC Compact ERRs Stated in Investment Memo with 
ERRs Calculated Using a 20-Year Time Frame: 

Armenia: 
Hurdle rate: 12.5 percent; 
ERR reported in investment memo[A]: 25 percent over 20 years; 
GAO calculation: 20-year ERR[B]: 27.4 percent. 

El Salvador[C]: 
Hurdle rate: 10.8 percent; 
ERR reported in investment memo[A]: 21 percent over 25 years; 
GAO calculation: 20-year ERR[B]: 17.0 percent[D]. 

Lesotho: 
Hurdle rate: 10.8 percent; 
ERR reported in investment memo[A]: 16.3 percent, period of years not 
specified; 
GAO calculation: 20-year ERR[B]: 15.9 percent[E]. 

Mozambique: 
Hurdle rate: 8.76 percent; 
ERR reported in investment memo[A]: 19.6 percent, period of years not 
specified; 
GAO calculation: 20-year ERR[B]: 17.2 percent. 

Source: GAO analysis of MCC economic analyses. 

[A] In some cases, MCC's underlying spreadsheets stated different ERRs 
than those reported in the investment memo. However; for Armenia, the 
revision to a 27.4 percent ERR occurred after the investment memo. We 
based our calculations on the spreadsheet calculations of the ERRs: for 
Armenia, 27.4 percent over 20 years; for El Salvador, 20.6 percent over 
25 years; for Lesotho, 16.4 percent over 20 years; and for Mozambique, 
18.7 percent over 24 years. 

[B] These alternative ERRs use the method used by MCC for the original 
ERR--that is, weighted averages for El Salvador and Lesotho and sums of 
net benefits for Armenia and Mozambique. 

[C] MCC provided updated spreadsheets for its education and water and 
sanitation projects that reduced some expected benefits and incorrectly 
entered some compact administration costs for another project. Using 
the weighted average method originally used by MCC, the ERR with these 
revisions would be 16.3 percent at 20 years. 

[D] The formula used to calculate the ERR for the Productive 
Development project in El Salvador returns an error when used to 
calculate the ERR for 20 years. However, since the total net benefits 
for the project at 20 years are negative, the ERR also would be 
negative. We used a zero ERR over 20 years in place of a negative ERR 
for this project in calculating the compact ERR. The total net benefits 
and ERR for the Productive Development project become positive at 21 
years. 

[E] MCC originally calculated the Lesotho compact-level ERR using a 
weighted average of ERRs, some of which did not match the ERRs in its 
underlying spreadsheet calculations and the investment memo. We have 
used the ERRs in these underlying spreadsheets to calculate the 20-year 
ERR. MCC also originally used a 30-year ERR for the Metolong Dam 
project to calculate the 20-year compact-level ERR. We have used the 20-
year ERR for the Metolong Dam for this 20-year compact ERR calculation. 

[End of table] 

Methods: 

Figure 8 illustrates the two different methods--summing net benefits 
and calculating a weighted average--that MCC used to determine a 
compact-level ERR. 

Figure 8: MCC's Alternative Methods for Calculating Compact ERR: 

[See PDF for image] 

This figure is an illustration of MCC's alternative methods for 
calculating Compact ERR, as follows: 

Alternative 1: Sum of net benefits across all projects, for each year: 

Net benefits project A, plus Net benefits project B, plus Net benefits 
project C, equal Compact net benefits; Calculate compact ERR, yields 
Compact ERR. 

Alternative 2: Weighted average of project ERRs: 

ERR Project A, times Cost share A, equals Weighted project ERR A; 
ERR Project B, times Cost share B, equals Weighted project ERR B; 
ERR Project C, times Cost share C, equals Weighted project ERR C; 
Combined to calculate overall weighted average, yields Compact ERR. 

Source: GAO synthesis of MCC information. 

[End of figure] 

We compared the results of the compact ERR calculation using different 
methods and determined that the choice of method affects the results, 
but does not reduce the ERR below the hurdle rate. See table 4 for 
details of our analysis. 

Table 4: Compact ERR Using Alternative Methods: 

Country: Armenia; 
ERR reported in investment memo: 25 percent over 20 years; 
ERR from underlying spreadsheets[A]: 27.4 percent over 20 years; 
ERR using alternative method: 25.6 percent over 20 years; 
Difference: - 1.8. 

Country: El Salvador; 
ERR reported in investment memo: 21 percent over 25 years; 
ERR from underlying spreadsheets[A]: 20.6 percent over 25 years; 
ERR using alternative method: 19.6 percent over 25 years; 
Difference: -1. 

Country: Lesotho; 
ERR reported in investment memo: 16.3 percent, period of years not 
specified; 
ERR from underlying spreadsheets[A]: 16.4 percent over 20 years; 
ERR using alternative method: 15.4 percent over 20 years; 
Difference: -1. 

Country: Mozambique; 
ERR reported in investment memo: 19.6 percent, period of years not 
specified; 
ERR from underlying spreadsheets[A]: 18.7 percent over 24 years; 
ERR using alternative method: 17.3 percent over 24 years; 
Difference: -1.4. 

Source: GAO analysis of MCC economic analyses: 

[A] The ERR in MCC's underlying spreadsheets for Armenia, Lesotho, and 
Mozambique was different from that stated in its investment memos. 
However; for Armenia, the revision to a 27.4 percent ERR occurred after 
the investment memo. In order to determine the ERRs from an alternative 
method, we needed to use these spreadsheets; therefore they are the 
appropriate comparison for determining the magnitude of the change in 
results from using the alternative method. El Salvador's reported ERR 
was rounded in the investment memo. 

[End of table] 

[End of section] 

Appendix IV: Impact of Alternative Beneficiary Counts on El Salvador 
Compact Impact Projections: 

During the course of our engagement, MCC changed its method for 
determining the beneficiaries and benefits of education projects in El 
Salvador. In its initial calculations for both projects, MCC assumed 
that all students entering the programs would experience an increase in 
income. In its revised calculations for the formal education project, 
MCC assumed that all nongraduates would have zero income increases--an 
underestimate of the impact--because of lack of data. For the informal 
education project, MCC estimated an income increase of more than 200 
percent for those who obtain employment, based on a study conducted by 
the training institute in El Salvador. However, MCC assumed that many 
entering students would not complete the program or obtain employment 
and their income would therefore not increase. (See table 5.) 

Table 5: MCC's Compact-Level Impact Estimates for El Salvador, with 
Alternative Assumptions for Estimating Beneficiaries of Education 
Projects: 

Results of compact-level analysis: 

Incomes in the region: 5 years; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals entering the program will experience increase in income: 18 
percent increase; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals completing the program[B] will experience increase in 
income[C]: 10 percent increase. 

Incomes in the region: 10 years; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals entering the program will experience increase in income: 26 
percent increase; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals completing the program[B] will experience increase in 
income[C]: 13 percent increase. 

Annual per capita income of beneficiaries: 5 years; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals entering the program will experience increase in income: 
$123 increase; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals completing the program[B] will experience increase in 
income[C]: $73 increase. 

Annual per capita income of beneficiaries: 10 years; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals entering the program will experience increase in income: 
$189 increase; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals completing the program[B] will experience increase in 
income[C]: $97 increase. 

Number of persons for whom poverty is alleviated: 10 years; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals entering the program will experience increase in income: 
145,000 persons; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals completing the program[B] will experience increase in 
income[C]: 83,000 persons. 

Poverty rate in the Northern Zone: 5 years; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals entering the program will experience increase in income: 10 
percentage point decrease; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals completing the program[B] will experience increase in 
income[C]: 6 percentage point decrease. 

Poverty rate in the Northern Zone: 10 years; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals entering the program will experience increase in income: 15 
percentage point decrease; 
Assumption for estimating beneficiaries of education projects[A]: 
Individuals completing the program[B] will experience increase in 
income[C]: 8 percentage point decrease. 

Source: GAO analysis of MCC data. 

Note: These figures also reflect MCC's corrections of formula errors. 

[A] MCC's compact with El Salvador comprises a formal education project 
and an informal education project. We consider these two projects 
jointly for the purposes of this summary table. 

[B] For the informal education project, MCC estimated the number of 
individuals who complete the program and obtain employment. 

[C] The revised projections also reflect MCC changes to (1) the number 
of beneficiaries of the productive development project, and (2) 
projected income increases resulting from the water and sanitation 
project. 

[End of table] 

[End of section] 

Appendix V: Comments from the Millennium Challenge Corporation: 

Note: GAO comments supplementing those in the report text appear at the 
end of this appendix. 

Millennium Challenge Corporation: 
Reducing Poverty Through Growth: 
875 Fifteenth Street NW: 
Washington, DC 20005-2221: 
(202)521-3600: 
(202521-3700 (Fax): 
[hyperlink, http://www.mcc.gov]: 

May 23, 2008 

Mr. David B. Gootnick: 
Director, International Affairs and Trade: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20001: 

Dear Mr. Gootnick: 

Thank you for the opportunity to respond to GAO's draft report, MCC: 
Independent Reviews and Consistent Approaches Will Strengthen 
Projections of Program Impact. 

MCC appreciates GAO's recognition of the ground-breaking nature of 
MCC's projections of program impact, reflected in the report's 
conclusion: "As it works in some of the world's poorest countries, MCC 
has taken on the difficult challenge of projecting its compacts' likely 
ERR, and economic growth and poverty effects." MCC concurs with GAO's 
overall recommendations for enhanced guidance and a secondary 
independent review of economic analyses. Indeed, as GAO notes, MCC has 
already "taken steps to standardize elements of its economic analyses 
and centralize its records management" in addition to establishing an 
independent peer review process. 

Independent Reviews: 

Economic analysis, conducted for the purpose of informing MCC 
investment decisions, already undergoes a series of reviews as part of 
our standard practice. Much of the initial analysis is performed by 
professional counterparts in our partner countries and by private 
consultants hired by them or by MCC. In every case, MCC economists 
represent an independent review of that preliminary analysis. 

MCC supports an additional layer of review and, as the report notes, 
has already established an independent peer review process. This 
process, which is managed by the Chief Economist, provides a review of 
the models, formulae and parameters used to estimate the expected 
impact of programs. MCC has already used the new process to review 
forecasts for one compact program currently under consideration, and 
has begun the process for a second country. [See comment 1] 

MCC has taken the unprecedented step of making our economic analysis 
accessible on our public website, opening our work to the broader 
scrutiny of interested NGOs, academics, and private analysts, which we 
believe will generate useful feedback further strengthening our 
analysis. MCC has posted the economic models and resulting impact 
estimates for nine countries, along with explanatory descriptions and 
documentation. We will post the analysis for all other compacts before 
the end of this fiscal year. Independent observers, including senior 
officials at Center for Global Development and Bread for the World 
Institute, have lauded this initiative as setting a new standard for 
transparency in government. [See comment 1] 

Consistency in Technical Approaches: 

The report recognizes that "MCC has refined its guidance over time, and 
continues to do so." MCC is still a new and evolving agency, and we are 
continually refining our standard procedures, for economic analysis and 
in other areas. MCC's rigorous, transparent use of economic analysis to 
estimate the cost-effectiveness of potential interventions is 
exceptional within the international development community and U.S. 
foreign assistance agencies. 

We endorse GAO's conclusion that "establishing a consistent approach 
with preferred methods - which could be modified if required by 
specific conditions - would help MCC to reduce the likelihood of 
errors." MCC has an effort underway to develop standard practices and 
baseline templates for a number of core sectors prevalent in past 
proposals, including roads, water services & sanitation, and 
agriculture & irrigation. We are also considering such templates for 
education, land reform, and micromicrofinance. [See comment 1] 

Placing Consistency in Context: 

GAO analyzed more than 20 projects' economic models, most of which 
comprise numerous spreadsheets incorporating data from reams of 
academic research, country sources, and professional judgment. The 
report cited a number of "inconsistencies" in MCC practices, and GAO 
identified some as minor, with trivial effect on our estimates. In many 
cases, these "inconsistencies" are technically appropriate variations 
of standard practices that can be explained by the specific country 
context. [Footnote 48] 

There is simply no single cost-benefit "standard practice," because 
many of the technical details are by necessity context specific. In 
some cases, the data available allow reasonable estimation using 
different models, and MCC needs the flexibility, as the GAO report 
notes, to apply professional judgment as to when the attainment of 
uniformity in practice is either not cost-effective or will yield 
perverse results. For example, when existing data allow the use of a 
different model and the collection of new data needed for the standard 
model would require both significant time and cost, MCC might find the 
alternative model acceptable. Similarly, when application of a standard 
time horizon does not appropriately reflect a project whose useful life 
is either shorter or longer, MCC will prefer to "deviate" [See comment 
2] from consistent practice rather than inaccurately estimate the 
economic impact of the proposed investment. In all such cases, however, 
MCC will fully subject these decisions to peer review and document them 
for public viewing and any subsequent external assessment. MCC will 
also use sensitivity analysis to explore the implications of 
alternative assumptions or parameters. [See comment 1] 

MCC's ERRs Estimate the Impact on Local Incomes: 

GAO differentiates between ERRS and what the report refers to as MCC's 
"projections of compacts' impact on income and poverty," which implies 
that our ERRS are an aggregate measure of impact disconnected from 
welfare levels experienced by the low-income residents in our partner 
countries. This characterization is based on a common misunderstanding 
about how MCC calculates ERRs and what those numbers represent. 

MCC follows standard cost-benefit practices in most ways, but includes 
only incremental increases in incomes earned by households and domestic 
firms. By excluding other possible benefit streams that do not affect 
domestic incomes, MCC's ERRs represent a direct estimation of the 
magnitude by which local incomes will rise as a result of the MCC 
program. When MCC reports high returns for our projects, these 
estimates do not reflect a broad impersonal measure of economic 
activity, but rather a much more tangible estimate of the project's 
effect on people's lives. [See comment 3] 

Conclusion: 

MCC uses economic analysis and other technical work to direct taxpayer 
funds to investments that will generate significant benefits in well-
governed developing countries. MCC's performance of cost-benefit 
assessments for virtually all proposed investments, posted on our 
public website, is unprecedented. This assessment represents a critical 
tool for accountability. For every proposed project, MCC assesses 
whether it is a wise investment of American taxpayer funds that will 
generate ample returns for our intended beneficiaries. 

It is instructive to compare MCC practices to those found in other 
foreign assistance agencies around the world. Most other aid agencies 
rarely, if ever, subject their projects to such technical scrutiny. MCC 
does it as a matter of practice, and our practices have generally been 
both reliable and transparent. 

Both our use of economic analysis as a tool for decision-making and our 
openness to public review are critical parts of the MCC model. MCC has 
established a new standard for project efficacy and transparency, and 
we will continue to review and enhance our practices to meet the high 
standards we have set for ourselves. 

Sincerely, 

Signed by: 

Rodney G. Bent: 
Deputy CEO: 
Millennium Challenge Corporation: 

[End of letter] 

The following are GAO's comments from the Millennium Challenge 
Corporation letter, dated May 23, 2008. 

GAO Comments: 

1. We recognize MCC's constructive responses to the issues identified 
in the course of our audit--including posting economic analyses on the 
Internet, developing standard practices and templates, and initiating 
an independent peer review process. 

2. We acknowledge that there is not a single standard practice; 
however, we maintain that consistent analytic approaches are needed to 
ensure the reliability and comparability of MCC's analyses. According 
to MCC, the choice of analytic time frames was in some cases based on 
the judgment of the individual lead economist rather than on 
established criteria. The steps that MCC has stated it is taking--using 
a default time frame for analyses and subjecting the time frame 
decision to peer review--will help to enhance the comparability of its 
of ERR calculations across compacts. 

3. We disagree that MCC's ERRs estimate the project's effect on 
people's lives. ERRs do not provide information about MCC's impact on 
any specific population group, including the poor. ERRs also are a 
relative measure of benefits in relation to costs and can fluctuate 
owing to changes in costs alone. For example, if costs increase and the 
income benefits remain the same, the ERR would decrease. Therefore, 
ERRs cannot be considered an absolute measure of income benefits. 

[End of section] 

Appendix VI: GAO Contact and Staff Acknowledgments: 

GAO Contact: 

David B. Gootnick, Director, 202-512-3149 or gootnickd@gao.gov: 

Staff Acknowledgments: 

In addition to the person named above, Emil Friberg, Jr. (Assistant 
Director), Todd M. Anderson, Gergana Danailova-Trainor, Reid Lowe, 
Michael Simon, and Seyda Wentworth made key contributions to this 
report. Also, C. Etana Finkler, Ernie Jackson, and Tom McCool provided 
technical assistance. 

[End of section] 

Footnotes: 

[1] An MCC compact is an agreement between the U.S. government, acting 
through MCC, and the government of a country eligible for MCC 
assistance. MCC's authorizing legislation, Public Law 108-199, limits 
compact duration to no more than 5 years. 

[2] The due diligence review also includes an evaluation of countries' 
consultative processes used to develop the proposal, donor 
coordination, and environmental and social impact, among other things. 

[3] Economic rate of return is the expected annual average return to 
the countries' firms, individuals, or sectors for each dollar that MCC 
spends on the project. For example, if MCC spent $100,000 on a project 
in year 1 and expected that the project would yield net benefits of 
$120,000 in year 2, the project's ERR for year 1 would be (120,000- 
100,000)/100,000=0.2, or 20 percent. 

[4] Transaction teams comprise MCC staff, personnel from other U.S. 
agencies, and consultants. 

[5] The investment committee consists of MCC's Chief Executive Officer 
(CEO), vice presidents, and other senior officials. The committee 
reviews the memo and decides whether to recommend proceeding to compact 
negotiations. 

[6] GAO, Millennium Challenge Corporation: Vanuatu Compact Overstates 
Projected Program Impact, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-07-909] (Washington, D.C.: July, 11, 2007). 

[7] An internal control is an integral component of an organization's 
management that provides reasonable assurance that the agency is 
achieving: effectiveness and efficiency of operations, reliability of 
financial reporting, and compliance with applicable laws and 
regulations. For OMB guidance, see OMB circular A-123. For GAO 
guidance, see Standards for Internal Control in the Federal Government, 
[hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-21.3.1] 
(Washington, D.C.: November 1999). 

[8] We could not review MCC's calculations for the Lesotho compact, 
because MCC based its projections on a previous World Bank economic 
growth model rather than its own calculations. 

[9] MCC first issued guidelines for the compact development process in 
April 2005. MCC issued revised guidelines in January 2006 and, most 
recently, in November 2006. 

[10] MCC also must report to Congress prior to obligating funds. 

[11] The Secretary of State serves as MCC Board chair, and the 
Secretary of the Treasury serves as vice-chair. Other board members are 
the U.S. Trade Representative, the Administrator of the U.S. Agency for 
International Development (USAID), the CEO of MCC, and up to four 
Senate-confirmed public members who are appointed by the President from 
lists of individuals submitted by congressional leadership. 

[12] We recently reported on MCC's progress in developing and 
implementing compacts. See GAO, Millennium Challenge Corporation: 
Analysis of Compact Development and Future Obligations and Current 
Disbursements of Compact Assistance, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO-08-577R], (Washington, D.C.: Apr. 11, 2008). 

[13] MCC stated that these analyses do not capture all aspects of MCC's 
potential impact, such as the effect of government reforms or policy 
changes. 

[14] MCC's guidelines discuss this type of economic analysis under the 
heading "Beneficiary Analysis." For the purposes of our report, we 
refer to this type of analysis as projections of impact on income and 
poverty or simply impact projections. 

[15] In El Salvador, the water project is a component of the Community 
Infrastructure project. 

[16] This amount includes both obligations and commitments. As of March 
2008, MCC had provided a total of $5.5 billion for all 16 countries 
with signed compacts. When we began our work in August 2007, MCC had 
signed 13 compacts totaling $3.85 billion. 

[17] In some cases, the ERRs we refer to were calculated by MCC at the 
activity level. 

[18] The four compacts' minimum ERRs are 12.5 percent for Armenia, 10.8 
percent for El Salvador, 10.8 percent for Lesotho, and 8.76 percent for 
Mozambique. MCC set the minimum ERR for each compact applying 
definitions contained in the guidelines current at the time. The 
initial ERRs for all but two projects--the El Salvador Community 
Infrastructure project and the Lesotho Rural Water project--met the 
respective minimums for each country. See app. II for further 
discussion of MCC's minimum ERR. 

[19] Specifically, the Rio Lurio-Metoro Road segment ERR is 7.2 
percent, the Namialo-Rio Lurio ERR is 5.9 percent, and the Nampula-Rio 
Ligonha ERR is 6.3 percent. 

[20] MCC's April 2005 guidance stated that project ERRs should be 
defined "over the natural life of that component." 

[21] According to MCC, the urban water analysis was revised after the 
investment memo to phase in costs. 

[22] Salvage value is the estimated value of an asset at the end of its 
useful life. 

[23] MCC used an alternative method for calculating road project 
benefits in El Salvador. Because the existing road was nearly 
impassable, MCC thought its preferred method based on existing traffic 
estimates was a poor predictor of the project's impact. MCC instead 
substituted a measure of increases in land values as a proxy for income 
growth. MCC's transaction team reported both scenarios to the 
investment committee--the ERR would be 24 percent using a land value- 
based measure and between 13.8 percent and 14.7 percent using the 
traffic count method. The ERR in either method exceeds MCC's minimum 
ERR of 10.8 percent. 

[24] MCC's April 2005 guidance noted that overall compact ERRs may be 
calculated by using cost-weighted averages of project components, 
combining cost and benefit flows, or another approach--depending on the 
facts and circumstances of the compact. The January 2006 guidance and 
the November 2006 guidance do not define a procedure for calculating 
compact-level ERR. 

[25] See OMB Circular A-123, "Management's Responsibility for Internal 
Control," revised Dec. 21, 2004; and GAO, Standards for Internal 
Control in the Federal Government, [hyperlink, http://www.gao.gov/cgi-
bin/getrpt?GAO/AIMD-00-21.3.1] (Washington, D.C.: November 1999). 

[26] MCC's November 2006 Guidelines for Economic and Beneficiary 
Analysis states that poverty may be defined according to country- 
specific definitions, such as the official poverty line, or according 
to international standards such as the World Bank's extreme poverty 
definition of $1.08 per capita per day in purchasing power parity, or 
$2 per day. In making its public statements of poverty impact in the 
countries we examined, MCC used country-specific definitions. 

[27] We reviewed MCC's impact analysis spreadsheets for these three 
compacts. For Lesotho, although MCC economists conducted ERR analyses 
for activities and projects under the compact, MCC ultimately based its 
compact-level projections on a previous World Bank economic growth 
model. As such, we were not able to assess MCC's compact-level impact 
projections for Lesotho. 

[28] In its initial projections of the impact of increased incomes on 
poverty in Mozambique, MCC used two types of income estimates - GDP and 
GDP per capita - to compare with-and without-project scenarios. 

[29] Poverty elasticity measures the extent to which economic growth 
reduces poverty by estimating the percentage change in poverty caused 
by a 1 percent change in income. 

[30] As of May 2008, MCC has posted spreadsheets on its Web site 
showing calculations of ERR for El Salvador projects. These 
spreadsheets do not include MCC's compact ERR calculation or 
calculations of compact impact on income and poverty. 

[31] According to OMB and GAO guidelines, an effective control 
environment for data processing may include edit checks. OMB Circular A-
123 calls on federal agencies to establish management controls to 
ensure that reliable and timely information is maintained for decision 
making. See OMB Circular A-123, "Management's Responsibility for 
Internal Control," revised Dec. 21, 2004. GAO's Standards for Internal 
Control in the Federal Government cites edit checks as an example of a 
control activity used in information processing [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-21.3.1] (Washington, 
D.C.: November 1999). 

[32] Income growth is defined as the percentage change of income from 
year to year. 

[33] Specifically, MCC projected that the compact could nearly double 
GDP growth by the end of the 5-year compact implementation period 
(using the International Monetary Fund's baseline of 2.6 percent). MCC 
also stated that the acceleration of GDP growth was expected to 
continue, propelling growth toward 7 percent per annum within 5 years 
after compact completion. 

[34] The World Bank model tracks flows of all transactions, from sector 
to sector, within the economy. In most countries, the economists could 
not use such analysis either because no well-calibrated model exists or 
because the MCC package is not easily incorporated into existing 
models. 

[35] The World Bank, Lesotho Country Economic Memorandum: Growth and 
Employment Options Study, Report No. 35359-LS, Apr. 21, 2005. 

[36] An alternative scenario developed by the World Bank involves an 
approximate $50 million investment in infrastructure by the government 
of Lesotho. MCC did not choose to extrapolate the results of this 
scenario. 

[37] MCC did not publicly report an estimate of poverty impact for 
Lesotho. 

[38] Poverty analysis generally involves measuring economic welfare of 
individuals and constructing poverty lines to determine the number of 
people deemed poor--the poverty rate--and the depth of poverty--the 
poverty gap. Viable measures of economic welfare include, for example, 
income per capita, consumption per some "standardized" adult, food 
share of total expenditures, and nutritional indicators. For each 
compact, MCC measured poverty by the country's headcount poverty rate-
-that is, the number of people with incomes below the poverty line. 

[39] Cross-country data are data drawn from multiple countries. MCC's 
memorandum additionally noted the possibility of using a mixed approach 
that assumes a nonlinear relationship between poverty and income, using 
different values for specific ranges of poverty. 

[40] MCC's November 2006 guidelines state that impact on poverty should 
be measured in terms of both the poverty rate and the poverty gap. 
However, MCC did not estimate changes in the poverty gap for any of the 
four compacts we reviewed. According to the guidelines, "the poverty 
gap is calculated as the sum of money required to bring all poor 
households up to the poverty line, and the effect of an MCC investment 
on the poverty gap would reflect incremental income to poor households 
in aggregate. The poverty rate, in contrast, would not reflect, for 
example, significant improvements in income levels for households 
remaining below the poverty line." As a result of estimating the impact 
on the poverty rate but not on the poverty gap, MCC estimates its 
compacts' impacts on the number of beneficiaries lifted out of poverty 
but does not evaluate potential impact on the severity of 
beneficiaries' poverty. 

[41] In MCC's public documents, MCC cited the entire population of the 
Northern Zone of El Salvador (850,000 people) as the total number of 
compact beneficiaries. In addition, in its 2006 annual report, MCC 
characterizes these beneficiaries as poor. However, according to MCC's 
investment memo, 450,000 (53 percent) of these people are poor. The 
investment memo further defines poverty in the context of its 
Productive Development Project in two ways: (1) relative poverty as 
defined by El Salvador's General Directorate for Statistics and Census 
and (2) more than half the population of the Northern Zone living on 
less than $2 dollar per day and more than 25 percent living on less 
than $1 per day. However, the investment memo does not specify the 
poverty definition used to generate the estimate of 450,000 poor 
people. 

[42] As of March 2008, MCC had 16 signed compacts. 

[43] These internal documents are restricted from public dissemination 
based on MCC policy, but MCC made them available to us for analysis. As 
of April 2008, MCC had posted its spreadsheets showing calculations of 
project-level ERR for El Salvador projects on its Web site, see 
[hyperlink, http://www.mcc.gov/programs/err/]. As of April 2008, MCC 
has not released the supporting ERR spreadsheets for Armenia, Lesotho, 
and Mozambique, but MCC officials have told us they plan to release all 
such spreadsheets. 

[44] See OMB circular A-123. For GAO guidance, see Standards for 
Internal Control in the Federal Government, [hyperlink, 
http://www.gao.gov/cgi-bin/getrpt?GAO/AIMD-00-21.3.1] (Washington, 
D.C.: November 1999). 

[45] See [hyperlink, http://www.mcc.gov/about/reports/] and [hyperlink, 
http://www.mcc.gov/countries/index.php]. 

[46] MCC's April 2005 guidelines noted that the hurdle rate definition 
will be revised based on MCC's subsequent experience and data on the 
experiences of developing countries in general. Although other 
development agencies, such as the World Bank and the Asian Development 
Bank, link their hurdle rate to the notion of opportunity cost of 
capital rather than GDP growth rate, in practice the results are 
similar, in that the resulting hurdle rate generally remains between 10 
percent and 12 percent. According to MCC officials, they considered 
other measures but decided on a method that was tied to economic growth 
and did not require extensive and debatable data analysis--MCC's 
current hurdle rates are generally between 10 percent and 15 percent. 

[47] MCC's November 2006 guidelines note that the hurdle rates will be 
set once per year, using data available in the September edition of the 
International Monetary Fund's World Economic Outlook Database for the 3 
previous years. 

[48] In one example, local counterparts developed initial calculations 
for a road project using a 25-year time horizon instead of the 20-year 
horizon used by MCC in most other countries. MCC's economist reviewed 
the analysis and found the difference unimportant to the outcome. The 
economist decided it was better to accept the high-quality work done 
locally than to insist on revisions solely for the sake of attaining
uniformity of practice. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office: 
441 G Street NW, Room LM: 
Washington, D.C. 20548: 

To order by Phone: 
Voice: (202) 512-6000: 
TDD: (202) 512-2537: 
Fax: (202) 512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: