This is the accessible text file for GAO report number GAO-10-657
entitled 'Electronic Records Archive: Status Update on the National
Archives and Records Administration's Fiscal Year 2010 Expenditure
Plan' which was released on June 11, 2010.
This text file was formatted by the U.S. Government Accountability
Office (GAO) to be accessible to users with visual impairments, as
part of a longer term project to improve GAO products' accessibility.
Every attempt has been made to maintain the structural and data
integrity of the original printed product. Accessibility features,
such as text descriptions of tables, consecutively numbered footnotes
placed at the end of the file, and the text of agency comment letters,
are provided but may not exactly duplicate the presentation or format
of the printed version. The portable document format (PDF) file is an
exact electronic replica of the printed version. We welcome your
feedback. Please E-mail your comments regarding the contents or
accessibility features of this document to Webmaster@gao.gov.
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed
in its entirety without further permission from GAO. Because this work
may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this
material separately.
Report to Congressional Committees:
United States Government Accountability Office:
GAO:
June 2010:
Electronic Records Archive:
Status Update on the National Archives and Records Administration's
Fiscal Year 2010 Expenditure Plan:
GAO-10-657:
GAO Highlights:
Highlights of GAO-10-657, a report to congressional committees.
Why GAO Did This Study:
Since 2001, the National Archives and Records Administration (NARA)
has been working to develop an electronic records archive (ERA) to
preserve and provide access to massive volumes and all types of
electronic records. NARA certified initial operating capability of the
first two phases of ERA in June 2008 and December 2008 and plans to
achieve full operating capability for the system by 2012. As required
by the Consolidated Appropriations Act, NARA submitted an expenditure
plan to Congress to support its request for fiscal year 2010 ERA
funding. The act also requires that this plan meet six conditions,
including a review by GAO. GAO’s objectives in reviewing the fiscal
year 2010 plan were to (1) determine whether the plan satisfies the
legislative conditions, (2) determine the extent to which NARA has
implemented prior GAO recommendations, and (3) provide any other
observations on the plan or the ERA acquisition. To do this, GAO
reviewed the expenditure plan and other agency documents and
interviewed NARA officials.
What GAO Found:
NARA’s fiscal year 2010 expenditure plan satisfies five of the six
legislative conditions and partially satisfies one. Specifically, it
partially satisfies the condition that NARA develop a process for
reviewing the progress of capital investments. Best practices call for
such a process to include an oversight entity to review a project’s
progress, take corrective actions in response to cost or schedule
problems, and ensure the corrective actions are implemented. However,
while NARA has conducted regular meetings with senior-level agency
management to review ERA progress, there is little evidence that
officials identified corrective actions or ensured the actions were
implemented. As a result, it is uncertain whether ERA is receiving the
necessary level of executive oversight.
NARA has fully implemented three of GAO’s five prior recommendations
and partially implemented two. The agency implemented recommendations
to provide additional information on its plans for the remainder of
fiscal year 2009, add an analysis of costs and benefits associated
with the ERA component dedicated to presidential records, and develop
a contingency plan for ERA in case of system failure. NARA partially
implemented a recommendation to provide additional information in the
fiscal year 2010 expenditure plan by adding information on ERA’s cost,
schedule, and performance. However, the plan lacks key information,
including the total cost of one of the increments to be funded. NARA
has also developed but not yet implemented a plan to implement best
practices in its process for measuring program progress, as GAO
recommended.
GAO has three observations on the expenditure plan and ERA acquisition:
* The estimated life-cycle cost of the ERA system has increased, and
the development is behind schedule. Over the last 3 fiscal years, the
estimated cost has increased by about 7 percent, from about $531
million to about $567 million. In addition, the planned completion
dates for the two increments currently under development are about 1
year later than dates established in program planning documents.
* NARA has not detailed what system capabilities will be delivered in
the final two ERA increments. While the expenditure plan identifies
some high-level functions, it does not identify specific capabilities
to be delivered or dates for completion.
* NARA has not effectively defined or managed ERA’s requirements to
ensure that the functionality delivered satisfies the objectives of
the system. Although NARA established an initial set of high-level
requirements, it lacks firm plans to implement about 43 percent of
them. In addition, NARA has not updated its requirements document to
reflect reinterpreted requirements and did not provide evidence that
it always conducted reviews required by its internal policies.
Without ensuring adequate oversight and establishing specific plans to
complete ERA, it is increasingly unlikely that NARA will deliver the
completed ERA system by 2012 with the originally envisioned
capabilities.
What GAO Recommends:
GAO is recommending that the Archivist of the United States ensure
that NARA’s investment review process includes adequate executive-
level oversight and that ERA’s requirements are being managed using a
disciplined process. In commenting on a draft of this report, the
Archivist generally agreed and described steps taken or planned to
address our recommendations.
View [hyperlink, http://www.gao.gov/products/GAO-10-657] or key
components. For more information, contact David A. Powner at (202) 512-
9286 or pownerd@gao.gov.
[End of section]
Contents:
Letter:
Conclusions:
Recommendations for Executive Action:
Agency Comments:
Appendix I: Briefing to Staff of Congressional Committees on NARA's
Fiscal Year 2010 Expenditure Plan:
Appendix II: Comments from the National Archives and Records
Administration:
Appendix III: GAO Contact and Staff Acknowledgments:
[End of section]
United States Government Accountability Office:
Washington, DC 20548:
June 11, 2010:
The Honorable Richard J. Durbin:
Chairman:
The Honorable Susan M. Collins:
Ranking Member:
Subcommittee on Financial Services and General Government:
Committee on Appropriations:
United States Senate:
The Honorable José E. Serrano:
Chairman:
The Honorable Jo Ann Emerson:
Ranking Member:
Subcommittee on Financial Services and General Government:
Committee on Appropriations:
House of Representatives:
Since 2001, the National Archives and Records Administration (NARA)
has been developing a modern Electronic Records Archive (ERA). This
major information system is intended to preserve and provide access to
massive volumes of all types and formats of electronic records,
independent of their original hardware or software. NARA plans for the
system to manage the entire life cycle of electronic records, from
their ingestion through preservation and dissemination to customers.
The ERA system is being developed in five phases, or increments, the
first of which is referred to as the "ERA base." According to NARA,
this portion of the system achieved initial operating capability (IOC)
in June 2008. The second increment includes the Executive Office of
the President (EOP) system or "ERA EOP," and NARA certified that it
reached IOC in December 2008. NARA plans to complete development of
the remaining increments and achieve full operating capability by 2012.
As mandated by the Consolidated Appropriations Act,[Footnote 1] NARA
is required to submit an expenditure plan before obligating multi-year
funds for the ERA program. In November 2009, NARA submitted its fiscal
year 2010 expenditure plan to support its request for $85.5 million in
ERA funding, which includes $61.7 million in multi-year funds. As in
the previous year, the plan must satisfy six legislative conditions,
including a review by GAO. Our objectives in reviewing the plan were
to (1) determine whether NARA's fiscal year 2010 expenditure plan
satisfies the legislative conditions, (2) determine the extent to
which NARA has implemented prior GAO recommendations, and (3) provide
any other observations about the expenditure plan and the ERA
acquisition.
To assess compliance with the legislative conditions, we analyzed the
expenditure plan submitted by NARA in November 2009 and reviewed its
budget submission to the Office of Management and Budget (OMB), along
with other program documentation. To determine the extent to which
NARA had implemented our prior recommendations, we obtained and
reviewed agency documents, which included quarterly reports to
Congress and the ERA contingency plan. To develop observations on the
ERA expenditure plan and acquisition, we analyzed fiscal year 2009 and
2010 schedule information contained in the expenditure plan, the ERA
requirements document, and other agency data. We also interviewed NARA
officials.
To assess the reliability of the data in the expenditure plan, we
obtained and reviewed NARA budget documents as well as its
consolidated financial statement results for the fiscal year 2009
Performance and Accountability Report. We also interviewed NARA
officials to gain an understanding of the data and discussed our use
of the data.
We conducted this performance audit from January 2010 to June 2010 at
NARA's College Park, Maryland, location in accordance with generally
accepted government auditing standards. Those standards require that
we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and
conclusions based on our audit objectives. We believe that the
evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
On April 6, 2010, we briefed your staffs on the results of our review.
This report transmits the material we used at the briefing and
provides the recommendations that we made to the Archivist of the
United States. The full briefing materials, including details on our
scope and methodology, are reprinted as appendix I.
In summary, we made the following major points:
* NARA's fiscal year 2010 expenditure plan satisfies five of the six
legislative conditions and partially satisfies one. Specifically, it
partially satisfies the condition that it develop a process for
reviewing the progress of capitol investments. Best practices call for
such a process to include an oversight entity to review a project's
progress, take corrective actions in response to cost or schedule
problems, and ensure the corrective actions are implemented. However,
while NARA has conducted regular meetings with senior-level agency
management to review ERA progress, there is little evidence that
officials identified corrective actions to address performance
problems or ensured the actions were implemented. As a result, it is
uncertain whether ERA is receiving the necessary level of executive
oversight.
* NARA has fully implemented three of our prior recommendations and
partially implemented two:
- NARA implemented our recommendations to provide additional
information to Congress describing plans for the remainder of fiscal
year 2009, add an analysis of the cost and benefits of using the EOP
system to respond to presidential records requests, and develop an ERA
contingency plan.
- NARA has partially implemented two recommendations. First, in
response to our recommendation that NARA provide additional
information in the fiscal year 2010 expenditure plan on what was spent
and delivered for deployed increments, NARA added information on
completed increments and those planned for fiscal year 2010 with their
associated costs. However, our review of the Expenditure Plan showed
that it does not fully describe how NARA will expend fiscal year 2010
funds. Second, in response to our recommendation to strengthen its
earned value management processes, NARA developed an action plan to
implement best practices identified in GAO's Cost Estimating Guide;
[Footnote 2] however, this plan has not been fully implemented. We
currently have work under way to evaluate NARA's earned value process
and its implementation of the action plan.
* We made three observations related to the ERA program and fiscal
year 2010 expenditure plan:
- The cost of ERA is rising, and the development of several system
components is behind schedule. Specifically, over the last 3 fiscal
years, the estimated life-cycle cost has increased by about 7 percent,
from about $531 million to about $567 million. NARA attributed the
change to increases in the complexity of the system being developed.
In addition, NARA plans to complete Increment 3 in June 2010 and
Increment 4 in mid-2011, both of which are about 1 year later than the
milestones established in program planning documents. NARA officials
stated that the delivery date is consistent with the current project
schedule, which was not documented because they determined it was not
cost effective to do so.
- NARA has not detailed what capabilities will be delivered by the
final two ERA phases, or increments. For example, the expenditure plan
indicates that NARA will begin implementing ERA's preservation
framework in Increment 4 but does not contain specific dates for
completion or identify the associated capabilities that are to be
delivered. Without including detailed plans for the final two
increments, including specific dates for completion and associated
functionality to be delivered, Congress will have limited insight to
evaluate NARA's ongoing progress.
- NARA has not effectively defined or managed requirements for the ERA
system. Among other things, requirements should be documented at a
high level and traceable throughout the system's life cycle to ensure
that functionality satisfies the objectives of the system. Further,
any changes to the program's scope should be reflected by updating the
requirements in a structured way. Although NARA established an initial
set of high-level requirements to guide the system's development,
these requirements are not traceable to work in later phases, or
increments, of system. Specifically, about 43 percent of the
requirements have not been allocated to the remaining two increments,
and NARA officials stated that it is uncertain whether they will be
implemented at all. Consequently, it is unclear whether system
development work performed during the last two phases of the system's
development will result in functionality that satisfies the intended
objectives. Further, NARA has not updated its ERA requirements
document to reflect reinterpreted requirements. These weaknesses can
be attributed to the fact that NARA did not manage requirements by
conducting and documenting requirement reviews near the beginning of
each increment, as called for in its guidance. NARA officials stated
that requirements reviews for Increment 3 were conducted throughout
the increment's development but not specifically documented. They
added that a requirements review for Increment 4 would be conducted
during the week of March 29th. Without better defined and managed
requirements, NARA will have little assurance that the ongoing system
development will meet its needs.
Conclusions:
While NARA's fiscal year 2010 expenditure plan met five of the six
legislative conditions, the lack of documentation demonstrating the
appropriate level of senior management review, approval, and oversight
limits NARA's ability to ensure that the project makes expected
progress. In addition, the lack of a documented current baseline
schedule leaves NARA with limited ability to identify significant
delays. The lack of key oversight documentation limits NARA's
assurance that schedule delays and cost increases will not continue
during ERA's development.
In addition, without specific plans for completing the final two
increments, NARA lacks assurance that the remaining increments will
adequately fulfill its mission needs. Specifically, NARA's failure to
follow its own guidance on requirements management has resulted in a
set of requirements that is incomplete and out of date, which could
lead to the system being completed without addressing all necessary
requirements or the development of functionality to meet requirements
that are no longer valid. Addressing these weaknesses is becoming even
more critical as the projected completion of the project approaches;
if they continue, it will be increasingly unlikely that NARA will be
able to deliver the ERA system by 2012 with the capabilities
originally envisioned or to effectively use the system to meet the
needs of its users in support of NARA's mission of preserving and
providing access to the nation's electronic records.
Recommendations for Executive Action:
To enhance NARA's ability to complete the development of ERA within
reasonable funding and time constraints, we are recommending that the
Archivist of the United States take the following two actions:
* Ensure that NARA's investment review process has adequate executive-
level oversight by maintaining documentation of the results of
reviews, including changes to the program's cost and schedule baseline
and any other corrective actions taken as a result of changes in ERA
cost, schedule, and performance.
* Ensure that ERA's requirements are managed using a disciplined
process that results in requirements that are traceable throughout the
project's life cycle and are kept current.
Agency Comments:
In written comments on a draft of this report, which are reprinted in
appendix II, the Archivist of the United States generally agreed with
our recommendations and summarized NARA actions taken or planned to
address them. Specifically, he stated that NARA is now documenting the
minutes and actions of Senior Staff meetings where ERA is discussed
and has recently begun an intensive effort to review ERA requirements
within the context of new technologies and changing business needs.
The Archivist also stated that NARA managers recently met to determine
top ERA functional priorities to be completed by the end of the
contract. Finally, he added that the ERA Requirements Manager is
mapping these functional priorities to their underlying contract
requirements and plans to update the ERA requirements document.
We are sending copies of this report to the Archivist of the United
States. The report will also be available at no charge on the GAO Web
site at [hyperlink, http://www.gao.gov].
If you or your staffs have any questions concerning this report,
please contact me at (202) 512-9286 or by e-mail at pownerd@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who
made major contributions to this report are listed in appendix III.
Signed by:
David A. Powner:
Director, Information Technology Management Issues:
[End of section]
Appendix I: Briefing to Staff of Congressional Committees on NARA's
Fiscal Year 2010 Expenditure Plan:
Review of the National Archives and Records Administration's Fiscal
Year 2010 Electronic Records Archive Expenditure Plan:
Briefing for Staff Members of the Subcommittee on Financial Services
and General Government, Committee on Appropriations, U.S. Senate
and the Subcommittee on Financial Services and General Government,
Committee on Appropriations, House of Representatives:
April 6, 2010:
Contents:
* Introduction;
* Objectives, Scope, and Methodology;
* Results in Brief Background Results;
* Conclusions;
* Recommendations for Executive Action;
* Agency Comments and Our Evaluation;
* Attachment:
1. Comments from the Archivist of the United States.
Introduction:
Since 2001, the National Archives and Records Administration (NARA)
has been working to develop a modern Electronic Records Archive (ERA).
This major information system is estimated to cost more than $567
million and is intended to preserve and provide access to massive
volumes of all types and formats of electronic records, independent of
their original hardware or software. NARA plans for the system to
manage the entire life cycle of electronic records, from their
ingestion through preservation and dissemination to customers. It is
to consist of:
* infrastructure elements, including hardware and operating systems;
* business applications that will support the transfer, preservation,
dissemination, and management of all types of records; and;
* a means for public access via the Internet.
Because of the system's complexity, NARA awarded a contract to
Lockheed Martin to develop ERA in five phases, or increments, the
first of which is referred to as the "ERA base." According to NARA,
this system achieved initial operating capability (IOC) in June 2008.
The second increment includes the Executive Office of the President
(EOP) system or "ERA EOP," and NARA certified that it reached IOC in
December 2008. NARA plans to complete development of the remaining
increments and achieve full operating capability (FOC) by 2012.
As mandated by the Consolidated Appropriations Act,[Footnote 3] NARA
is required to submit an expenditure plan before obligating multi-year
funds for the ERA program. As in the previous year, the plan must
satisfy the following legislative conditions:
* meet the capital planning and investment control review requirements
established by the Office of Management and Budget (OMB), including
Circular A-11;
* comply with the agency's enterprise architecture;
* conform to the agency's enterprise life-cycle methodology;
* comply with the acquisition rules, requirements, guidelines, and
system acquisition management practices of the federal government;
* be approved by the agency and OMB; and;
* be reviewed by GAO.
In November 2009, the agency submitted its fiscal year 2010
expenditure plan to the relevant House and Senate appropriations
committees to support its request for $85.5 million in ERA funding. Of
this amount, $61.7 million is multi-year funds allocated to ERA. As of
March 2010, the committees had released $23.7 million of the $61.7
million multi-year funds.
[End of section]
Objectives, Scope, and Methodology:
Our objectives were to:
* determine whether NARA's fiscal year 2010 expenditure plan satisfies
the legislative conditions;
* determine the extent to which NARA has implemented prior GAO
recommendations; and;
* provide any other observations about the expenditure plan and the
ERA acquisition.
To assess compliance with the legislative conditions, we:
* reviewed NARA's briefings to senior management and OMB, minutes of
Information Technology Executive Committee meetings, and the fiscal
year 2010 exhibit 300 submission[Footnote 4] to OMB to determine the
extent to which the agency has complied with OMB's capital planning
and investment control requirements;
* obtained and reviewed data on NARA's enterprise architecture to
determine the status of the agency's enterprise architecture efforts;
* reviewed NARA's ERA system life-cycle processes, which include
processes for managing system investments, configuration management,
and managing risks, and reviewed related agency documentation
describing how these processes were implemented for the ERA project,
such as minutes of oversight boards and the risk management plan;
* obtained and reviewed NARA's internal assessment of ERA compliance
with federal requirements for agencies information technology
acquisitions;
* obtained and reviewed OMB's approval of the expenditure plan; and;
* reviewed and analyzed the fiscal year 2010 expenditure plan
submitted by the agency in November 2009.
To determine the extent to which NARA has implemented our prior
recommendations, we obtained and reviewed agency documents, which
include quarterly reports to Congress, the ERA contingency plan, and
NARA's earned value management (EVM) action plan.
To develop observations on the ERA expenditure plan and acquisition,
we analyzed fiscal year 2009 and 2010 schedule information contained
in the expenditure plan, ERA briefings, and congressional status
reports. We reviewed and analyzed agency documents such as the ERA
Requirements Document, Online Public Access Search Functionality, and
plans for the ERA preservation prototype. We also interviewed NARA
officials.
To assess the reliability of the data in the expenditure plan, we
interviewed NARA officials in order to gain an understanding of the
data and discussed our use of the data in this briefing. In addition,
we obtained and reviewed NARA budget documents as well as its
consolidated financial statement results for the fiscal year 2009
Performance and Accountability Report. We did not, however, assess the
accuracy and reliability of the information in these documents.
We conducted this performance audit from January 2010 to April 2010 at
NARA's College Park, Maryland, location in accordance with generally
accepted government auditing standards. Those standards require that
we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and
conclusions based on our audit objectives. We believe that the
evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.
[End of section]
Results in Brief:
NARA's fiscal year 2010 expenditure plan satisfies five of the six
legislative conditions contained in the 2010 Consolidated
Appropriations Act and partially satisfies one. Specifically, it
partially satisfies the condition that it develop capital planning and
investment control review processes designed to help ensure that
projects are being implemented at an acceptable cost and within
reasonable and expected time frames. NARA has conducted regular
meetings with senior-level agency management to review ERA progress
and regularly assesses risks, but there is little evidence that this
process addresses other key investment management practices, including
identifying corrective actions and ensuring that they are taken and
being tracked to closure. As a result, it is uncertain whether ERA is
receiving the necessary level of executive oversight to ensure that it
is being implemented at an acceptable cost and within reasonable and
expected time frames.
NARA fully implemented three of our prior recommendations and
partially implemented two:
* NARA implemented our recommendations to provide additional
information to Congress describing plans for the remainder of fiscal
year 2009, add an analysis of the cost and benefits of using the EOP
system to respond to presidential records requests, and develop an ERA
contingency plan.
* NARA has partially implemented two recommendations. First, in
response to our recommendation that NARA provide additional
information in the fiscal year 2010 expenditure plan on what was spent
and delivered for deployed increments, NARA added information on
completed increments and those planned for fiscal year 2010 with their
associated costs. However, our review of the Expenditure Plan showed
that it does not fully describe how NARA will expend fiscal year 2010
funds. Second, in response to our recommendation to strengthen its
earned value management processes, NARA developed an action plan to
implement best practices identified in GAO's Cost Estimating Guide;
[Footnote 5] however, this plan has not been fully implemented. We
currently have work under way to evaluate NARA's earned value process
and its implementation of the action plan.
We have three observations related to the ERA program and fiscal year
2010 expenditure plan:
* The cost of ERA is rising and the development of several system
components is behind schedule. Specifically, over the last 3 fiscal
years, the estimated life-cycle cost has increased by about 7 percent,
from about $531 million to more than $567 million. NARA attributed the
change to increases in the complexity of the system being developed.
In addition, NARA plans to complete Increment 3 in June 2010, and
Increment 4 in mid-2011, both of which are about 1 year later than the
milestones established in program planning documents. NARA officials
stated that the delivery date is consistent with the current project
schedule, which was not documented because they determined it was not
cost effective.
* NARA has not detailed what capabilities will be delivered by the
final two ERA phases, or increments. For example, the expenditure plan
indicates that NARA will begin implementing ERA's preservation
framework in Increment 4 but does not contain specific dates for
completion or identify the associated capabilities that are to be
delivered. Without including detailed plans for the final two
increments, including specific dates for completion and associated
functionality to be delivered, Congress will have limited insight to
evaluate NARA's ongoing progress.
* NARA has not effectively defined or managed requirements for the ERA
system. Among other things, requirements should be documented at a
high level and traceable throughout the system's life cycle to ensure
that functionality satisfies the objectives of the system. Further,
any changes to the program's scope should be reflected by updating the
requirements in a structured way. Although NARA established an initial
set of high-level requirements to guide the system's development,
these requirements are not traceable to work in later phases, or
increments, of the system. Specifically, about 43 percent of the
requirements have not been allocated to the remaining two increments,
and NARA officials stated that it is uncertain whether they will be
implemented at all. Consequently, it is unclear whether system
development work performed during the last two phases of the system's
development will result in functionality that satisfies the intended
objectives. Further, NARA has not updated its ERA requirements
document to reflect reinterpreted requirements. These weaknesses can
be attributed to the fact that NARA did not manage requirements by
conducting and documenting requirement reviews near the beginning of
each increment, as called for in its guidance. NARA officials stated
that requirements reviews for Increment 3 were conducted throughout
the increment's development but not specifically documented. They
added that a requirements review for Increment 4 would be conducted
during the week of March 29th. Without better defined and managed
requirements, NARA will have little assurance that the ongoing system
development will meet its needs.
The identified deficiencies in NARA's management of the ERA project
leave it with little assurance that ERA will be able to avoid
additional cost increases and schedule delays. As a result, it is
increasingly unlikely that NARA will be able to deliver the ERA system
by the planned date of 2012 with the capabilities originally
envisioned or to effectively use the system to meet the needs of its
users in support of NARA's mission.
To enhance NARA's ability to complete the development of ERA within
reasonable funding and time constraints, we are recommending that the
Archivist of the United States take the following two actions:
* Ensure that NARA's investment review process has adequate executive-
level oversight by maintaining documentation of the results of
reviews, including changes to the program's cost and schedule baseline
and any other corrective actions taken as a result of changes in ERA
cost, schedule, and performance.
* Ensure that ERA's requirements are managed using a disciplined
process that results in requirements that are traceable throughout the
project's life cycle and are kept current.
In a letter commenting on a draft of this briefing, the Archivist of
the United States stated that he would take steps to address our
recommendations and identified several of the ERA program's recent
accomplishments. However, he also stated that the briefing did not
accurately describe the program's current state, including
strengthened management oversight, adherence to a revised schedule,
and that the program's history of success does not support our
conclusion that ERA is unlikely to be completed as planned by 2012. We
disagree. First, our briefing discusses delays in the current
increments based on ERA's program management plan and information from
last year's expenditure plan. NARA did not provide any evidence to
demonstrate the existence of the rebaselined schedule described by the
Archivist. Second, the lack of documentation, limited time remaining
to complete development, and the lack of adequate controls in several
key areas indicate critical weaknesses, rather than strong oversight.
Because of the lack of adequate controls and the limited time
remaining to complete ERA by its original deadline, we believe that on-
time completion of ERA that meets all of its original requirements is
increasingly unlikely.
The Archivists full comments are included as attachment 1.
[End of section]
Background:
The ability to find, organize, use, share, appropriately dispose of,
and save records—the essence of records management—is vital for the
effective functioning of the federal government. In the wake of the
transition from paper-based to electronic processes, records are
increasingly electronic, and the volumes of electronic records
produced by federal agencies are vast and rapidly growing, providing
challenges to NARA as the nation's record keeper and archivist.
Besides sheer volume, other factors contributing to the challenge of
electronic records include their complexity and their dependence on
software and hardware. Specifically, the computer operating systems
and the hardware and software that are used to create electronic
documents can become obsolete. If they do, they may leave behind
records that cannot be read without the original hardware and
software. Further, the storage media for these records are affected by
both obsolescence and decay. Media may be fragile, have limited shelf
life, and become obsolete in a few years. For example, few computers
today have disk drives that can read information stored on 8-or 5 1/4-
inch diskettes, even if the diskettes themselves remain readable.
Another challenge is the growth in electronic presidential records.
The Presidential Records Act gives the Archivist of the United States
responsibility for the custody, control, and preservation of
presidential records upon the conclusion of a President's term of
office.[Footnote 6] The act states that the Archivist has an
affirmative duty to make such records available to the public as
rapidly and completely as possible consistent with the provisions of
the act.
In response to these widely recognized challenges, the Archives began
a research and development program to develop a modern archive for
electronic records. In 2001, NARA hired a contractor to develop
policies and plans to guide the overall acquisition of an electronic
records system. In December 2003, the agency released a request for
proposals for the design of ERA. In August 2004, NARA awarded two firm-
fixed-price[Footnote 7] contracts for the design phase, totaling about
$20 million—one to Harris Corporation and the other to Lockheed Martin
Corporation. On September 8, 2005, NARA announced the selection of
Lockheed Martin Corporation to build the ERA system. The total value
of the contract with Lockheed through 2012 is about $317 million,
which includes provisions for award fees based on how well the
contractor meets technical, program management, and cost-control
criteria. As of fiscal year 2009, NARA has paid Lockheed $156.0
million, including $144.2 million for development and $11.8 million
for operations and maintenance.
As currently planned, the ERA system is to consist of six key
functions:
* Ingest enables the transfer of electronic records from federal
agencies.
* Archival Storage enables stored records to be managed in a way that
guarantees their integrity and availability.
* Records Management supports scheduling,[Footnote 8] appraisal,
[Footnote 9] description, and requests to transfer custody of all
types of records, as well as ingesting and managing electronic
records, including the capture of selected records data (such as
origination date, format, and disposition).
* Preservation enables secure and reliable storage of files in formats
in which they were received, as well as creating backup copies for off-
site storage.
* Local Services and Control regulates how the ERA components
communicate with each other, manages internal security, and enables
telecommunications and system network management.
* Dissemination enables users to search descriptions and business data
about all types of records, and to search the content of electronic
records and retrieve them.
NARA currently plans to deliver these components in five separate
increments. Each increment involves multiple releases that are to
deliver specific functionality. Below is a summary of the specific
releases delivered or planned for delivery in each increment:
* Increment 1 was deployed in two releases. Release 1 established the
ERA base system—the hardware, software, and communications needed to
deploy the system. Release 2 enabled functional archives with the
ability to preserve electronic data in their original format, enable
disposition agreements and scheduling, and receive unclassified and
sensitive data from four federal agencies; according to NARA
officials, this increment was certified as complete in June 2008.
However, additional enhancements were made to Increment 1, release 2,
and were completed in March 2010.
* Increment 2 includes the EOP system, which was designed to handle
records from the Executive Office of the President. This increment was
to include the content searching and management for special access
requests.[Footnote 10] The EOP system was certified for initial
operating capability (IOC) in December 2008. However, NARA did not
finish ingesting the presidential records it received until September
2009, 9 months after IOC.
* Increment 3 is expected to include the following:
- Storage and access capabilities for electronic records of the
Congress and Supreme Court. NARA deployed the first release of
Increment 3—the congressional component—in January 2010.
- Upgrades to the ERA base system to, among other things, search,
view, and print records.
- Public access to provide the public with tools needed to search and
access electronic records. NARA plans to launch a test version of
public access capabilities for NARA staff use in April 2010, and
public use of the system is to begin in December 2010.
- Planning for preservation to include development of a preservation
framework prototype. The prototype is to include the capability to
plan, execute, and monitor preservation activities.
- By June 2010, NARA plans to implement the remaining two releases of
Increment 3.
* Increment 4 is planned to build upon the base architecture delivered
as part of Increment 3, and NARA plans to insert newly available
technology, particularly for preservation capabilities. NARA plans to
begin work on this increment in 2010 and complete it in fiscal year
2011.
* Increment 5 is expected to expand on system capabilities implemented
in the prior increments.
NARA plans for the ERA system to reach full operational capability
(FOC) in 2012. As of February 2010, the life-cycle cost for ERA
through March 2012 was estimated at more than $567 million, which
includes not only the development contract costs, but also program
management, research and development, and program office support,
among other things.
Table 1 shows reported spending from the program's inception to the
end of fiscal year 2009.
Table 1: Summary of ERA Spending from Fiscal Year 2002 through Fiscal
Year 2009:
Project category: Development Contract—Lockheed Martin;
Spending: $156.0 million.
Project category: System Analysis and Design Contracts—Lockheed Martin
and Harris Corporation;
Spending: $40.8 million.
Project category: Program Management;
Spending: $39.2 million.
Project category: Program Office Support Team;
Spending: $28.0 million.
Project category: Research and Development;
Spending: $22.6 million.
Project category: Integrated Deployment and Support;
Spending: $13.3 million.
Project category: Independent Verification and Validation;
Spending: $7.2 million.
Project category: Security;
Spending: $0.2 million.
Project category: End of Year Balance;
Spending: $0.3 million.
Project category: Adjustments;
Spending: -$1.9[A] million.
Project category: Total;
Spending: $305.6[B] million.
Source: GAO analysis of NARA data.
[A] Recoveries of prior year funds, adjustments to obligations
incurred, obligations against prior years, and carryover funds
expiring at the end of fiscal year 2009.
[B] Total number may not equal the sum of individual items due to
rounding.
[End of table]
NARA's estimated ERA obligations for fiscal year 2010, including both
single-year and multi-year funds, are $85.6 million. Table 2 shows how
NARA planned to distribute funds across the ERA program in fiscal year
2010.
Table 2: Summary of NARA's Fiscal Year 2010 Estimated Obligations for
ERA:
Project Category: Development Contract;
Description: Activities performed under the ERA system acquisition
contract with Lockheed Martin (includes EOP);
Estimated obligations: $59.2 million.
Project Category: Program Management;
Description: Salaries and benefits, supplies, equipment, and
telecommunications;
Estimated obligations: $10.0 million.
Project Category: Research and Development;
Description: Research performed with other agencies;
Estimated obligations: $4.7 million.
Project Category: Program Office Support Team;
Description: Labor, contracts, and materials to support ERA program
management;
Estimated obligations: $5.2 million.
Project Category: Integrated Deployment and Support;
Description: Interagency agreements for ERA facilities[A];
Estimated obligations: $2.0 million.
Project Category: Independent Verification and Validation[B];
Description: Verification and validation activities;
Estimated obligations: $1.8 million.
Project Category: Program Execution;
Description: Contractor program and engineering management;
Estimated obligations: $2.7 million.
Project Category: Total;
Estimated obligations: $85.6 million[C].
Source: GAO analysis of NARA data.
[A] ERA facilities include Allegany Ballistics Lab at Rocket Center,
West Virginia, and the Naval Meteorology and Oceanography Command at
Stennis, Mississippi.
[B] NARA contracted with Northrop Grumman to perform independent
verification and validation on policies and plans produced by the ERA
program and contract deliverables produced by Lockheed Martin.
[C] Total may not equal the sum of individual items due to rounding.
Totals include appropriated funds only.
Note: NARA obligated $30,000 towards security, but this is not shown
in the table because the amount is equal to zero when converted into
millions and rounded to one decimal place.
[End of table]
Background: Prior GAO Work:
We have issued several reports on ERA and its development.[Footnote
11] In November 2009,[Footnote 12] we testified that NARA had
completed two of its five planned increments, but experienced delays
and cost overruns, and several functions planned for the system's
initial release were deferred. We further testified that although NARA
initially planned for the system to be capable of ingesting federal
and presidential records in September 2007, the two system increments
to support those records did not achieve initial operating capability
until June 2008 and December 2008, respectively. In addition, NARA
reportedly spent about $80 million on the base increment, compared to
its planned cost of about $60 million.
In our review[Footnote 13] of NARA's fiscal year 2009 expenditure
plan, we made four observations about the expenditure plan and ERA
acquisition:
* The fiscal year 2009 expenditure plan did not specifically identify
whether completed system increments included all planned functionality
or what functionality would be included in future increments,
including the outcomes NARA expected from the remainder of its fiscal
year 2009 funding.
* The expenditure plan stated that it relied on earned value
management (EVM) for project management, which is intended to provide
objective reports on program status. However, NARA did not fully
implement practices necessary to make effective use of EVM, limiting
the reliability of its progress reports.
* Although NARA certified IOC for the EOP system in December 2008,
less than 3 percent of the electronic records from the Bush
administration had been ingested into the system at the time of our
review, and NARA did not expect the remainder to be ingested until
October 2009.
* NARA lacked a contingency plan for the ERA system in the event of a
failure or disruption.
Accordingly, we made recommendations to NARA to improve the
expenditure plan and ERA acquisition.
[End of section]
Results: Legislative Conditions:
Objective 1: NARA's expenditure plan satisfies five of the fiscal year
2010 legislative conditions and partially satisfies one.
Table 3: Fiscal Year 2010 Expenditure Plan Provisions for Satisfying
Legislative Conditions:
Legislative condition: 1. Meets OMB capital planning and investment
control review requirements;
Status: Partially satisfied;
Expenditure plan provisions: OMB requires agencies to develop capital
review processes that planning and help ensure that projects are being
implemented at acceptable cost and within reasonable and investment
control expected time frames and that they are contributing to
observable improvements in mission performance. In order to do this,
agencies should establish an oversight entity that periodically
reviews capital assets (e.g., the ERA system) to determine how mission
requirements might have changed and whether the asset continues to
fulfill mission requirements and deliver intended benefits to the
agency and customers. Further, agencies should indicate (i.e.,
document) that the investment has been reviewed and approved by the
responsible oversight entity. Additionally, best practices such as our
IT investment management framework call for oversight boards to take
corrective actions at the first sign of cost, schedule, and
performance problems. They also call for oversight boards to ensure
that corrective actions and related efforts are executed by the
project management team and tracked until the desired outcomes occur.
NARA partially meets this condition; it has established groups to
oversee ERA's progress and provided them with regular briefings.
However, NARA does not document the results of briefings to its senior
management oversight group and thus there is little evidence that this
body has reviewed and approved the progress of the ERA system. There
is also little evidence that the group identified or took appropriate
corrective actions or ensured that the actions were taken and tracked
to closure. Without adequate oversight that evaluates ERA progress,
including documenting feedback and action items from senior
management, NARA will not be able to ensure that the system is being
implemented at acceptable cost and within reasonable and expected time
frames.
Legislative condition: 2. Complies with NARA's enterprise architecture;
Status: Satisfied;
Expenditure plan provisions: OMB requires NARA to include ERA in its
agency-level enterprise architecture, which is updated on a yearly
basis. NARA has developed an agencywide enterprise architecture that
includes ERA. The current agency enterprise architecture—version 5.5
updated in May 2009—includes ERA and consists of several component
architectures, including business, data, systems, application,
operations, and information technology security architectures. In
addition, OMB requires that any major IT investment be mapped to and
support the Federal Enterprise Architecture. The business case for the
investment must also demonstrate the relationship between the
investment and the business, performance, data, services, application,
and technology layers of the agency's architecture. NARA's budget
submission business case for the ERA system certifies compliance with
these requirements and was approved by OMB.
Legislative condition: 3. Conforms with NARA's enterprise life-cycle
methodology;
Status: Satisfied;
Expenditure plan provisions: The ERA project conforms to NARA's life-
cycle methodology. For example, the expenditure plan includes
descriptions of the incremental approach the agency has adopted for
acquiring ERA and its management of program risks. In particular, the
risk management methodology calls for the agency to identify and
categorize risks, qualify the probabilities and consequences of the
risks, specify a strategy to mitigate each risk, communicate risk
status, and formulate actions needed to mitigate the risk. NARA
manages risks using an agency-level risk review board, a program-level
risk review board, and a technical risk review team. In addition, the
ERA program office produces monthly reports that include top
identified risks and specify associated mitigation strategies. The
office also generates reports of pending or active risks from its risk
management database that specify the probability and consequences of
identified risks. Further, risk status is communicated to senior NARA
management and OMB on a monthly basis and Congress on a quarterly
basis. The quarterly reports also identify executive actions needed to
mitigate risks.
Legislative condition: 4. Complies with the acquisition rules,
requirements, guidelines, and systems acquisition management practices
of the federal government[A].
Status: Satisfied;
Expenditure plan provisions: NARA satisfied this provision by
implementing key processes that reflect best practices for acquiring
software-intensive systems, like ERA. The quality of software is
governed largely by the quality of the processes involved in
developing or acquiring it and maintaining it. Carnegie Mellon
University's Software Engineering Institute (SEI),[B] recognized for
its expertise in software processes, has developed models and methods
that define and determine organizations' software process maturity.
NARA conducted internal assessments in 2002 and 2004 that used the
institute's SA-CMM[C] methods to determine the maturity of ERA's
system policies, processes, and practices and implemented a process to
address the assessment's recommendations. In addition, NARA's Chief
Information Officer certified that the ERA program continues to be in
compliance with the Clinger Cohen Act on November 19, 2009.[D]
Legislative condition: 5. Approved by NARA and OMB;
Status: Satisfied;
Expenditure plan provisions:
* NARA—October 2009;
* OMB—October 2009.
Legislative condition: 6. Reviewed by GAO
Status: Satisfied;
Expenditure plan provisions: GAO — April 6, 2010, briefing to
congressional appropriations subcommittees.
Sources: GAO analysis of NARA data and the Consolidated Appropriations
Act, 2010.
[A] We did not review the program's compliance with the Federal
Acquisition Regulation or other federal requirements beyond those
encompassed by the Software Engineering Institute's Capability
Maturity Model.
[B] SEI is a federally funded research and development center operated
by Carnegie Mellon University and sponsored by the Department of
Defense. Its objective is to provide leadership in software
engineering and in the transition of new software engineering
technology into practice.
[C] The Software Acquisition-Capability Maturity Model (SA-CMM)
identifies key process areas that are essential to effectively
managing software-intensive system acquisitions.
[D] Among other things, the Clinger-Cohen Act of 1996 required OMB to
establish processes to analyze, track, and evaluate the risks and
results of major capital investments in IT systems made by executive
agencies. As such, OMB developed policy and issued guidance for the
planning, budgeting, acquisition, and management of federal capital
assets.
[End of table]
Results: Prior Recommendation Status:
Objective 2: NARA has partially implemented our previous
recommendations.
In May 2009, we made five recommendations to NARA to help improve the
expenditure plan and ERA acquisition.
NARA has implemented three of our recommendations and partially
implemented two.
Table 4: Status of NARA's Progress in Implementing Prior GAO
Recommendations:
Prior GAO recommendation: Report to Congress on the specific outcomes
to be achieved by ERA program funding for the remainder of fiscal year
2009;
Implementation status: Implemented;
Status as of fiscal year 2010 plan: NARA provided quarterly reports to
Congress that identified outcomes to be achieved with the remainder of
ERA program funding for fiscal year 2009. These reports covered
reporting periods March-June 2009 and July-September 2009. The reports
include information on program costs, schedule, and performance.
Prior GAO recommendation: Include in NARA’s next expenditure plan an
analysis of the costs and benefits of using the EOP system to respond
to presidential records requests compared to other existing systems
currently being used to respond to such requests;
Implementation status: Implemented;
Status as of fiscal year 2010 plan: NARA’s 2010 expenditure plan
includes a detailed explanation of alternatives considered,
disadvantages of these alternatives, advantages, and the cost of using
the EOP system to respond to presidential records requests compared to
that of other existing systems.
Prior GAO recommendation: Develop and implement an ERA contingency
plan that follows contingency guidance for federal systems;
Implementation status: Implemented;
Status as of fiscal year 2010 plan: NARA developed and tested an ERA
contingency plan. Testing was completed on August 5, 2009, and the
plan was finalized on September 16, 2009.
Prior GAO recommendation: Provide detailed information in future
expenditure plans on what was spent and delivered for deployed
increments of the ERA system and cost and functional delivery plans
for future increments;
Implementation status: Partially implemented;
Status as of fiscal year 2010 plan: NARA's fiscal year 2010
expenditure plan includes additional information on ERA cost,
schedule, and performance. For example, NARA's 2010 plan discusses the
amount it planned to spend on Increment 2, as reported in the fiscal
year 2009 plan ($11.1 million), and the actual amount spent ($10.4
million). It also compares planned and actual delivery dates (both
December 2008). In addition, the plan identifies the overall cost for
Increment 3 ($25.6 million for fiscal year 2009 and $16.6 million for
fiscal year 2010, for a total of $42.2 million) and the planned
completion date for the increment (fourth quarter of fiscal year
2010). The plan also includes a high-level discussion of Increment 4
plans and associated costs for fiscal year 2010. However, it does not
fully discuss other key information. For example, although it
estimates the funding needed for Increment 4 in 2010, it does not
address the estimated total cost of the increment. In addition, the
plan does not fully describe how NARA will use 2010 funds for public
access and preservation capabilities, because it does not describe
which records (by type, volume, or source) will be made available or
preserved with 2010 funding. Finally, the plan does not discuss ERA's
estimated life-cycle cost estimate or the reasons for increases in the
estimate. Without this key information, the plan is of limited value
for overseeing program progress.
Prior GAO recommendation: Strengthen the earned value process so that
it follows practices described in GAO’s cost estimating guide and more
reliable cost, schedule, and performance information can be included
in future expenditure plans and monthly reports;
Implementation status: Partially implemented;
Status as of fiscal year 2010 plan: In response to our recommendation,
NARA developed but has not fully implemented an action plan to improve
its earned value processes to follow 13 best practices described in GAO’
s Cost Estimating Guide.[A] For 2 of the best practices we assessed as
not met, NARA plans to conduct a risk assessment analysis as part of
Increment 3. Other actions are planned but not yet implemented to
address the 6 practices we assessed as partially met. For example, the
ERA program office is developing a program management tool to
establish a joint performance measurement baseline between contractor
and government activities. We have ongoing work to evaluate the
adequacy of NARA’s EVM processes and implementation of its action plan.
Source: GAO analysis of NARA data.
[A] GAO, Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, GAO-09-3SP (Washington,
D.C.: March 2009).
[End of table]
Results: Observations:
Objective 3: Observations about NARA's ERA Expenditure Plan and
Acquisition:
Observation 1: The ERA system is experiencing expected cost increases
and schedule delays.
According to NARA officials, between fiscal years 2008 and 2009, the
ERA estimated life-cycle cost increased by about $20 million, from
about $531 million to $551.4 million.[Footnote 14] From fiscal year
2009 to 2010, the estimated life-cycle cost increased by an additional
$16 million, from $551.4 million to $567.4 million. This represents a
cumulative increase of 7 percent over the last three fiscal years.
NARA attributed the cost increase of $16 million to the increasing
complexity of the system being developed as determined by both NARA
and contractor subject matter experts, such as system engineers.
In addition, NARA currently plans to complete Increment 3 in June
2010, and Increment 4 in mid-2011, both of which are approximately 1
year later than the milestones established in its project management
plan. The completion date for Increment 3 is also inconsistent with
information in last year's plan, which included a graphic indicating
that Increment 3 would be completed early in 2010. According to NARA's
acquisition strategy and the project management plan, dated January
2005 and April 2006 respectively, Increment 3 was to be delivered
during calendar year 2009. NARA officials stated that the June 2010
delivery date is consistent with the current project schedule, which
was the result of a rebaselining in September 2006. However NARA
officials told us there is no written evidence to show executive
approval of this schedule change or link its current schedule to the
rebaselining effort because, according to NARA officials, it revised
its development methodology and determined that revising its program
documents to reflect the change would not be cost effective.
Further, ongoing updates to existing ERA increments have also fallen
behind schedule. Although NARA certified Increments 1 and 2 as
complete in 2008, it has continued to make enhancements to the
functionality of both increments. Specifically, fiscal year 2009 funds
provided for analysis, design, software development, integration and
testing, acceptance testing, and deployment of additional software
releases for Increment 1. These releases included, among other things,
functionality that enhanced the system's ability to ingest large
record files as well as properly displaying content in supported
Internet browsers. Of the six enhancements completed since March 2009,
four were delayed by between 17 and 140 days. NARA attributes these
schedule delays to deficiencies identified during software testing as
well as subsequent time needed to correct those deficiencies.
Continued cost increases and delays raise concerns about NARA's
ability to fully develop and implement the ERA system within budget
and as scheduled for full operating capability in fiscal year 2012.
Observation 2: NARA lacks detailed plans for completing the final two
increments of ERA.
Even though NARA plans to spend the bulk of its contract development
funds for 2010 on Increment 4, it has not fully defined the
functionality to be included in Increments 4 or 5. For Increment 4,
NARA requested about $37 million in fiscal year 2010 funds to
implement or expand a variety of capabilities, including those related
to preservation, access, ERA base, and backup and restore. About $18
million of this amount is dedicated to developing a framework for
preservation capabilities. However, while the expenditure plan
provides high-level descriptions of these capabilities, NARA does not
have fully defined plans, including specific dates for completion and
what associated capabilities are to be delivered. According to agency
officials, NARA is developing an addendum to the 2010 plan that will
expand on its plans for the remainder of the fiscal year.
In addition, NARA has not established specific plans for Increment 5.
While the prior year's expenditure plan provided a high-level
description of functionality to be included in Increment 5, such as
expanded preservation capacity and collaboration with other agencies,
the fiscal year 2010 plan does not include detailed discussion of
planned functionality for Increment 5. The 2010 plan's sole reference
to planned functionality for Increment 5 is a single item, related to
schedule resolution, in a table listing functions deferred from
Increment 1. Without including detailed plans for the final two
increments, including specific dates for completion and associated
functionality to be delivered, Congress will have limited insight to
evaluate NARA's ongoing progress. NARA intends to discuss Increment 5
in more detail in the 2011 expenditure plan.
Observation 3: NARA has not adequately defined or managed ERA's system
requirements.
System requirements describe the functionality needed to meet user
needs and perform as intended in the operating environment and should
be clearly defined and managed throughout the project in a disciplined
way. Specifically:
* Well-defined requirements are documented at a high (business) level
and are traceable forward and backward throughout the project's life
cycle, in order to ensure that the system's functionality satisfies
the intended objectives of the higher-level requirements.
* Well-managed requirements are maintained through a formal process
that involves updating changes to requirements in response to
modifications of project scope. As work products are developed and
more is learned about the system that is being developed, information
is occasionally found that requires a change to the original
requirements. Therefore, projects need to manage these changes to
requirements in a structured way.[Footnote 15]
In 2003, NARA finalized a Requirements Management Plan for ERA which,
among other steps, calls for system requirement reviews near the
beginning of each increment.
Although NARA developed a set of baseline system requirements for ERA,
these requirements are not fully traceable to work in later system
increments. NARA developed and documented a set of high-level business
requirements for ERA designed to meet the needs of a variety of users
such as NARA managers, researchers, and other federal agencies. NARA's
Requirements Document, which represents the system's full operating
capability, identifies 853 requirements that are to be developed and
fully implemented by 2012. However, our analysis of these high-level
requirements determined that more than 40 percent have yet to be
allocated to any portion of the system development. Specifically, we
found that 489 (57.3 percent) of the high-level requirements will be
implemented as part of Increments 1 through 3 by fiscal year 2010, but
that the remaining 364 requirements (42.7 percent) have not been
allocated to the remaining two increments. NARA officials stated that
it is uncertain when they will be allocated or whether they will be
implemented at all. Since these requirements have not been allocated,
it is unclear whether the system development work to be performed as
part of increments 4 and 5 will result in functionality that satisfies
the objectives of the higher-level requirements.
Further, NARA has not updated the requirements document in response to
changes as the project progressed. NARA acknowledged in its fiscal
year 2010 Expenditure Plan that in response to technical challenges,
it reinterpreted some of the requirements stated in the original
Requirements Document but has not updated the document. NARA
recognized that the lack of a current set of requirements is a
significant risk, stating that a system designed and built without a
clear understanding of the Requirements Document and strict adherence
to its intent may not meet NARA's needs and user expectations.
Failure to follow its guidance on requirements management compounded
problems with NARA's requirements development for ERA. Specifically,
NARA conducted and documented system requirements reviews for
increments 1 and 2, as called for in its requirements management plan,
but there is no documentation demonstrating that such a review was
conducted at the start of Increment 3. According to NARA officials, it
conducted ongoing requirements reviews with the contractor throughout
Increment 3, but did not document its reviews until recently because
of a change in its development approach. However, our review of the
Increment 3 documentation did not reveal any evidence of a
requirements review at the start of the increment. In addition, after
raising the issue with agency officials, they informed us that they
had scheduled a system requirements review for Increment 4 for the
week of March 29th, at which they plan to assess whether business
process changes have rendered some of the original requirements no
longer necessary.
Without appropriately traceable and current ERA system requirements,
NARA will have little assurance that ongoing development work is
contributing to its mission needs. Further, NARA will lack an adequate
basis for ensuring that its contractor is providing work that meets
the needs of the agency and, ultimately, the system's users.
[End of section]
Conclusions:
While NARA's fiscal year 2010 expenditure plan met five of the six
legislative conditions, the lack of documentation demonstrating the
appropriate level of senior management review, approval, and oversight
limits NARA's ability to ensure that the project makes expected
progress. In addition, the lack of a documented current baseline
schedule leaves NARA with limited ability to identify significant
delays. The lack of key oversight documentation limits NARA's
assurance that schedule delays and cost increases will not continue
during ERA's development.
In addition, without specific plans for completing the final two
increments, NARA lacks assurance that the remaining increments will
adequately fulfill its mission needs. Specifically, NARA's failure to
follow its own guidance on requirements management has resulted in a
set of requirements that is incomplete and out of date, which could
lead to the system being completed without addressing all necessary
requirements or the development of functionality to meet requirements
that are no longer valid. Addressing these weaknesses is becoming even
more critical as the projected completion of the project approaches;
if they continue, it will be increasingly unlikely that NARA will be
able to deliver the ERA system by 2012 with the capabilities
originally envisioned or to effectively use the system to meet the
needs of its users in support of NARA's mission of preserving and
providing access to the nation's electronic records.
[End of section]
Recommendations for Executive Action:
To enhance NARA's ability to complete the development of ERA within
reasonable funding and time constraints, we are recommending that the
Archivist of the United States take the following two actions:
* Ensure that NARA's investment review process has adequate executive-
level oversight by maintaining documentation of the results of
reviews, including changes to the program's cost and schedule baseline
and any other corrective actions taken as a result of changes in ERA
cost, schedule, and performance.
* Ensure that ERA's requirements are managed using a disciplined
process that results in requirements that are traceable throughout the
project's life cycle and are kept current.
[End of section]
Agency Comments and Our Evaluation:
In a letter commenting on a draft of this briefing, the Archivist of
the United States described several of the program's recent
accomplishments and stated that he would take steps to address our
recommendations, including documenting management reviews and actions
and ensuring adequate discipline is maintained in the definition and
allocation of requirements. However, he also stated that the briefing
did not accurately describe the program in its current state, adding
that NARA had strengthened its management oversight of the program
which has moved forward, meeting the baseline schedule established in
a corrective action plan for Increment 1. In addition, he stated that
the program's history of success does not support our conclusion that
ERA is unlikely to be completed as planned by 2012. We disagree for
several reasons. First, as discussed in our briefing, our discussion
on timeliness focuses on the increments currently being developed
(Increments 3 and 4) and is based on ERA's currently documented
management plan and information from last year's expenditure plan.
NARA did not provide any evidence to demonstrate the existence of the
rebaselined schedule described by the Archivist. Second, the lack of
documentation supporting key management activities we describe—
including performance against the cost and schedule baseline,
executive oversight, and requirements management—apparently contradict
the robust oversight described by the Archivist. Finally, while NARA
has made progress on several capabilities in the past year, the extent
to which these capabilities are fulfilling the project's original
goals cannot be determined due to NARA's lack of an adequate
requirements management process. With the lack of adequate controls in
several key areas and only about two years remaining to complete
development, we believe that on-time completion of ERA that meets all
of its original requirements is increasingly unlikely.
The Archivists comments are attached.
[End of section]
Attachment 1: Comments from the Archivist of the United States:
National Archives and Records Administration:
8601 Adelphi Road:
College Park Maryland 20740-6001:
[hyperlink, http://www.archives.gov]
April 2, 2010:
Government Accountability Office:
Director of Information Technology Management Issues:
Mr. David A. Powner:
441 G Street NW:
Washington, DC 20548:
Dear Mr. Powner:
We thank you for the opportunity to review and comment on the advance
copy of the GAO briefing, Review of the National Archives and Records
Administration's Fiscal Year 2010 Electronic Records Archive
Expenditure Plan for staff members of the Subcommittees on Financial
Services and General Government, Committees on Appropriations, U.S.
Senate and US. House of Representatives.
It has been my understanding that the GAO reports in previous years
have been very useful to improve the management of the program by the
ERA Program staff. However, I am concerned that this briefing does not
accurately describe the program in its current state. Specifically,
the briefing focuses on issues which may have been appropriate in
previous years but which are no longer valid.
As you know the program experienced some delays and cost overruns in
the early stages of Increment 1. Since that time, NARA has
strengthened its management oversight of the program, and as a result
the program has moved forward steadily meeting the baseline schedule
established during the corrective action plan for Increment 1. As you
noted in the briefing, ERA has stayed well within the plus/minus 10%
cost threshold established by the Office of Management and Budget
(OMB) for the work to date.
Our history of success, both in terms of accomplishments, schedule and
cost, does not seem to support your observation regarding assurances
that ERA will not be able to avoid additional cost increases and
schedule delays or that it is unlikely that NARA will be able to
deliver a system by the planned date of 2012 that will effectively can
be used to meet the needs of its users in support of NARA's mission.
We are very proud of the accomplishments of the program and look
forward to more success as we prepare for our next deployment, the
Online Public Access (OPA) instance. Examples of our success in
meeting major accomplishments include:
* Deployment of the Increment 1 Initial Operating Capability ("Base
Instance") which is now in use by NARA staff and staff from four
"early adopter" agencies. Twenty-one additional agencies will be added
within the next few months. By the end of the year we anticipate the
base instance to store over one terabyte of records.
* Deployment of the Executive Office of the President instance, which
now stores and provides search and access capabilities for
approximately 78 terabytes of electronic records, including
photographs from the George W. Bush Administration. Over 54,000
searches have been conducted since operations began.
* Deployment of the Congressional Records Instance, which on March 11,
2010 ingested two transfer groups of Homeland Security Committee
electronic records from the House of Representatives, including MS-
Office files, pdf, film, databases, audio, video and other multimedia
content. We remain ready to ingest additional files as they are
provided to us.
* Design of a pilot to provide public access to NARA's electronic
records, which will be deployed to the public later this year.
* Design of a prototype demonstrating a framework for preservation
technologies, most of which are at an early stage of development.
While I an, confident that structures now in place are adequate to
ensure that the program will continue to meet its cost and schedule
goals, we will take steps to address your recommendations. First, we
will ensure that an adequate audit trail is maintained to document
management reviews and actions. Second, we will review and adapt, as
necessary, our existing requirements management proems to ensure
adequate discipline is maintained in the definition and allocation of
requirements.
Again, I thank you for this opportunity and look forward to our future
interactions as we continue the ERA acquisition process.
Sincerely,
Signed by:
David S. Ferriero:
Archivist of the United States:
[End of Appendix I]
Appendix II: Comments from the National Archives and Records
Administration:
National Archives at College Park:
8601 Adelphi Road:
College Park, Maryland 20740-6001:
May 21, 2010:
Government Accountability Office:
Director of Information Technology Management Issues:
Mr. David A. Powner:
441 G Street NW:
Washington DC, 20548:
Dear Mr. Powner:
We thank you for the opportunity to review and comment on the draft
"Electronic Records Archives Status Update on the National Archives
and Records Administration's Fiscal Year 2010 Expenditure Plan."
This letter responds to the two conclusions provided in the report. To
address your conclusion regarding the level of senior management
review, approval and oversight, we are now documenting the minutes and
actions for Senior Staff meetings where ERA is discussed.
In response to your concern that ERA requirements have not been
defined and managed, NARA has not made any changes to the original
requirements documented in the 2003 Requirements Document. We have
recently begun an intensive effort to review the requirements
established in 2003 within the context of new technologies and
changing business needs, such as the impact of the newly established
National Declassification Center on ERA requirements relative to the
protection of restricted information and requests for records made
under the Freedom of Information Act.
Further, in April, 2010, NARA managers met to determine top ERA
functional priorities to be completed by the end of the current
contract. Concurrently, the ERA requirements manager is mapping those
functional priorities to their underlying contract requirements and
will be updating the ERA Requirements Document as appropriate. The
goal of these efforts is to identify the requirements from the
Requirements Document that remain to be developed by March 2012.
Again, I thank you for this opportunity and look forward to our future
interactions as we continue the ERA acquisition process.
Sincerely,
Signed by:
David S. Ferriero:
Archivist of the United States:
National Archives and Records Administration:
[End of section]
Appendix III: GAO Contact and Staff Acknowledgments:
GAO Contact:
David A. Powner (202) 512-9286 or pownerd@gao.gov:
Staff Acknowledgments:
In addition to the individual named above, key contributions to this
report were made by James R. Sweetman, Jr., Assistant Director; Monica
Perez Anatalio; Pamlutricia Greenleaf; Kush K. Malhotra; Lee A.
McCracken; and Karl W. Seifert.
[End of section]
Footnotes:
[1] Consolidated Appropriations Act, 2010, Pub. L. No. 111-117, div.
C., title V, 123 Stat. 3034, 3193 (Dec. 16, 2009).
[2] GAO, Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, [hyperlink,
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009).
[3] Consolidated Appropriations Act, 2010, Pub. L. No. 111-117, div.
C, title V, 123 Stat. 3034, 3193 (Dec. 16, 2009).
[4] Agencies develop an exhibit 300, also known as the Capital Asset
Plan and Business Case Summary, to justify each request for a major
information technology investment. OMB sets forth requirements for the
exhibit 300 in Circular A-11, Part 7, Planning, Budgeting,
Acquisition, and Management of Capital Assets.
[5] GAO, Cost Estimating and Assessment Guide: Best Practices for
Developing and Managing Capital Program Costs, [hyperlink,
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009).
[6] 44 U.S.C. § 2203(f)(1).
[7] According to the Federal Acquisition Regulation, a firm-fixed-
price contract provides for a price that is not subject to any
adjustment on the basis of the contractor's cost experience in
performing the contract. This type of contract places on the
contractor maximum risk and full responsibility for costs and
resulting profit or loss. 48 C.F.R. § 16.202-1.
[8] A record schedule is a document that describes agency records,
establishes a period for their retention by the agency, and provides
mandatory instructions for what to do with them when they are no
longer needed for current government business.
[9] Records appraisal is the process of determining the value and the
final disposition of records, making them either temporary or
permanent.
[10] These are requests NARA receives from the current and former
administrations, the Congress, and the courts for access to
presidential records. The priorities are determined by NARA's Office
of Presidential Libraries based on experience with the records of
previous administrations.
[11] GAO, Information Management Challenges in Managing and Preserving
Electronic Records, [hyperlink,
http://www.gao.gov/products/GAO-02-586] (Washington, D.C.: June 17,
2002); Records Management: Planning for the Electronic Records
Archives Has Improved, [hyperlink, http://www.gao.gov/products/GAO-04-
927] (Washington, D.C.: Sept. 23, 2004); Information Management
Acquisition of the Electronic Records Archives Is Progressing, vGAO-05-
802 (Washington, D.C.; July 15, 2005); Electronic Records Archives:
The National Archives and Records Administration's Fiscal Year 2006
Expenditure Plan, [hyperlink, http://www.gao.gov/products/GAO-06-906]
(Washington, D.C.: Aug. 18, 2006); Information Management The National
Archives and Records Administration's Fiscal Year 2007 Expenditure
Plan, [hyperlink, http://www.gao.gov/products/GAO-07-987] (Washington,
D.C.: July 27, 2007); and Information Management: Challenges in
Implementing an Electronic Records Archive, [hyperlink,
http://www.gao.gov/products/GAO-08-738T] (Washington, D.C.: May 14,
2008).
[12] GAO, National Archives: Progress and Risks in Implementing its
Electronic Records Archive Initiative, [hyperlink,
http://www.gao.gov/products/GAO-10-222T] (Washington, D.C.: Nov. 5,
2009).
[13] GAO, Electronic Records Archive: The National Archives and
Records Administration's Fiscal Year 2009 Expenditure Plan,
[hyperlink, http://www.gao.gov/products/GAO-09-733] (Washington, D.C.:
July 24, 2009).
[14] During a previous review, NARA officials told us that the
estimated life-cycle cost in 2008 was $453 million. However, after
reviewing information being used to develop this briefing, NARA
officials told us that the figure they provided previously did not
include costs for 2012, which account for approximately $78 million in
additional costs. NARA did not document its life-cycle cost estimates
in expenditure plans for fiscal years 2008 through 2010.
[15] GAO, Business System Modernization: IRS Needs to Complete Recent
Efforts to Develop Policies and Procedures to Guide Requirements
Development and Management, [hyperlink,
http://www.gao.gov/products/GA0-06-310] (Washington, D.C.: March 2006).
[End of section]
GAO's Mission:
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people.
GAO examines the use of public funds; evaluates federal programs and
policies; and provides analyses, recommendations, and other assistance
to help Congress make informed oversight, policy, and funding
decisions. GAO's commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony:
The fastest and easiest way to obtain copies of GAO documents at no
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each
weekday, GAO posts newly released reports, testimony, and
correspondence on its Web site. To have GAO e-mail you a list of newly
posted products every afternoon, go to [hyperlink, http://www.gao.gov]
and select "E-mail Updates."
Order by Phone:
The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site,
[hyperlink, http://www.gao.gov/ordering.htm].
Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537.
Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional
information.
To Report Fraud, Waste, and Abuse in Federal Programs:
Contact:
Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]:
E-mail: fraudnet@gao.gov:
Automated answering system: (800) 424-5454 or (202) 512-7470:
Congressional Relations:
Ralph Dawn, Managing Director, dawnr@gao.gov:
(202) 512-4400:
U.S. Government Accountability Office:
441 G Street NW, Room 7125:
Washington, D.C. 20548:
Public Affairs:
Chuck Young, Managing Director, youngc1@gao.gov:
(202) 512-4800:
U.S. Government Accountability Office:
441 G Street NW, Room 7149:
Washington, D.C. 20548: