This is the accessible text file for GAO report number GAO-10-40 
entitled 'Information Technology: Actions Needed to Fully Establish 
Program Management Capability for VA's Financial and Logistics 
Initiative' which was released on December 1, 2009. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Report to the Committee on Veterans' Affairs, House of Representatives: 

United States Government Accountability Office: 
GAO: 

October 2009: 

Information Technology: 

Actions Needed to Fully Establish Program Management Capability for 
VA's Financial and Logistics Initiative: 

GAO Highlights: 

Highlights of GAO-10-40, a report to the Committee on Veterans' 
Affairs, House of Representatives. 

Why GAO Did This Study: 

Since 2005, the Department of Veterans Affairs (VA) has been 
undertaking an initiative to develop an integrated financial and asset 
management system known as the Financial and Logistics Integrated 
Technology Enterprise (FLITE). FLITE is the successor to an earlier 
initiative known as the Core Financial and Logistics System (CoreFLS) 
that the department undertook in 1998 and discontinued in 2004 because 
it failed to support VA’s operations. In light of the past performance 
of CoreFLS and the Office of Management and Budget’s designation of 
FLITE as high risk, GAO was asked to (1) determine the status of pilot 
system development and (2) evaluate key program management processes, 
including VA’s efforts to institute effective human capital management, 
develop a reliable program cost estimate, use earned value management 
(a recognized means for measuring program progress), establish a 
realistic program schedule, employ effective requirements development 
and management, and perform independent verification and validation. To 
do so, GAO reviewed program documentation and interviewed relevant 
officials. 

What GAO Found: 

Contract award and performance of work tasks had been started for one 
of two planned pilot systems—the Strategic Asset Management system. 
However, as of mid-September, the project had fallen behind (by 2 
months) and the contractor had missed the deadline for initiating and 
completing planned tasks and delivering work products such as a system 
security plan. In particular, the contractor had not started 11 of 34 
tasks, including conducting a security assessment, and was behind 
schedule on 16 of the remaining 23 tasks, including analyzing business 
processes. Program officials generally attributed the delays to VA 
having insufficient program and acquisition staff to perform necessary 
activities associated with awarding and executing the pilot contract 
and to poor project management by the pilot system contractor. A second 
project—for the Integrated Financial Accounting System pilot—is 
expected to start in October 2009. 

VA has taken steps to institute effective management of FLITE; however, 
the department has not yet fully established capabilities needed to 
ensure that the program will be successfully implemented. Specifically, 
VA has: 

* recently filled long-standing staff vacancies, and only one program 
office staff opening remains; 

* not developed a cost estimate that includes total program costs or 
reconciled its estimate with an independent estimate; 

* not performed key actions necessary for reliable earned value 
management; 

* not yet established a schedule that is reliable; 

* not identified all mandatory federal financial management system 
requirements and ensured that system requirements are based on business 
requirements; and; 

* not addressed all of the findings of its independent verification and 
validation organization in a timely manner. 

Until VA reconciles its cost estimate, ensures compliance with earned 
value management system standards, establishes a reliable schedule, 
ensures all relevant federal and system requirements are identified and 
traceable, and addresses all independent verification and validation 
findings, it could continue to experience schedule delays and further 
increase its risk of not providing the financial and asset management 
capabilities that users need. 

What GAO Recommends: 

GAO is making recommendations aimed at improving program management. In 
written comments on a draft of this report, VA concurred with the 
recommendations and identified actions to address them. 

View [hyperlink, http://www.gao.gov/products/GAO-10-40] or key 
components. For more information, contact Valerie C. Melvin at (202) 
512-6304 or melvinv@gao.gov. 

[End of section] 

Contents: 

Letter: 

Background: 

Pilot Project Is Behind Schedule: 

VA Has Recently Filled FLITE Program Staff Positions but Has Not Yet 
Fully Established Other Program Management Capabilities: 

Conclusions: 

Recommendations for Executive Action: 

Agency Comments and Our Evaluation: 

Appendix I: Objectives, Scope, and Methodology: 

Appendix II: Comments from the Department of Veterans Affairs: 

Appendix III: GAO Contacts and Staff Acknowledgments: 

Table: 

Table 1: Description of FLITE Program Office Entities and Supporting VA 
Organizations: 

Figures: 

Figure 1: Simplified View of FLITE Components: 

Figure 2: Relationship of VA's Oversight and Management Structure to 
FLITE Program: 

Figure 3: Simplified FLITE Program Structure and Supporting VA 
Organizations: 

Figure 4: FLITE Timeline: 

Abbreviations: 

ANSI: American National Standards Institute: 

CoreFLS: Core Financial and Logistics System: 

EIA: Electronic Industries Alliance: 

EVM earned value management: 

FLITE: Financial and Logistics Integrated Technology Enterprise: 

FMS: Financial Management System: 

FSIO: Financial Systems Integration Office: 

IFCAP: Integrated Funds Distribution, Control Point Activity, 
Accounting, and Procurement: 

IFAS: Integrated Financial Accounting System: 

IT: information technology: 

MQAS: Management Quality Assurance Service: 

OIG: Office of Inspector General: 

OMB: Office of Management and Budget: 

PDO: program director's office: 

PMO: program management office: 

SAM: Strategic Asset Management: 

SEI: Software Engineering Institute: 

SPAWAR: Space and Naval Warfare Systems Command: 

VA: Department of Veterans Affairs: 

[End of section] 

United States Government Accountability Office: 
Washington, DC 20548: 

October 26, 2009: 

The Honorable Bob Filner: 
Chairman: 
The Honorable Steve Buyer: 
Ranking Member: 
Committee on Veterans' Affairs: 
House of Representatives: 

Since 2005, the Department of Veterans Affairs (VA) has been 
undertaking an initiative to develop an integrated financial management 
and information system known as the Financial and Logistics Integrated 
Technology Enterprise (FLITE). FLITE is the successor to an earlier 
initiative that the department undertook known as the Core Financial 
and Logistics System (CoreFLS). After having reportedly spent more than 
$249 million on its development, the department discontinued CoreFLS 
because the pilot system failed to support VA's operations. 

According to the department, FLITE is intended to fulfill the critical 
need for a modernized and integrated financial and asset management 
capability. Such a capability would support VA's strategic goal to 
deliver world-class service to veterans and their families through 
effective communication and management of people, technology, business 
processes, and financial resources. In early 2007, the Office of 
Management and Budget (OMB) designated FLITE as a high-risk information 
technology investment.[Footnote 1] 

In light of VA's past performance with CoreFLS and the designation of 
FLITE as high risk, you requested that we (1) determine the status of 
pilot system development and (2) evaluate key program management 
processes for the initiative, including VA's efforts to institute 
effective human capital management, develop a reliable program cost 
estimate, use earned value management (EVM), establish a realistic 
program schedule, employ effective requirements development and 
management, and perform independent verification and validation. 

To accomplish our objectives, we reviewed relevant program 
documentation and interviewed appropriate VA and contractor officials. 
Specifically, to determine the status of FLITE pilot system 
development, we reviewed documentation such as program management plans 
and project status reports. To evaluate key FLITE program management 
processes, we compared VA's activities to plans and best practices. 

We performed our work at the Department of Veterans Affairs 
headquarters in Washington, D.C., from November 2008 to October 2009 in 
accordance with generally accepted government auditing standards. Those 
standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. See appendix I for a more 
complete description of our objectives, scope, and methodology. 

Background: 

In recognition of their service to our country, the Department of 
Veterans Affairs (VA) provides medical care, benefits, social support, 
and lasting memorials to veterans and their families. It is the second- 
largest federal department with approximately 250,000 employees. In 
fiscal year 2008, VA reported incurring $97 billion in obligations for 
its overall operations. 

VA provides services to veterans and their families primarily through 
its three line administrations: 

* The Veterans Health Administration operates a nationwide network of 
154 hospitals, 995 outpatient clinics, 135 community living centers, 49 
residential rehabilitation treatment programs, and 232 community-based 
counseling centers. 

* The Veterans Benefits Administration provides assistance and benefits 
such as veterans' compensation, survivors' benefits, and employment 
assistance through 57 veterans' benefits regional offices. 

* The National Cemetery Administration manages 130 national cemeteries. 

To support its services to veterans and their families, VA relies on an 
assortment of business systems, including 13 different systems that 
currently support its asset and financial management. However, the 
department has long recognized that its business systems and processes 
are inefficient and do not effectively support the department's 
mission. For example, according to the department, 

* systems are not integrated, 

* manual entry that involves labor-intensive accounting processes is 
required, 

* business processes are not standardized,[Footnote 2] and: 

* processes and systems require multiple entry of business information 
and result in untimely financial reporting. 

Since fiscal year 1991, the department has reported on the need for an 
integrated financial management system and has reported financial 
management system functionality as a material weakness.[Footnote 3] 
This weakness continues to exist because many of VA's systems are 
outdated, leading to inefficiencies in the reliable, timely, and 
consistent preparation, processing, and analysis of financial 
information for the department's consolidated financial statements. To 
address this weakness and to improve stewardship and accountability 
over its resources, VA has for over a decade been pursuing improvements 
in its business processes and replacement of its existing financial and 
asset management systems with an integrated financial management 
system. 

Assessments of CoreFLS: 

The department's first attempt to replace its financial and asset 
management systems, CoreFLS, began in 1998. The goal of this 
modernization effort was to develop a single system to integrate the 
many financial and asset management systems used across the department. 
VA had planned to complete CoreFLS in March 2006; however, it 
terminated development of the system in July 2004 after CoreFLS pilot 
tests determined it did not fully support the department's operations 
and that the initiative suffered from significant project management 
weaknesses. According to VA's Office of Inspector General (OIG), the 
department had obligated about $249 million of the $472 million that 
had been budgeted for the initiative by the time of its termination. 
[Footnote 4] 

Following the failed CoreFLS pilot tests, VA hired Carnegie Mellon 
University's Software Engineering Institute (SEI) to perform an 
independent assessment of the project. In June 2004, SEI identified a 
number of management and technical deficiencies that had undermined the 
success of the initiative.[Footnote 5] SEI identified multiple findings 
related to problematic technical and functional execution, as well as 
poor management execution. Technical and functional problems included 
CoreFLS's inability to perform essential financial management 
functions, security weaknesses, and usability. Management problems were 
identified in the areas of acquisition and program management, business 
process re-engineering, and transition planning. 

In addition, in August 2004, VA's OIG reported multiple findings 
related to CoreFLS deployment, such as inadequate training, inability 
to monitor fiscal and acquisition operations, inaccurate data, and 
project management and security weaknesses.[Footnote 6] 

Further, in August 2007, VA's Management Quality Assurance Service 
(MQAS) summarized findings from four CoreFLS reviews completed between 
August 2005 and August 2006.[Footnote 7] Among the findings, MQAS 
identified numerous fiscal and contract administration issues resulting 
from poor administrative internal controls such as improper 
reimbursements of task orders and travel expenses. 

Collectively, VA identified 141 findings related to problems with the 
CoreFLS initiative, which the department categorized into functional 
areas of responsibility such as acquisition management, organizational 
change management, program management, and systems engineering. 
[Footnote 8] In a subsequent effort to capture lessons learned[Footnote 
9] and ensure that mistakes from CoreFLS would not be repeated in later 
initiatives, VA developed a repository, in which it aggregated the 
findings from the three independent reviews of the initiative. 

Establishment and Management of the FLITE Program: 

In September 2005, in a subsequent effort to replace its financial and 
asset management systems, VA began work on FLITE. In this regard, the 
department undertook activities related to planning and requirements 
development. For example, the department: 

* documented business requirements and business processes, 

* initiated coordination for reporting and financial data warehouse 
development, 

* conducted a market analysis of providers with the software and 
hosting capability to support VA's existing financial management 
system, 

* established key personnel requirements to provide program support and 
awarded a program support contract, and: 

* started developing numerous planning documents (e.g., program 
management plan, acquisition plan, and concept of operations). 

According to VA's planning documents, FLITE is a multiyear development 
effort that is projected to deliver a fully operational system by 2014 
at a total estimated cost of $608.7 million.[Footnote 10] The overall 
objectives of the FLITE program are to: 

* implement accessible and enterprise-level standardized business 
processes that result in increased efficiencies and enhanced internal 
controls; 

* provide VA executives and managers with timely, transparent financial 
and asset management information to make and implement effective 
policy, management, stewardship, and program decisions; and: 

* provide business data and information in a secure, shareable, open, 
and efficient manner to facilitate a service-oriented atmosphere. 

The FLITE program includes two main projects to acquire the integrated 
asset and financial management system: an asset management component, 
referred to as the Strategic Asset Management (SAM) initiative, and the 
financial management component, referred to as the Integrated Financial 
Accounting System (IFAS). The program also includes a third project, to 
acquire a data warehouse that is intended to provide financial and 
logistics data reporting and analysis. 

* SAM is intended to consolidate the asset and inventory management 
functions and the associated work management processes currently 
performed by multiple legacy applications into an advanced integrated 
system. It is to be the system of record for VA's physical assets and 
perform asset and inventory management, real property management, 
information technology (IT) asset management, and work order and 
project management functions currently performed by multiple legacy 
applications. VA has chosen IBM's Maximo Enterprise Asset Management 
software suite[Footnote 11] to implement these capabilities. 

* IFAS is to be the financial, procurement, and accounting management 
component, and, together with SAM, is intended to replace VA's legacy 
Financial Management System (FMS) and the Integrated Funds 
Distribution, Control Point Activity, Accounting, and Procurement 
(IFCAP)[Footnote 12] system. 

* The data warehouse is projected to consolidate data from multiple 
transactional systems, primarily SAM and IFAS, for improved reporting, 
querying, and analysis capability. It is also intended to allow users 
to run larger and more complex queries and reports faster, without 
affecting the performance of the source systems. 

Figure 1 shows a simplified view of the program's components. 

Figure 1: Simplified View of FLITE Components: 

[Refer to PDF for image: illustration] 

Strategic Asset Management and Integrated Financial Accounting System 
interact, and both provide material to the Data Warehouse. 

Strategic Asset Management: 
Facility management; 
Inventory management; 
Asset management; 
Requisitioning and procurement; 
Item management; 
Real property management; 
Work order management. 

Integrated Financial Accounting System: 
Project accounting; 
Accounting; 
Debt management; 
Reporting; 
Travel; 
Payment management; 
Vendor management; 
Requisitioning and procurement; 
Fixed assets; 
Budget execution. 

Source: GAO analysis of VA data. 

[End of figure] 

The program is a collaborative effort between the Assistant Secretary 
for Information and Technology, who serves as VA's Chief Information 
Officer, and the Assistant Secretary for Management, who serves as VA's 
Chief Financial Officer. Various groups within VA have different roles 
and responsibilities for overseeing and managing programs. Figure 2 
depicts the relationships between these oversight groups and the FLITE 
program. 

Figure 2: Relationship of VA's Oversight and Management Structure to 
FLITE Program: 

[Refer to PDF for image: illustration] 

FLITE Program Office; interaction with: 

FLITE Oversight Board; Budget execution and other issues interaction 
with: 
* Budgeting and Near Term Issues Board; and; 
* Programming and Long Term Issues Board; 
Interaction with: 

Strategic Management Council; interaction with: 

VA Executive Board; Budget execution and other issues interaction with: 
Information Technology Leadership Board. 

Budgeting and Near Term Issues Board; and; 
Programming and Long Term Issues Board; interaction with: 
Information Technology Leadership Board. 

Source: GAO analysis of VA data. 

[End of figure] 

The roles and responsibilities of each oversight group are as follows: 

* The VA Executive Board provides the Secretary of Veterans Affairs 
with a forum for discussing programs with senior leadership before 
decisions are made. 

* The Strategic Management Council makes recommendations about programs 
to the VA Executive Board. 

* The Programming and Long Term Issues Board focuses on long term 
multiyear program planning. 

* The Budgeting and Near Term Issues Board is responsible for 
overseeing budget formulation and execution activities. 

* The IT Leadership Board is responsible for adjudicating inter-and 
intraboard issues about programs that cannot be resolved between the 
Programming and Long Term Issues and Budgeting and Near Term Issues 
Boards. 

* The FLITE Oversight Board is responsible for making decisions 
regarding FLITE business requirements, policies, and standards. 

The FLITE Program Office is responsible for overseeing and coordinating 
all aspects of the program. The office is responsible for performing 
these functions through the Program Director's Office (PDO), which is 
responsible for business requirements and processes, and the IT Program 
Management Office (PMO), which is responsible for technical solutions. 
Project teams are responsible for managing SAM, IFAS, and the data 
warehouse. In addition, other VA organizations provide the office with 
quality assurance, acquisition, and technology support. These program- 
specific and VA supporting organizations are depicted in figure 3. 

Figure 3: Simplified FLITE Program Structure and Supporting VA 
Organizations: 

[Refer to PDF for image: illustration] 

Systems Quality Assurance Service: 
Center for Acquisition Innovation: 
Austin Information Technology Center: 

(supporting organizations for): 

FLITE Program Office: 
* Program Director’s Office; 
* IT Program Management Office; 

IFAS project team: 
SAM project team:
Data warehouse: 

Also provide support to FLITE Program Office. 

Source: GAO analysis of VA data. 

[End of figure] 

Table 1 describes the components that comprise the program office and 
supporting VA organizations. 

Table 1: Description of FLITE Program Office Entities and Supporting VA 
Organizations: 

Title: FLITE Program Office; 
Description: This office includes the PDO and the IT PMO. Together, 
these offices are responsible for overseeing and coordinating all 
aspects of the FLITE program, including systems engineering, 
organizational change management, training, program management, 
communications, and risk and investment management, and for directing 
multidisciplinary efforts of VA and contractor personnel to accomplish 
various tasks. 

Title: IFAS project team; 
Description: This team is responsible for replacing VA's current FMS 
and portions of the IFCAP system. 

Title: SAM project team; 
Description: This team is responsible for consolidating the asset and 
inventory management, real property management, work order, and project 
management functionality currently performed by multiple legacy 
applications. 

Title: Data warehouse; 
Description: This project office has not been formally defined by the 
program office. 

Title: Center for Acquisition Innovation; 
Description: This entity is responsible for providing acquisition 
services to procure supplies and services to support FLITE. 

Title: Austin Information Technology Center; 
Description: This entity is responsible for providing IT enterprise 
solutions to support information technology needs, including providing 
design and conceptual support for IFAS and systems hosting services, 
application administration, and operational support for SAM and the 
financial reporting data warehouse. 

Title: Systems Quality Assurance Service; 
Description: This entity is responsible for managing and providing 
oversight of independent verification and validation activities and 
processes employed throughout the FLITE, SAM, and IFAS system 
development life cycles. 

Source: GAO analysis of VA data. 

[End of table] 

Planned Implementation of FLITE: 

VA is employing a multiphase approach for both the SAM and IFAS 
projects, which are to be implemented by contractors using commercial 
off-the-shelf systems. Specifically, these components are to be 
implemented through sequenced acquisitions and phased deployment and 
integration. The systems are planned to be implemented initially at 
pilot sites and subsequently refined and validated at beta sites before 
national deployment. The purpose of the pilot phase is to perform a 
final validation of the selected commercial off-the-shelf system and 
associated business processes in a production environment, gain 
experience in deploying the system, and obtain acceptance from the user 
community. The beta phase is to further hone the rollout capabilities 
by deploying the system to a limited number of sites that span the 
range of VA's organizational environments. Following the beta phase, 
the department plans to incorporate lessons learned from both phases 
and produce a set of repeatable processes that can be employed during 
national deployment of the system. 

For SAM, the department's plans include implementation at one pilot 
site and 15 beta sites. The SAM pilot contractor is to evaluate and 
analyze VA's business processes and requirements for a fit with the 
Maximo software's capabilities and produce updated business process 
documents based on the department's needs. Also, the contractor is to 
train the users at the pilot site, as well as provide operations and 
maintenance and help desk services. The pilot phase is expected to last 
for an estimated 12 months. Subsequent to the pilot, the department 
plans to deploy SAM at 15 VA beta sites over a period of approximately 
12 months. The component is expected to be deployed nationwide over 21 
months, with its completion expected by May 2013. 

Plans for IFAS include implementing the FMS replacement at five pilot/ 
beta sites and implementing the IFCAP replacement at two pilot/beta 
sites. The IFAS pilot phase is currently scheduled to begin in the 
first quarter of fiscal year 2010. The department plans to deploy this 
component in two separate subphases over approximately 4 years. The 
first subphase, which will replace FMS with a commercial off-the-shelf 
financial management system, is expected to take about 2 years to 
complete. The second subphase, which is planned to be done concurrently 
with the first phase, will replace IFCAP with the IFAS commercial off- 
the-shelf financial management system and is expected to take just over 
4 years to complete. 

VA's approach to implementing the data warehouse calls for developing 
the warehouse after the underlying data structures of SAM and IFAS are 
defined and stabilized. The department expects to complete the data 
warehouse in the first quarter of fiscal year 2014. 

Figure 4 depicts the program timeline, from program proposal through 
deployment of the SAM, IFAS, and data warehouse components. 

Figure 4: FLITE Timeline: 

[Refer to PDF for image: timeline] 

Program initiation activities: 
FLITE proposed: FY 2005, Q4; 
PDO established: FY 2007, Q2; 
IT PMO established: FY 2007, Q3. 

SAM: 
Pilot implementation: FY 2009, Q3 to FY 2010, Q4; 
Acquisition planning and source selection: FY 2009, Q3 to FY 2010, Q4; 
Beta: FY 2010, Q3 to FY 2012, Q1; 
National deployment: FY 2012, Q1 to FY 2013, Q3. 

IFAS: FMS replacement: 
Acquisition planning and source selection: FY 2009, Q2 to FY 2009, Q4; 
Pilot: FY 2010, Q1 to FY 2010, Q4; 
National deployment: FY 2011, Q1 to FY 2012, Q4. 

IFAS: IFCAP replacement: 
Development: FY 2010, Q1 to FY 2011, Q1; 
Pilot: FY 2011, Q1 to FY 2011, Q2; 
Beta: FY 2011, Q2 to FY 2011, Q4; 
National deployment: FY 2011, Q4 to FY 2014, Q1. 

Data warehouse: 
Acquisition planning and source selection: FY 2009, Q4 to FY 2011, Q1;
Development: FY 2011, Q1 to FY 2012, Q1; 
Pilot/Beta: FY 2012, Q1 to FY 2012, Q3; 
Deployment: FY 2012, Q3 to FY 2014, Q1. 

Source: GAO analysis of VA data. 

[End of figure] 

In 2009, the program office undertook various activities, including 
issuing the IFAS request for proposals (February), awarding a program 
management support contract (March), awarding the SAM pilot project 
contract and beginning work (April), issuing a request for proposals 
for independent verification and validation support (July), and 
initiating planning for the data warehouse (September). 

According to program officials, as of September 2, 2009, the department 
had spent approximately $90.8 million on FLITE. This amount included 
$73.0 million for about 40 contract actions on behalf of the program 
office: $28.5 million for program management and technical support, 
$27.8 million for software licenses, $10.9 million for the SAM project, 
$5.5 million for analyses (e.g., requirements analyses), and $0.3 
million for other program activities (e.g., training). 

Prior GAO and OIG Reviews of FLITE: 

Both we and VA's OIG have previously reported on the FLITE initiative. 
In a September 2008 report, we noted that key planning documents 
related to the initiative lacked specificity and detail, and that VA 
had not addressed all the findings in the CoreFLS findings repository. 
[Footnote 14] We recommended that VA add more specificity and details 
to key planning documents, such as the concept of operations and work 
breakdown structure, and address all findings in the CoreFLS findings 
repository to minimize risk to the successful implementation of FLITE. 
In response to our report, as of September 2009, VA had updated key 
planning documents and reported that it had taken actions that 
addressed all of the findings identified in the repository. 

In September 2009, VA's OIG reported on VA's effectiveness in managing 
the FLITE program.[Footnote 15] The office noted, among other things, 
that although program managers had taken steps toward addressing the 
CoreFLS findings, deficiencies similar to those found in CoreFLS were 
also evident in FLITE. For example, OIG reported that FLITE program 
functions were not fully staffed. 

Pilot Project Is Behind Schedule: 

VA and its contractor have begun one of the two planned pilot systems--
the SAM component. Specifically, in April 2009, the department 
contracted with General Dynamics Information Technology Inc. to 
implement Maximo at the VA Medical Center in Milwaukee, Wisconsin. 
Among the activities the contractor is expected to perform are 
analyzing business processes, documenting requirements, configuring 
Maximo, and performing system tests. 

As of mid-September 2009, VA reported that, with the contractor only 5 
months into the 1-year time period planned to complete the pilot, the 
project had fallen 2 months behind schedule. This 2-month schedule slip 
was a consequence of the contractor falling behind in its efforts to 
perform tasks and deliver products that are necessary to implement the 
pilot system. Specifically, of the 34 tasks planned to be undertaken by 
mid-September, the contractor reported that 11 had not yet been 
started--including conducting a security assessment and predeployment 
testing--and that of 23 tasks that had been initiated, 16 were behind 
schedule. For example, among the tasks that the contractor noted as 
behind schedule were analysis of security requirements, business 
process analysis, and system configuration. Regarding the seven 
remaining tasks, two had reportedly been completed and five were 
identified as being on schedule. The contractor reported that it had 
completed a requirements traceability matrix and was on schedule with 
respect to starting up a project management office, performing 
organizational change management activities, and developing quality 
assurance and control programs. 

Further, with respect to the delivery of products, the contractor 
reported that it had delivered only 7 of 37 products due by mid- 
September. The SAM project management plan and the requirements 
management plan were among the products that were delivered. Products 
that had not yet been delivered included the Maximo system 
configuration document, intended to provide detailed instructions to 
enable a trained Maximo administrator to incorporate all VA 
configuration requirements, and the SAM system security plan. 

VA attributed the project being 2 months behind schedule to a shortage 
of FLITE program office human capital resources and poor project 
management by the contractor. Specifically, according to the program 
director, the program did not have the personnel it needed during the 
initial months of the SAM pilot project to provide the contractor with 
the information it needed to make planned progress. Regarding the 
contractor's project management, VA stated that the contractor: 

* provided a project manager who did not possess the skills necessary 
to deliver quality and timely products, 

* delayed hiring a project scheduler and used an initial project 
scheduling approach that was incorrect, 

* used an ineffective and inefficient approach to analyzing VA's 
business processes and underestimated the time needed to obtain a 
thorough understanding of the processes, and: 

* underestimated the effort necessary to configure a database server 
used in the pilot's development environment. 

In mid-September, the FLITE program director stated that the department 
had filled almost all of the program office vacancies and that the 
contractor had begun to improve its project management weaknesses. 
Nevertheless, according to the program director, while the department 
does not expect any further delays in completing the SAM pilot, it does 
not expect to recover the 2-month schedule slippage that has already 
occurred. As a result, the department projected completion of the pilot 
in 14 months, instead of 12 months as originally planned. 

Additionally, activities are under way to initiate the IFAS pilot. 
Specifically, the department issued a request for proposals for a pilot 
contractor in February 2009. A contract for the IFAS pilot is planned 
for award in late October 2009. 

VA Has Recently Filled FLITE Program Staff Positions but Has Not Yet 
Fully Established Other Program Management Capabilities: 

VA has taken steps to institute effective management of the FLITE 
program; however, the department has not yet fully established key 
capabilities needed to ensure that system components will be 
implemented as planned. The department recently made progress toward 
filling program office staff vacancies. Nonetheless, more work is 
needed to fully establish program management capabilities in areas that 
are important to the development of its integrated financial and 
logistics system. Until VA completes efforts to develop and reconcile 
its cost estimate; comply with EVM system standards; implement 
performance measures for its schedule; include all relevant federal and 
system requirements; and perform effective, independent verification 
and validation, it will have increased risk that FLITE will experience 
cost overruns and schedule delays and will not provide the capabilities 
that users need. 

VA Recently Filled Vacant Program Office Positions: 

Our past work has found that the success of federal programs depends on 
having effective strategic human capital management and, in particular, 
having the right number of people with the right mix of knowledge and 
skills.[Footnote 16] VA has recently taken steps to fill long-standing 
vacancies in the FLITE program that have adversely impacted the 
program's ability to maintain schedules. Specifically, in mid-
September, the program acquired 36 staff, filling 111 of 112 required 
positions. According to the Acting Assistant Secretary for the Office 
of Management, vacant FLITE program positions were filled by 
individuals who were reassigned, detailed, or newly hired when the VA 
Deputy Secretary became aware of the program's need for staff 
resources. As a result of the department's recent actions to fill 
vacant positions, the office should be better positioned to effectively 
manage the program. 

VA's FLITE Cost Estimate Is Incomplete: 

Federal guidelines[Footnote 17] recommend that operations and 
maintenance costs over the entire estimated life cycle of an investment 
be included in a cost estimate. Inclusion of these costs over the time 
period corresponding to the life of the investment is encouraged by the 
federal government's guidance for managing capital assets because such 
costs are a key element for establishing the total cost of ownership. 
Further, our Cost Estimating and Assessment Guide[Footnote 18] 
describes effective cost-estimating practices, including performance of 
a risk and uncertainty analysis and development of an independent cost 
estimate that provides an unbiased test of whether the program's 
estimate is reasonable. Typically, the two estimates are reconciled. 

In August 2008, the FLITE program office developed a program cost 
estimate of $608.7 million for fiscal years 2007 through 2014--when 
FLITE systems are expected to achieve full operational capability. 
However, the office did not project operations and maintenance costs 
over the entire estimated life of the FLITE investment, and it did not 
perform a risk and uncertainty analysis as encouraged by best 
practices. Program officials stated that they did not consider life- 
cycle operations and maintenance costs in their estimate because they 
wanted to capture only the cost for developing the FLITE system up to 
its full operational capability. Also, rather than perform a risk and 
uncertainty analysis of their own, the program office planned to rely 
on risk analyses by an outside entity, the Department of the Navy Space 
and Naval Warfare Systems Command (SPAWAR), that the department engaged 
to generate a risk adjusted independent cost estimate. 

Completed in April 2009, the SPAWAR estimate identified costs totaling 
$1.899 billion for the life of the program and included $1.061 billion 
of estimated operations and maintenance costs for fiscal year 2015 
through fiscal year 2024, which represented the entire estimated life 
of the initiative. According to SPAWAR officials, they used our Cost 
Estimating and Assessment Guide as the method for developing the 
independent estimate.[Footnote 19] Also, to align with VA's estimate, 
SPAWAR used standardized cost elements and definitions to develop a 
probability-based estimate of $837.8 million for fiscal years 2007 
through 2014. This estimate was $229.1 million higher than the 
department's estimate for this period. The department's estimate was 
not derived based on standardized cost elements and probability-driven 
risk and uncertainty costs assessments. 

VA has not yet reconciled its cost estimate with SPAWAR's estimate. 
According to department officials, a significant number of end-of- 
fiscal-year procurement requests and the department's prioritization of 
IT acquisitions had affected the timing of plans to reconcile the 
estimates. Program officials stated that they intend to incorporate 
federal polices and requirements, as well as address funding, 
budgetary, or contractual issues necessitated by the reconciliation. 
According to the officials, the department plans to initiate this work 
in December 2009 and to complete it in March 2010. Until the 
reconciliation is completed, effective administration of FLITE program 
planning, budgeting, acquisition, and performance management activities 
could be jeopardized if accurate cost data are not available to guide 
the execution of these functions. Completion of the reconciliation, 
which should include estimated operations and maintenance costs for the 
life of the program, is essential to increase the reliability of the 
FLITE cost estimate and reduce the risk that acquisition plans, 
budgets, and performance management activities will be unsuccessful or 
inefficient. 

VA Has Not Performed Key Actions Necessary for Reliable EVM for FLITE: 

OMB and department policies require major programs to use EVM to 
measure and report program progress.[Footnote 20] EVM is a tool for 
measuring program progress by comparing the value of work accomplished 
with the amount of work expected to be accomplished. Such a comparison 
permits actual performance to be evaluated, based on variances from the 
cost and schedule baselines--collectively referred to as a performance 
measurement baseline. Identification of significant variances and 
analysis of their causes helps program managers determine the need for 
corrective actions. Before EVM analysis can be reliably performed, 
developing a credible cost estimate is necessary to provide program 
managers with a clear definition of the cost, schedule, and risks 
associated with the scope of work planned. These inputs are then used 
to create a performance measurement baseline for EVM analysis. In 
addition, federal policy requires that systems used to collect and 
process EVM data be compliant with the industry standard developed by 
the American National Standards Institute (ANSI) and Electronic 
Industries Alliance (EIA), ANSI/EIA Standard 748.[Footnote 21] 

Program officials have recognized the importance of reliable EVM and 
finalized the FLITE Program Measurement Earned Value Management Plan in 
August 2009. The plan identified roles and responsibilities, applicable 
policy and guidance, and the program's EVM implementation approach. 
According to program officials, programwide earned value reporting that 
will include government, program management support, and SAM project 
work activities is expected to begin in October 2009. 

However, while VA plans to begin reporting earned value performance in 
October 2009, a reliable cost estimate, which is necessary for EVM 
reporting, is not expected to be completed by that time. Specifically, 
as noted earlier, the department has not reconciled its cost estimate 
for the program with SPAWAR's independent cost estimate. Program 
officials do not expect reconciliation of the cost estimate to begin 
until 2 months after earned value reporting is scheduled to begin. 

Additionally, VA officials have not yet ensured that all EVM systems 
for FLITE are certified for compliance with ANSI/EIA Standard 748. 
[Footnote 22] These compliance assessments are necessary to demonstrate 
the capability of providing reliable cost and schedule information for 
earned value reporting. Specifically, the compliance assessment for the 
SAM pilot contractor's system has not yet been completed. While program 
officials did not provide information that explained why a compliance 
assessment of the contractor's EVM system had not yet been completed, 
they stated that the contractor has a plan to obtain system 
certification. This activity is not expected to be complete until 
January 2010, 3 months after earned value reporting for the program is 
scheduled to begin. 

Until the agency has completed reconciling its cost estimate and 
ensured that contractors comply with EVM system industry standards, VA 
will have an increased risk of reporting and managing the program based 
on unreliable performance data. 

VA Has Not Established a Reliable Program Schedule for FLITE: 

GAO's Cost Estimating and Assessment Guide[Footnote 23] states that the 
success of a program depends in part on having a reliable schedule that 
realistically depicts the program's work activities to a specific 
degree of detail, reasonably indicates when those work activities will 
occur, estimates how long they will take to complete, and shows how the 
work activities are related to each other. For example, a reliable 
schedule would indicate when one work activity depends upon the 
completion of another before it can start and that required resources 
(e.g., labor and materials) are assigned to all activities. Overall, 
the schedule provides the road map for the orderly execution of a 
program, helps identify and address potential problems, provides a 
baseline to gauge progress, and promotes accountability. 

VA has not yet established a schedule for the program that is reliable. 
Program officials stated that they baselined (i.e., formally 
established) an integrated master schedule in January 2009. However, in 
the program's August and September 2009 Risk & Issues reports, program 
officials noted that the integrated master schedule was not complete 
and did not represent all program requirements. The reports also 
identified that the SAM pilot schedule (a key component of the overall 
program schedule) did not include sufficient detail to trace project 
tasks to contract requirements. 

Our analysis also concluded that the schedule was unreliable and noted 
that, in addition to issues VA identified with the program schedule, 
the integrated master schedule did not include key program management 
activities for reconciling the program cost estimate and implementing 
EVM, nor did it identify resources assigned to activities already under 
way or expected to start in the near future. Further, the schedule did 
not identify all dependencies and activities and did not break down all 
dependencies and activities to a sufficient level of detail to measure 
performance. Program officials acknowledged these deficiencies and 
stated that program management staffing shortages and delays in 
receiving a reliable project schedule from the SAM contractor have 
affected their ability to produce a reliable schedule for the program. 
They stated that in July 2009, they began working with stakeholders to 
address schedule issues and plan to improve the reliability of their 
schedule by finalizing a revised integrated master schedule by October 
2009. 

Until VA completes a revised integrated master schedule that includes 
all key program activities broken down to a sufficient level of detail 
and identifies all resources and dependencies, the program's efforts to 
measure progress and identify potential problems will be impaired, and 
the program will have increased risk of missing critical milestones for 
system delivery. 

VA Has Not Identified and Effectively Managed All FLITE Requirements: 

According to SEI guidance, the requirements for a system should 
describe the functionality needed to meet user needs and perform as 
intended in the operational environment.[Footnote 24] Federal agencies 
also must ensure that their financial management systems comply with 
federal standards mandated by the Federal Financial Management 
Improvement Act of 1996.[Footnote 25] Also according to SEI guidance, 
an organization can ensure system requirements are based on business 
requirements by tracking the requirements from inception of the project 
and agreement on a specific set of business requirements to development 
of the system requirements, detailed design, code implementation, and 
test cases necessary for validating the requirements. Requirements must 
be traceable forward and backward (i.e., bidirectional traceability) 
through the development life cycle. Traceability helps reduce the risks 
of fielding a system that does not meet the needs of its users, 
incurring schedule delays, and increasing costs. 

VA has developed an initial set of 1,700 requirements that need to be 
addressed in the development of the SAM and IFAS components. FLITE 
requirements consist of core financial and procurement requirements 
related to the IFAS project, as well as inventory, supply, and real 
property requirements related to the SAM project. To develop the 
initial set of requirements for FLITE, program officials stated that 
they analyzed VA's current and planned financial and asset management 
business processes and researched the Financial Systems Integration 
Office's (FSIO)[Footnote 26] Core Financial System Requirements and 
Inventory, Supplies, and Materials System Requirements publications. 
[Footnote 27] The initial set of requirements was further defined and 
refined by obtaining input from consultants and VA financial and asset 
management experts. The department included all mandatory core 
financial system requirements in its IFAS requirements but did not 
include all mandatory inventory, supplies, and materials requirements 
in its SAM requirements. For example, our analysis showed that VA did 
not include requirements for recording whether goods and services are 
accepted or rejected and for performing a systematic review and follow-
up of overdue in-transit items.[Footnote 28] Program officials 
explained that they did not include these requirements because they had 
not determined whether the requirements were applicable to the SAM 
project. The officials agreed to incorporate the missing requirements. 
VA is also in the process of finalizing its real property requirements 
for the SAM beta phase and still plans to develop additional 
requirements related to procurement for IFAS. Further, the department 
is identifying data analysis and reporting requirements for the data 
warehouse.[Footnote 29] 

Regarding requirements traceability, SAM project officials acknowledge 
that mapping system requirements to the related business requirements 
is fundamental to effective requirements management. However, according 
to FLITE officials, they made a business decision not to establish 
bidirectional traceability between the business and system requirements 
included in the SAM pilot request for proposals. Instead, they decided 
to require the pilot contractor to establish traceability between the 
business and system requirements after the contractor analyzes and 
refines the requirements. According to the officials, the contractor 
plans to complete these tasks by December 2009. In addition, program 
officials stated that they plan to establish bidirectional traceability 
between IFAS business and system requirements under the IFAS 
implementation contract scheduled to be awarded in October 2009. In 
this regard, the IFAS request for proposals states that the 
implementation contractor will be required to finalize IFAS 
requirements, as well as maintain and document the traceability of all 
requirements to design, develop, integrate, and test specifications. 

As the department develops its requirements, it is important that all 
relevant and applicable federal financial management system 
requirements be identified and incorporated into the program's 
requirements to ensure its planned financial management systems meet 
users' needs and comply with applicable federal laws. Further, until 
they have established traceability between the business and system 
requirements, VA will not be positioned to know whether the system 
requirements are complete and effectively address each business 
requirement. 

VA Has Begun Independent Verification and Validation of the FLITE 
Program, but All Findings Have Not Been Addressed in a Timely Manner: 

According to recognized industry standards[Footnote 30] and our prior 
reports,[Footnote 31] the purpose of independent verification and 
validation is to provide an independent review of system processes and 
products to ensure that quality standards are being met. As we have 
previously noted, the use of independent verification and validation is 
a recognized best practice for large and complex system development and 
acquisition programs such as FLITE and involves an independent 
organization conducting unbiased reviews of processes, products, and 
results to verify and validate that they meet stated requirements and 
standards. VA policy recognizes the importance of addressing 
independent verification and validation results in a timely manner. 

Recognizing the importance of independent verification and validation, 
the department's Systems Quality Assurance Service was tasked with 
performing independent verification and validation activities for the 
FLITE program.[Footnote 32] In April 2009, this organization developed 
a Software Quality Assurance Plan[Footnote 33] to guide independent 
verification and validation activities for the program. The plan was 
developed consistent with industry standards and generally contained 
the required elements. The plan also outlined reviews that would be 
performed by the Systems Quality Assurance Service, including product 
(e.g., program and project deliverables), process, internal controls, 
test readiness, and production readiness reviews. In addition, the 
Systems Quality Assurance Service is responsible for advising and 
assisting with the program's implementation of a suite of tools to 
support requirements management, change management, risk, and test 
management. 

Independent verification and validation of the FLITE program has been 
focused primarily on the review of program and project deliverables. 
According to program officials, as of September 2009, the Systems 
Quality Assurance Service had reviewed 30 FLITE work products and 
provided findings and recommendations to document owners.[Footnote 34] 
Out of 1,064 total findings, 947 (approximately 89 percent) had been 
fully addressed by the program or had been identified as obsolete by 
the Systems Quality Assurance Service. Of the 117 remaining findings, 
59 had been addressed but had not yet been reflected in revised 
documents, and 58 required additional attention. Of the 58 findings and 
recommendations that remained open, the SAM pilot site readiness plan 
accounted for 18 that were identified in December 2008. According to 
the Systems Quality Assurance Service, these findings focused on the 
need for consistency with other project documentation, clarity in the 
timing of site activities, and incorporation of planned site-level 
activities into the program work breakdown structure. In addition, 
according to department officials, the FLITE acquisition strategy has 
two findings and recommendations that were identified in December 2008 
and that remain to be addressed. These findings are related to VA's 
approach for acquiring SAM and IFAS integration support and the 
program's focus on front-end acquisition activities, rather than full 
life cycle acquisition processes. Unknown or incomplete system 
integration requirements may result in significant rework and adversely 
impact the program's cost, schedule, and quality. 

According to FLITE program officials, they have not had the human 
capital resources they need to address all the independent verification 
and validation findings and recommendations in a timely manner. As a 
result, independent verification and validation findings that highlight 
important program issues (e.g., determining an approach for integrating 
SAM and IFAS) have not received the attention that they need. As 
discussed earlier, the staff resources recently added could help 
address the program's inability to focus sufficient attention on 
resolving findings from initial independent verification and validation 
activities. It remains unclear whether the program office will be 
positioned to efficiently resolve findings raised when the scope of 
independent verification and validation activities expands to include 
system testing and production readiness reviews, which affect the 
extent to which FLITE components will meet stated requirements and 
quality standards. 

Conclusions: 

The pilot for VA's new asset management system has experienced a 2- 
month schedule delay just 5 months after award of the contract. While 
VA has recently taken steps to address the staffing shortages that have 
substantially contributed to this delay, it has not yet fully 
established the management capability necessary for FLITE to be 
successful. For example, the department's program cost estimate did not 
represent total program costs, nor has the estimate been reconciled 
with an independent estimate--a process that could increase its 
reliability. Further, it has not conducted EVM that is needed to ensure 
the reliability of the department's programwide reporting on the 
initiative. Also, VA has not yet made revisions that are needed to 
increase the reliability of the program's integrated master schedule. 
In addition, the requirements for the two major program systems, SAM 
and IFAS, do not yet address all the functions expected of federal 
asset management and financial management systems. Finally, key 
findings from independent reviews of the program have not been fully 
addressed on a timely basis. 

As a consequence, the department is faced with significant challenges 
in implementing FLITE's pilot systems as planned, while simultaneously 
working to fully establish program management capabilities. Program 
officials recognize the importance of reconciling their cost estimate, 
ensuring compliance with EVM system standards, establishing a reliable 
schedule, ensuring all relevant federal and system requirements are 
identified and traceable, and addressing all independent verification 
and validation findings. Further, they have stated that they plan to 
take such actions. However, just as program officials needed the 
department's support in filling long-standing program office vacancies, 
the full support of the department's top management is critical to 
ensuring that planned actions are executed. If the program is not 
effective in addressing its management weaknesses, the department 
increases the risk of repeating its unsuccessful earlier attempt to 
modernize the department's financial and logistics systems. 

Recommendations for Executive Action: 

To help guide and ensure successful completion of FLITE, the Secretary 
of Veterans Affairs should direct and ensure that the Assistant 
Secretary for Management and the Assistant Secretary for Information 
and Technology take the following five actions: 

* Improve the reliability of the program cost estimate by ensuring that 
the estimate includes system operations and maintenance costs and that 
the estimate is reconciled with the independent cost estimate. 

* Improve the reliability of program earned value management reporting 
by ensuring that contractor earned value management systems comply with 
industry standards. 

* Complete a revised integrated master schedule that includes all key 
program activities, including reconciliation of the program cost 
estimate and implementation of earned value management, and identifies 
all resources and dependencies. 

* Ensure that all relevant and applicable federal financial management 
system requirements are included in FLITE's requirements and establish 
and maintain requirements traceability. 

* Ensure that all comments from independent verification and validation 
reviews are addressed. 

Agency Comments and Our Evaluation: 

The VA Chief of Staff provided written comments on a draft of this 
report. In its comments, the department concurred with our 
recommendations and described actions to address them. For example, the 
department stated that it plans to reconcile the FLITE program cost 
estimate with the independent cost estimate by the second quarter of 
fiscal year 2010; ensure that future contractors' EVM systems comply 
with industry standards and begin an independent review of the 
program's EVM compliance by the first quarter of 2010; and include the 
reconciled program cost estimate in the integrated master schedule by 
the third quarter of fiscal year 2010. Further, the department stated 
that it plans to validate the completeness of FLITE requirements by mid-
November 2009 and ensure that outstanding comments from independent 
verification and validation reviews are addressed by mid-December 2009. 
If the recommendations are properly implemented, they should better 
position VA to effectively manage the FLITE program. 

The department also provided a technical comment, which we have 
addressed in the report as appropriate. The department's written 
comments are reproduced in appendix II. 

As agreed with your offices, unless you publicly announce its contents 
earlier, we plan no further distribution of this report until 30 days 
from the date of this letter. At that time, we will send copies of the 
report to interested congressional committees, the Secretary of 
Veterans Affairs, and other interested parties. In addition, the report 
will be available at no charge on our Web site at [hyperlink, 
http://www.gao.gov]. 

If you or your staffs have questions about this report, please contact 
Valerie C. Melvin at (202) 512-6304 or melvinv@gao.gov, or Kay L. Daly 
at (202) 512-9095 or dalykl@gao.gov. Contact points for our Offices of 
Congressional Relations and Public Affairs may be found on the last 
page of this report. Key contributors to this report are listed in 
appendix III. 

Signed by: 

Valerie C. Melvin: 
Director, Information Management and Human Capital Issues: 

Signed by: 

Kay L. Daly: 
Director, Financial Management and Assurance: 

[End of section] 

Appendix I: Objectives, Scope, and Methodology: 

As requested, the objectives of our study were to (1) determine the 
status of the Financial and Logistics Integrated Technology 
Enterprise’s (FLITE) pilot system development and (2) evaluate key 
program management processes, including the Department of Veterans 
Affairs’ (VA) efforts to institute effective human capital management, 
develop a reliable program cost estimate, use earned value management, 
establish a realistic program schedule, employ effective requirements 
development and management, and perform independent verification and 
validation. 

To determine the status of the pilot system development, we: 

* obtained and analyzed program documentation, including program 
management plans, contracts, schedules, briefing slides, meeting 
minutes, and project status reports to identify from these reports the 
planned FLITE pilot activities and deliverables and determined to what 
extent these tasks had been completed; and; 

* supplemented department program documentation and our analyses by 
interviewing department and contractor officials, such as the program 
director, and observing project status meetings.
We also evaluated VA’s progress toward implementing our prior 
recommendations related to adding specificity and details to key 
planning documents by comparing updated documents, including the 
Program Management Plan and Strategic Asset Management (SAM) Concept of 
Operations to prior versions. 

To evaluate key program management processes, we: 

* compared program staffing plans with the program’s staffing resource 
reports to determine the extent to which program human capital needs 
have been met; 

* compared the program cost estimate and estimating activities to 
Office of Management and Budget guidance and GAO’s Cost Estimating and 
Assessment Guide[Footnote 35] to determine the estimate’s completeness 
and the effectiveness of the estimating activities; 

* reviewed department documentation, such as the program’s plan for 
earned value management implementation, and compared them to federal 
policy and GAO’s Cost Estimating and Assessment Guide to determine the 
department’s preparedness for conducting reliable earned value 
management; 

* reviewed the program schedule and compared it to planned activities, 
deliverables, and practices described in GAO’s Cost Estimating and 
Assessment Guide to assess the schedule’s reliability; 

* analyzed program documentation, including the department’s business 
requirements, concept of operations for FLITE, traceability matrix, and 
requirements management plan, to determine the extent to which they 
reflect practices such as those recognized by SEI[Footnote 36] and 
include federal financial management system requirements; and; 

* reviewed program documentation, such as the software quality 
assurance plan, quality management plan, and technical review reports, 
to determine the extent to which the program has addressed independent 
verification and validation findings. 

We conducted this performance audit at VA headquarters in Washington, 
D.C., from November 2008 through October 2009 in accordance with 
generally accepted government auditing standards. Those standards 
require that we plan and perform the audit to obtain sufficient, 
appropriate evidence to provide a reasonable basis for our findings and 
conclusions based on our audit objectives. We believe that the evidence 
obtained provides a reasonable basis for our findings and conclusions 
based on our audit objectives. 

[End of section] 

Appendix II: Comments from the Department of Veterans Affairs: 

The Secretary Of Veterans Affairs: 
Washington: 

October 19, 2009: 

Ms. Valerie Melvin: 
Director: 
Information Management and Human Capital Issues: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Ms. Melvin: 

The Department of Veterans Affairs (VA) has reviewed the Government
Accountability Offices (GAO) draft report, Information Technology; 
Actions Needed to Fully Establish Program Management Capability for 
VA's Financial and Logistics Initiative (GAO-10-40) and generally 
agrees with GAO's conclusions and concurs with GAO's recommendations to 
the Department. 

The enclosure specifically addresses GAO's recommendations and provides 
additional comments to the draft report. VA appreciates the opportunity 
to comment on your draft report. A similar response has been sent to 
Ms. Kay Daly. 

Sincerely, 

Signed by: 
John R. Gingrich: 
Chief of Staff: 
Enclosure: 

[End of letter] 

The Secretary Of Veterans Affairs: 
Washington: 

October 19, 2009: 

Ms. Kay Daly: 
Director: 
Financial Management and Assurance: 
U.S. Government Accountability Office: 
441 G Street, NW: 
Washington, DC 20548: 

Dear Ms. Daly: 

The Department of Veterans Affairs (VA) has reviewed the Government 
Accountability Office's (GAO) draft report, Information Technology: 
Actions Needed to Fully Establish Program Management Capability for 
VA's Financial and Logistics Initiative (GAO-10-40) and generally 
agrees with GAO's conclusions and concurs with GAO's recommendations to 
the Department. 

The enclosure specifically addresses GAO's recommendations and provides 
additional comments to the draft report. VA appreciates the opportunity 
to comment on your draft report. A similar response has been sent to 
Ms. Valerie Melvin. 

Sincerely, 

Signed by: 

John R. Gingrich: 
Chief of Staff: 

Enclosure: 

[End of letter] 

Enclosure: 

Departments of Veterans Affairs (VA) Comments to Government 
Accountability Office (GAO) Draft Report, Information Technology: 
Actions Needed to Fully Establish Program Management Capability for 
VA's Financial and Logistics Initiative (GAO-10-40): 

GAO recommendation: 

To help guide and ensure successful completion of FLITE, the Secretary 
of Veterans Affairs should direct and ensure that the Assistant 
Secretary for Management and the Assistant Secretary for Information 
and Technology take the following five actions: 

Recommendation 1: Improve the reliability of the program cost estimate 
by ensuring that the estimate includes system operations and 
maintenance costs and that the estimate is reconciled with the 
independent cost estimate. 

VA response: Concur. Financial and Logistics Integrated Technology 
Enterprise (FLITE) completed a program manager's cost estimate that 
formed the basis of the e300 submission in December 2008 and 
subsequently issued a short-term contract for an independent cost 
estimate. The Department of Veterans Affairs (VA) fully intends to 
complete the cost estimation work for the ELITE program by reconciling 
it with the independent cost estimate provided by SPAWAR in July 2009. 
VA is contracting for a follow-on independent cost analysis which will 
include reconciliation of the Program Managers and independent cost 
estimates to validate assumptions. The Government expects to complete 
the cost estimate reconciliation by Q2 FY 2010. 

Recommendation 2: Improve the reliability of program earned value 
management reporting by ensuring that contractor earned value 
management systems comply with industry standards. 

VA response: Concur. VA is committed to continually improving its 
program and project management activities, to include the reliability 
of earned value management (EVM) reporting. To increase this 
reliability, VA will: 

* Conduct thorough validation activities to ensure all future FLITE 
contractors' EVM systems comply with the applicable American National 
Standards Institute (ANSI) standards, specifically ANSI/EIA Standard 
748 and VA Earned Value Application guide and Directive 6061. 

* On a consistent basis, conduct integrated baseline reviews on all 
ELITE contracts with an EVM requirement. 

* Obtain an independent compliance review of the FLITE program's
implementation of EVM. This activity will begin in Q1 FY 2010. 

* Ensure all Requests for Proposals issued by the FLITE program include 
mandatory EVM compliance requirements. 

* Ensure that when awarding contracts, there is formal proof of a 
contractors compliance with ANSI standards for use of an EVM system. 

* Ensure agreement with any applicable contractor, before and after 
award as appropriate, in order to verify the Performance Measurement 
Baseline (PMB). The PMB is the basis against which all performance will 
be measured and EVM data generated. 

* Maintain a change management process to ensure integrity of the cost 
and schedule baseline, This includes submission of formal change 
requests, based upon pre-determined thresholds and parameters, which 
will be reviewed by the overall FLITE Change Control Board. 

Recommendation 3: Complete a revised integrated master schedule that 
includes all key program activities, including reconciliation of the 
program cost estimate and implementation of earned value management and 
identifies all resources and dependencies. 

VA response: Concur. VA has included tasks for creating and maintaining 
integrated master schedules (IMS) in all its applicable contracts. The 
FLITE program has an existing overall IMS, which is supported by the 
project levels schedules. This IMS includes all key program activities 
and is fully developed. 

The FLITE program IMS will include the reconciled program cost estimate 
by Q3 FY 2010. 

Recommendation 4: Ensure that all relevant and applicable federal 
financial management system requirements are included in FLITE's 
requirements and establish and maintain requirements traceability. 

VA response: Concur. The Strategic Asset Management (SAM) system 
Project Office believes that the applicable federal financial 
management system requirements were included when the original SAM 
business requirements were developed. However, to ensure and validate 
this assumption, the SAM Project Office will cross reference these 
requirements with SAM requirements. 

Any requirement that may have been omitted will be added to the SAM 
Business Requirements document following established change control 
procedures. Any federal financial management system requirement 
determined not to be applicable to VA will be documented as to the 
reason why. The validation process will be accomplished by November 13, 
2009. 

Recommendation 5: Ensure that all comments from independent 
verification and validation reviews are addressed. 

VA response: Concur. The 59 findings that were addressed are being 
included into the appropriate documents during their planned updates. 
The 58 findings requiring additional attention are being reviewed for 
document revision update. The planned updates and the review process 
will be completed by December 15, 2009. 

In addition, VA's Systems Quality Assurance Service will provide a 
biweekly status report for use by the FLITE program and project 
managers. The report will outline program and project quality 
assurance, testing activities and status. In addition all program and 
project level product reviews will be included in a current period 
activity table and a cumulative status tracking format which will allow 
for easy progress assessment for follow-up action. 

[End of section] 

Appendix III: GAO Contacts and Staff Acknowledgments: 

GAO Contacts: 

Valerie C. Melvin, (202) 512-6304 or melvinv@gao.gov: 

Kay L. Daly, (202) 512-9095 or dalykl@gao.gov: 

Staff Acknowledgments: 

In addition to the contacts named above, key contributions to this 
report were made by Mark T. Bird, Assistant Director; Michael S. 
LaForge, Assistant Director; Heather A. Collins; Neil J. Doherty; 
Rebecca Eyler; David A. Hong; Jacqueline K. Mai; Yvonne D. Moss; Robert 
L. Williams, Jr.; and Leonard E. Zapata. 

[End of section] 

Footnotes: 

[1] Investments that are designated as high risk require special 
attention from the highest level of agency management and oversight 
authorities due to size, complexity, or nature of the risk of the 
project. 

[2] VA currently does not have standardized systems or processes for 
managing physical assets, supply inventories, and related work order 
management across the department. While VA uses some common inventory, 
procurement, and asset management applications, each VA facility 
operates a customized version of each system. 

[3] The material weakness in financial management system functionality 
is linked to VA’s outdated legacy financial systems, impacting VA’s 
ability to prepare, process, and analyze financial information that is 
reliable, timely, and consistent. Legacy system deficiencies 
necessitated significant manual workarounds and a large number of 
general ledger adjustments, increasing the risk of processing errors 
and misstatements in the financial statements. 

[4] VA OIG, Issues at VA Medical Center Bay Pines, Florida and 
Procurement and Deployment of the Core Financial and Logistics System 
(CoreFLS), 04-01371-177 (Washington, D.C., Aug. 11, 2004). 

[5] Carnegie Mellon SEI, Report of the Independent Technical Assessment 
of the Department of Veterans Affairs CoreFLS Program (June 2004). 

[6] VA OIG, Issues at VA Medical Center Bay Pines, Florida and 
Procurement and Deployment of the Core Financial and Logistics System 
(CoreFLS), 04-01371-177. 

[7] VA MQAS, VA Lessons Learned: Findings and Recommendations Summary 
from CoreFLS Reviews, 07-04-SAD002 (Austin, Tex., Aug. 1, 2007). 

[8] The aggregated list of 141 lessons learned included 80 lessons 
learned from SEI, 22 from OIG, and 39 from MQAS. VA officials 
subsequently reduced the total number of lessons learned to 103 by 
eliminating duplicate findings. 

[9] The use of lessons learned is a principal component of an 
organizational culture committed to continuous improvement. Sharing 
such information serves to communicate acquired knowledge more 
effectively and to ensure that beneficial information is factored into 
planning, work processes, and activities. Lessons learned can be based 
on positive experiences or on negative experiences that result in 
undesirable outcomes. 

[10] VA developed their initial cost estimate for FLITE in August 2008. 
In April 2009, an independent cost estimate adjusted for program risks 
and uncertainties estimated FLITE costs to be $837.8 million. 

[11] Maximo is a Web-based asset management suite used to manage the 
complete life cycle of strategic assets, including planning, 
procurement, deployment, tracking, maintenance, and retirement. 

[12] IFCAP is a decentralized procurement, funds control, and front-end 
accounting system that complements the FMS functionality. It is used at 
the VA Medical Centers and certain Regional and Administrative Offices. 
IFCAP integrates functions of Fiscal Service, Acquisition and 
Logistics, and other VA Medical Center services that request supplies 
and services for VA. There are more than 150 stand-alone instances of 
IFCAP across VA. 

[13] SAM and IFAS are intended to be transactional systems but will not 
be optimized for query and reporting activities, which is the role that 
the data warehouse is intended to fill. 

[14] GAO, Veterans Affairs: Additional Details Are Needed in Key 
Planning Documents to Guide the New Financial and Logistics Initiative, 
[hyperlink, http://www.gao.gov/products/GAO-08-1097] (Washington, D.C.: 
Sept. 22, 2008). 

[15] VA OIG, Department of Veterans Affairs: Audit of FLITE Program 
Management’s Implementation of Lessons Learned, 09-01467-216 (Sept. 16, 
2009). 

[16] For example, our prior work has shown negative cost and schedule 
implications for complex services acquisitions at the Department of 
Homeland Security that did not have adequate staff. See GAO, Department 
of Homeland Security: Better Planning and Assessment Needed to Improve 
Outcomes for Complex Service Acquisitions, [hyperlink, 
http://www.gao.gov/products/GAO-08-263] (Washington, D.C.: Apr. 22, 
2008). 

[17] Executive Office of the President, Office of Management and 
Budget, Circular No. A-11, Part 7, Planning, Budgeting, Acquisition, 
and Management of Capital Assets (Washington, D.C., June 2008). The OMB 
guidelines state that risk adjusted life cycle costs include the 
overall estimated cost over the time period corresponding to the life 
of the investment, including periodic and continuing costs of 
operations and maintenance. 

[18] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for 
Developing and Managing Capital Program Costs, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009). 

[19] To develop probability-based estimated costs leading to a risk-
adjusted independent cost estimate for the FLITE program, SPAWAR used a 
risk-based cost-estimating technique called Latin Hypercube sampling. 
Latin Hypercube sampling is a sampling technology designed to ensure 
high forecast efficiency and accurately recreate the input distribution 
through sampling. Sampling is forced to represent values in each 
interval and, thus, is forced to recreate the input probability 
distribution. For the independent cost evaluation, SPAWAR used 50,000 
iterations of the model to generate estimates from the model to enhance 
statistical efficiency. SPAWAR also obtained additional inputs from the 
FLITE PMO and obtained specific responses to independent cost 
evaluation team questions from the VA PMO, and subsequently 
reconsidered and modified some cost-related assumptions that VA used in 
creating their estimate. 

[20] OMB issued policy guidance (M-05-23) to agency chief information 
officers on improving technology projects that includes requirements 
for reporting performance to OMB using EVM (August 2005). VA, VA Earned 
Value Management System, VA Directive 6061, (February 2006). 

[21] See OMB, Capital Programming Guide, II.2.4, Establishing an Earned 
Value Management System. Reflected in the Federal Acquisition 
Regulation (FAR), 48 C.F.R. subpart 34.2. 

[22] Typically, an independent organization conducts the compliance 
review of an EVM system. Upon successful completion of the review, 
system acceptance should be documented, showing how each of the 32 
ANSI/EIA Standard 748 guidelines have been satisfied. 

[23] [hyperlink, http://www.gao.gov/products/GAO-09-3SP]. 

[24] SEI is a federally funded research and development center whose 
objective is to provide leadership in software engineering and the 
transition of new software engineering technology into practice. 

[25] Specifically, the Federal Financial Management Improvement Act of 
1996 states that financial management systems must comply substantially 
with financial management systems requirements located in a series of 
publications entitled Federal Financial Management Systems 
Requirements, which specifies the mandatory functional and technical 
requirements that agencies’ systems must meet. 

[26] FSIO is responsible for coordinating the work related to federal 
financial management systems requirements. 

[27] FSIO, Core Financial System Requirements, OFFM-NO-0106 
(Washington, D.C., January 2006) and Inventory, Supplies, and Materials 
System Requirements, JFMIP-SR-03-02 (Washington, D.C., August 2003). 

[28] Recording the acceptance or rejection of goods and services, as 
well as resolving overdue in-transit items, are useful when evaluating 
the performance of suppliers and in reconciling asset management data 
with financial data such as payment and general ledger posting 
activities. Additionally, recording the date goods and services are 
accepted is a critical data element for ensuring an agency complies 
with the Prompt Payment Act. (Prompt Payment Act codified at 31 U.S.C. 
§§ 3901-3904 and implemented at 5 C.F.R. 1315.) 

[29] According to the VA FLITE Real Property Strategy Working Paper, VA 
needs to perform additional analysis of VA business needs to determine 
if more real property business requirements are needed. Additionally, 
the FLITE program office plans to develop the additional requirements 
related to procurement, data analyses, and reporting during the IFCAP 
Replacement Development phase of the IFAS project and the data 
warehouse scheduled to begin in October 2009 and September 2010, 
respectively. 

[30] Institute of Electrical and Electronics Engineers, Inc., IEEE 
Standard for Software Verification and Validation, IEEE Std 1012-2004 
(New York, N.Y., June 8, 2005). 

[31] GAO, Office of Personnel Management: Improvements Needed to Ensure 
Successful Retirement Systems Modernization, [hyperlink, 
http://www.gao.gov/products/GAO-08-345] (Washington, D.C.: Jan. 31, 
2008) and Homeland Security: First Phase of Visitor and Immigration 
Status Program Operating, but Improvements Needed, [hyperlink, 
http://www.gao.gov/products/GAO-04-586] (Washington, D.C.: May 11, 
2004). 

[32] VA’s Systems Quality Assurance Service is organizationally 
independent of the FLITE Program and reports directly to the Office of 
Business Oversight and Office of Management for the escalation of 
significant issues. 

[33] Standards have been developed by the Institute of Electrical and 
Electronics Engineers (IEEE) to guide the development of software 
quality assurance plans, which typically include information on roles 
and responsibilities, tasks, metrics, software reviews, system tests, 
and independent verification and validation tools that will be utilized 
to support the software quality assurance process. 

[34] The document owner is responsible for managing the document review 
process; this includes identifying appropriate reviewers, consolidating 
adjudicated comments, and determining the acceptability of work 
products. 

[35] GAO, GAO Cost Estimating and Assessment Guide: Best Practices for 
Developing and Managing Capital Program Costs, [hyperlink, 
http://www.gao.gov/products/GAO-09-3SP] (Washington, D.C.: March 2009). 

[36] SEI is a federally funded research and development center whose 
objective is to provide leadership in software engineering and in the 
transition of new software engineering technology into practice. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: