DOD Business Systems Modernization:

Planned Investment in Navy Program to Create Cashless Shipboard Environment Needs to Be Justified and Better Managed

GAO-08-922: Published: Sep 8, 2008. Publicly Released: Sep 8, 2008.

Additional Materials:

Contact:

Randolph C. Hite
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

GAO has designated the Department of Defense's (DOD) multi-billion dollar business systems modernization efforts as high risk, in part because key information technology (IT) management controls have not been implemented on key investments, such as the Navy Cash program. Initiated in 2001, Navy Cash is a joint Department of the Navy (DON) and Department of the Treasury Financial Management Service (FMS) program to create a cashless environment on ships using smart card technology, and is estimated to cost about $320 million to fully deploy. As requested, GAO analyzed whether DON is effectively implementing IT management controls on the program, including architectural alignment, economic justification, requirements development and management, risk management, security management, and system quality measurement against relevant guidance.

Key IT management controls have not been effectively implemented on Navy Cash, to the point that further investment in this program, as it is currently defined, has not been shown to be a prudent and judicious use of scarce modernization resources. In particular, Navy Cash has not been (1) assessed and defined in a way to ensure that it is not duplicative of programs in the Air Force and the Army that use smart card technology for electronic retail transactions and (2) economically justified on the basis of reliable analyses of estimated costs and expected benefits over the program's life. As a result, DON cannot demonstrate that the investment alternative that it is pursuing is the most cost-effective solution to satisfying its mission needs. Moreover, other management controls, which are intended to maximize the chances of delivering defined and justified system capabilities and benefits on time and within budget, have not been effectively implemented. System requirements have not been effectively managed. For example, neither policies nor plans that define how system requirements are to be managed, nor an approved baseline set of requirements that are justified and needed to cost-effectively meet mission needs, exist. Instead, requirements are addressed reactively through requests for changes to the system based primarily on the availability of funding. Program risks have not been effectively managed. In particular, plans, processes, and procedures that provide for identifying, mitigating, and disclosing risks have not been defined, nor have risk-related roles and responsibilities for key stakeholders. System security has not been effectively managed, thus putting the confidentiality, integrity, and availability of deployed and operating shipboard devices, applications, and data at increased risk of being compromised. For example, the mitigation of system vulnerabilities by applying software patches has not been effectively implemented. Key aspects of system quality are not being effectively measured. For example, data for determining trends in unresolved system change requests, which is an indicator of system stability, as well as user feedback on system satisfaction, are not being collected and used. Program oversight and management officials acknowledged these weaknesses and cited turnover of staff in key positions and their primary focus on deploying Navy Cash as reasons for the state of some of these IT management controls. Collectively, this means that, after investing about 6 years and $132 million on Navy Cash and planning to invest an additional $60 million to further develop the program, the department has yet to demonstrate through verifiable analysis and evidence that the program, as currently defined, is justified. Moreover, even if further investment was to be demonstrated, the manner in which the delivery of program capabilities is being managed is not adequate. As a result, the program is at risk of delivering a system solution that falls short of cost, schedule, and performance expectations.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: Because of the uncertainty surrounding whether Navy Cash, as defined, represents a cost-effective solution, the Secretary of Defense should direct the Secretary of the Navy to limit further investment of modernization funding in the program to only (1) deployment to remaining ships of already developed and tested capabilities; (2) correction of information security vulnerabilities and weaknesses on ships where it is deployed and operating; and (3) development of the basis for an informed decision as to whether further development and modernization is economically justified and in the department's collective best interests.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department has taken actions consistent with our recommendation. Regarding part 1 of the recommendation, the department limited the use of modernization funding to fielding and maintaining the currently developed and tested Navy Cash system. Further, it continued this approach when installing the system on the remaining ships. Regarding part 2, the Navy fielded software security patches automatically to ships on which Navy Cash is deployed and operating and monitored that they were successfully applied. Regarding part 3, Navy completed a revised economic analysis in November 2009, which concluded that further development and modernization is economically justified. This analysis was then used as a basis to help inform the Milestone Decision Authority's January 26, 2010 decision to proceed with further investment.

    Recommendation: To develop the basis for an informed decision about further Navy Cash development, the Secretary of Defense should direct the appropriate DOD organizations to (1) examine the relationships among DOD's programs for delivering military personnel with smart card technology for electronic retail and banking transactions; (2) identify, in coordination with the respective program offices, alternatives for optimizing the relationships of these programs in a way that minimizes areas of duplication, maximizes reuse of shared services across the programs, and considers opportunities for a consolidated stored value card program across the military services; and (3) share the results with the appropriate organizations for use in making an informed decision about planned investment in Navy Cash.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department took actions consistent with our recommendation. Regarding parts 1 and 2 of the recommendation, in November 2009, the department revised the Navy Cash economic analysis, including examining the smart card technology used by the Air Force and Army, and considered an alternative in which it would adopt one of these other systems rather than Navy Cash. However, it reported that this alternative was not technically feasible because although there was some overlap with the capabilities of the systems used by the Air Force and Army, these systems did not meet all of the Navy's requirements. For example, one Navy requirement was that the system help reduce the workload of shipboard disbursement personnel. Navy Cash met this requirement by providing sailors with cards that allow sailors to reload value on their cards without assistance from disbursement personnel, and that allow sailors to access personal funds in banks or credit unions ashore. The analysis stated that the systems used by the Air Force and Army did not have these capabilities and, thus, did not meet Navy requirements. Regarding part 3 of the recommendation, the Navy Cash program shared the results of the economic analysis with the Milestone Decision Authority, who used the results in deciding to approve, in January 2010, further investment in the Navy Cash program.

    Recommendation: To further develop this basis for an informed decision about Navy Cash development, the Secretary of Defense should direct the Secretary of the Navy to ensure that the appropriate Navy organizational entities prepare a reliable economic analysis that encompasses the program's total life cycle costs, including those of FMS, and that (1) addresses cost-estimating best practices and complies with relevant Office of Management and Budget (OMB) cost-benefit guidance and (2) incorporates data on whether deployed Navy Cash capabilities are actually producing benefits.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department has taken steps to address this recommendation. Specifically, in December 2009, the department revised its 2002 economic analysis for the Navy Cash program to address the cost-estimating best practices that we reported as missing, comply with relevant OMB guidance, and incorporate data on actual benefits. For example, the revised analysis now provides more current estimates of the program's cost and benefits, it considers the costs and benefits of three alternative solutions, and it includes data on actual benefits being achieved, such as reduced labor costs. Thus, the revised economic analysis should provide a more reliable basis for investment decision making.

    Recommendation: To address Navy Cash information security management weaknesses and improve the operational security of the system, Secretary of Defense should direct the Secretary of the Navy to ensure that the Navy Cash program manager, in collaboration with the appropriate organizations, develop and implement a patch management approach based on National Institute of Standards and Technology (NIST) guidance, which includes a complete Navy Cash systems inventory; an automated patch deployment capability; and a patch management performance vulnerability measurement capability, including metrics for susceptibility to attack and mitigation response time.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Navy provided evidence that it has implemented an automated patch management process as part of an update to the Navy Cash system. The system also allows the viewing of metrics related to patch deployment.

    Recommendation: To address Navy Cash information security management weaknesses and improve the operational security of the system, Secretary of Defense should direct the Secretary of the Navy to ensure that the Navy Cash program manager, in collaboration with the appropriate organizations, institute a process to plan, implement, evaluate, and document remedial actions for deficiencies in Navy Cash information security policies, procedures, and practices, and ensure that this process meets Financial Information Security Management Act requirements, as well as applicable OMB and NIST guidance.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Navy Cash Program Office has developed a Plan of Action and Milestones in accordance with OMB and NIST guidance to identify and track progress in planning, implementing, evaluating, and documenting remedial actions for information security vulnerabilities in the Navy Cash system.

    Recommendation: To address Navy Cash information security management weaknesses and improve the operational security of the system, Secretary of Defense should direct the Secretary of the Navy to ensure that the Navy Cash program manager, in collaboration with the appropriate organizations, update the Naval Supply Systems Command (NAVSUP)/FMS memorandum of agreement, in collaboration with FMS, to establish specific security requirements for FMS and the financial agent to periodically perform information security control reviews, including applicable management, operational, and technical controls, of the Navy Cash system, and to provide NAVSUP with copies of the results of these reviews that pertain to the Navy Cash system and its supporting infrastructure.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Navy, in coordination with its Treasury FMS partner, has acted to address this recommendation. Specifically, in July, 2012, Treasury FMS provided the Navy Cash program executive with a security assessment report that included an assessment of Navy Cash security controls, identified threats, identified individuals with access to sensitive information, and assessed vulnerabilities and efforts to remediate vulnerabilities. In addition, Treasury provided the program with a certification statement regarding the security of the Navy Cash system, as well as new policies that FMS established with its financial agent regarding personnel security measures, and additional physical security measures. Further, the Navy Cash program executive stated in July 2012 that the program would work with Treasury to update the memorandum agreement to reflect the security guidelines that are placed on the financial agent.

    Recommendation: To address Navy Cash information security management weaknesses and improve the operational security of the system, Secretary of Defense should direct the Secretary of the Navy to ensure that the Navy Cash program manager, in collaboration with the appropriate organizations, develop a complete contingency plan to include a sequence of recovery activities.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In August 2011, the Navy provided evidence that it had updated its contingency planning guide, tested its contingency plan, and documented the results.

    Recommendation: To address Navy Cash information security management weaknesses and improve the operational security of the system, Secretary of Defense should direct the Secretary of the Navy to ensure that the Navy Cash program manager, in collaboration with the appropriate organizations, develop a complete contingency plan to include procedures for notifying ship personnel with contingency plan responsibilities to begin recovery activities; and to test the contingency plan in accordance with NIST guidance, including documenting lessons learned from testing.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: In August 2011, the Navy provided evidence that it had updated its contingency planning guide, tested its contingency plan, and documented the results.

    Recommendation: To address DON information security guidance limitations, the Secretary of Defense should direct the Secretary of the Navy to ensure that the Navy Operational Designated Approving Authority, as part of the Naval Network Warfare Command, updates its certification and accreditation guidance to require the development of plans of action and milestones for all above identified security weaknesses.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Navy Cash program office developed a plan of action and milestones as part of the authority to operate that was granted to the Navy Cash system in December 2008. The plan of action and milestones documents security weaknesses, such as technical vulnerabilities, and the status and plans of action to mitigate these vulnerabilities based on the requirements in Department of Defense Instruction 8510.01.

    Recommendation: If further investment in development of Navy Cash can be justified, the Secretary of Defense should direct the Secretary of the Navy, through the appropriate chain of command, to ensure that the Navy Cash program manager with respect to requirements development and management, (1) develop detailed system requirements; (2) establish policies and plans for managing changes to requirements, including defining roles and responsibilities, and identifying how the integrity of a baseline set of requirements will be maintained; and (3) maintain bi-directional requirements traceability.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department concurred with the recommendation and, in response, took specific and appropriate actions: (1) In August 2011, the department revised requirements for its ongoing modernization of the Navy Cash system and developed detailed system requirements to guide the design and development of this effort. (2) In March 2009, it developed its policy for managing changes to requirements. The policy identifies what information must be included in change requests, how the requests are to be prioritized, and a standardized set of procedures for documenting, managing, controlling, and approving changes to requirements. Also, in July 2009, the charter for a change management approval group was approved, which defined specific responsibilities for the review of change requests and procedures for implementing approved requests. (3) To help maintain bi-directional traceability, in February 2009, the department revised quality assurance procedures to require that requirements be linked to test plans and to other related requirements to help ensure that no requirements are overlooked during testing and to identify how a change to a requirement might affect other related requirements.

    Recommendation: If further investment in development of Navy Cash can be justified, the Secretary of Defense should direct the Secretary of the Navy, through the appropriate chain of command, to ensure that the Navy Cash program manager with respect to risk management, (1) establish and implement a written plan and defined process for risk identification, analysis, and mitigation; (2) assign responsibility for managing risk to key stakeholders; (3) encourage program-wide participation in risk management; (4) include and track the risks discussed in this report as part of a risk inventory; and (5) apprise decision making and oversight authorities of the status of risks identified during program reviews.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The Navy has taken several actions that are consistent with our recommendation. First, the Navy Cash program office developed a risk management plan and a risk mitigation plan, which include a process for identifying, analyzing, and mitigating program risks. Second, the program office developed and periodically updates a risk inventory that includes, among other things, descriptions of program risks, and identifies the person responsible for managing each risk. Third, a risk management board was established to review each risk and the planned mitigation strategy, and to serve as a forum for affected stakeholders to state their risk-related concerns. Fourth, the program office tracks progress in mitigating the risks. Fifth, decision-making and oversight authorities (e.g. the milestone decision authority, as well as the Navy's Director of Disbursing, and the Treasury's manager of stored value programs) are apprised of risk status at bi-monthly Program Management Review meetings.

    Recommendation: If further investment in development of Navy Cash can be justified, the Secretary of Defense should direct the Secretary of the Navy, through the appropriate chain of command, to ensure that the Navy Cash program manager with respect to system quality measurement, collect and use sufficient data for (1) determining trends in unresolved change requests and (2) understanding users' satisfaction with the system.

    Agency Affected: Department of Defense

    Status: Closed - Implemented

    Comments: The department concurred with the recommendation, and in response, took appropriate actions. For example, to determine trends in unresolved change requests, it developed metrics on the number of new unresolved change requests, the number that were closed (resolved), and the associated cost for each of the past 4 fiscal years. These metrics are reported and monitored at monthly program management reviews. Regarding understanding user satisfaction with the system, the department conducted a survey in October 2010 of ships using the Navy Cash system. The program office analyzed the survey comments, identified areas for improvement, and, in December 2010, briefed department management on actions being undertaken to address those areas. For example, one area for improvement related to help desk responses that were sometimes slow or of poor quality. The program office stated that it would track help desk metrics to assure standards were met. Another area for improvement was the need to provide more "hands-on" training for shipboard disbursing and IT personnel to enable them to better troubleshoot and maintain the Navy Cash system. The program office stated that the training was being revised to include more hands-on training. As a result of these actions, the department has better information with which to monitor the quality of the Navy Cash system and more effectively manage the system.

    Apr 2, 2014

    Feb 26, 2014

    Feb 12, 2014

    Jan 13, 2014

    Nov 13, 2013

    Nov 6, 2013

    Sep 12, 2013

    Sep 11, 2013

    Looking for more? Browse all our products here