Skip to main content

Information Technology: Defense Information Systems Agency Can Improve Investment Planning and Management Controls

GAO-02-50 Published: Mar 15, 2002. Publicly Released: Mar 15, 2002.
Jump To:
Skip to Highlights

Highlights

The Defense Information Systems Agency (DISA) provides the military with computing, telecommunications, and acquisition services on a cost reimbursable basis. In fiscal year 2001, DISA's service reimbursements totaled $2.5 billion. DISA also runs joint warfighting and related mission support command, control, and communications systems funded by direct appropriations, which in fiscal year 2001 were $1 billion. In March 2001, DISA issued a 500 Day Action Plan for Supporting DoD Decision Superiority. The plan contains 140 ongoing or planned actions involving the investment of resources. In developing the plan, DISA appropriately focused on understanding and satisfying customer concerns and needs. DISA did not, however, adequately address other elements of effective plan development, such as ensuring that planned investments were cost-effective. Although the agency did not establish baseline commitments in developing its action plan, DISA has since established some, but not all, baselines and is beginning to monitor progress against these commitments. DISA's 500 Day Action Plan is part of management actions to improve mission performance. These actions address some, but not all, of the institutional management controls that can help an agency effectively adjust to shifts in strategic direction. These controls include strategic planning, information technology (IT) human capital management, organizational structure management, enterprise architecture management, IT investment management, customer relations management, and knowledge management. Some of these management controls are in their more formative stage, while others have progressed much farther. Until each control is fully functioning, DISA will be challenged in its strategic direction and in maximizing its performance and accountability.

Recommendations

Recommendations for Executive Action

Agency Affected Recommendation Status
Department of Defense To improve the Defense Information Systems Agency's (DISA) development and execution of its current and future information technology (IT) investment action plans, the Secretary of Defense should direct the DISA director, through the assistant secretary of defense for command, control, communications, and intelligence, to follow a structured and disciplined IT investment management process for selection, control, and evaluation of the initiatives in current and future action plans.
Closed – Implemented
DISA has taken several actions that are consistent with a structured IT investment process. For example, DISA has established an IT investment review board that is responsible for ensuring that funding decisions for new IT proposals are in accordance with DISA's IT portfolio management process, which includes the selection, control, and evaluation of IT projects within an agency-wide portfolio. It also drafted an IT portfolio management instruction that defines policies, roles and responsibilities, and procedures for portfolio management within DISA. In addition, DISA has defined criteria for selecting IT investments that includes risk factors (such as size, project longevity, and technical risk), as well as business impact or mission effectiveness, customer needs, organizational impact, and expected improvement.
Defense Information Systems Agency For plan development, the DISA director should define the general scope of actions and establish preliminary life-cycle cost, schedule, benefit, and risk baselines for actions.
Closed – Implemented
DISA reported that it has defined the general scope of actions and established preliminary life-cycle cost, schedule, benefit, and risk baselines for each planned action included in the 2002 500-day action plan, and its revised 2004 action plan.
Defense Information Systems Agency For plan development, the DISA director should perform a preliminary, high-level assessment of return on investment for proposed actions to gauge their cost-effectiveness.
Closed – Implemented
DISA reported that it assessed and considered the return on investment in its IT investments' business cases for its 2004 500-day action plan.
Defense Information Systems Agency For plan implementation, the DISA Director should use approved baselines to develop meaningful results-oriented performance metrics.
Closed – Implemented
DISA has developed results-oriented performance metrics such as schedule, cost, risk, and exit criteria, and is using these metrics in its monthly investment tracking reports. DISA provided examples of these tracking reports for several action items.
Defense Information Systems Agency For plan implementation, the DISA Director should implement a formal process to (1) control significant changes to action baselines and closure of actions and (2) inform stakeholders of significant deviations in the action baselines.
Closed – Implemented
DISA reported that it implemented a process to control significant changes to action baselines and closure of actions by informing stakeholders of significant deviations in baselines and advising customers when it is closing an action. DISA reported that it was also obtaining customer concurrence with significant changes to action cost, scope, and schedule baselines. For example, in a letter from the DISA Director to the Assistant Secretary of Defense for Command, Control, Communications, and Intelligence (a DISA customer), the Director provided the status of specific actions, such as the number of actions completed and the number on schedule or delayed, and requested that the Assistant Secretary's staff review the status and comment on DISA's progress.
Defense Information Systems Agency For plan implementation, the DISA Director should, in monitoring implementation of the planned actions, update the scope of work, cost, schedule, benefit, and risk baselines for all actions, as appropriate, to ensure that actions remain cost-effective investment choices.
Closed – Implemented
DISA reported that it had developed a project monitoring process for ensuring that baseline commitments (scope of work, cost, schedule, benefit, and risk) were periodically updated. DISA reported that its personnel were also briefing the status of the actions' baseline commitments to the DISA Director at monthly meetings.
Defense Information Systems Agency For plan implementation, the DISA Director should establish a mechanism to track customer feedback to ensure that the customer concerns that led to the actions are resolved.
Closed – Implemented
DISA developed and implemented a formal process for seeking customer feedback and ensuring that customer concerns were resolved. As part of the process, DISA asked that customers validate or comment on DISA's progress in meeting their requests, describing DISA's status on action items for those customers. Customers responded to DISA's feedback requests by providing input on the status of DISA's actions.
Department of Defense To improve institutional management controls needed to respond to changes in strategic direction, the Secretary of Defense should direct the DISA director, through the assistant secretary of defense for command, control, communications, and intelligence, to make it an agency priority to establish the elements described in this report for each of the following management controls: (1) strategic planning, (2) organizational structure management, (3) enterprise architecture management, (4) IT investment management, (5) customer relations management, and (6) knowledge management. For IT human capital management, GAO makes no recommendations in light of the fact that DISA has either completed or is close to completing each of the important elements of effective IT human capital management discussed in this report.
Closed – Implemented
DISA has given priority to establishing and implementing the six key institutional management controls we recommended. For example, in the area of strategic planning, DISA reported that it established a corporate board, conducted senior management offsite meetings, and performed reviews to identify programmatic issues that offer potential for revision of the strategic plan. It also reported that it implemented a balanced score card approach to ensure that strategic planning extended across the organization. In the areas of organizational structure and customer relations management, DISA reported that it used the balanced score card approach for driving organizational priorities through the organization, reorganized, and created the Customer Advocacy Directorate to foster and sustain strong customer relations throughout the agency. DISA also reported that it developed processes to address and resolve customer issues and present new business opportunities to product and service managers. In the area of enterprise architecture (EA) management, DISA developed an EA program plan that describes its architecture process and approach and reported that it followed the CIO Council guidance on architecture management to develop an enterprise architecture for guiding its IT investments. In the area of IT investment management, DISA has established an IT investment review board; drafted a IT portfolio management instruction that outlines its policies, roles and responsibilities, and procedures for an IT portfolio management process; and established criteria for selecting IT investments. In the area of knowledge management, DISA reported that it has implemented a knowledge management program for sharing information and developed a web portal for supporting the program.
Defense Information Systems Agency To strengthen the agency's strategic planning, the DISA director should fully define approaches or strategies to achieve goals and objectives.
Closed – Implemented
DISA reported that it has implemented a balanced scorecard concept of operations that translates its mission, vision, and strategies into operational terms from the corporate level to the component level. According to DISA, this approach has pulled together all DISA organizational strategic planning to fully define the achievement of DISA's goals and objectives.
Defense Information Systems Agency To strengthen the agency's strategic planning, the DISA director should completely explain the relationship between the general goals and the annual performance goals.
Closed – Implemented
DISA reported that it implemented a balanced scorecard approach in its annual performance plans, which, according to DISA, incorporates general executive-level performance goals and tracks results in its annual performance plans and describes the connection between the executive-level goals and DISA's annual performance goals.
Defense Information Systems Agency To strengthen the agency's strategic planning, the DISA director should fully describe how program evaluations are used to establish and revise strategic goals.
Closed – Implemented
DISA reported that it established a corporate board, conducts senior management meetings, holds senior leadership offsite meetings, and performs individual program reviews to identify programmatic issues to establish and revise strategic goals.
Defense Information Systems Agency As part of its ongoing organizational structure management, the DISA director should evaluate and implement solutions for advancing coordination, productivity, and team-building.
Closed – Implemented
DISA stated that it has implemented a balanced scorecard approach to organizational management that clarifies DISA's strategic direction and customer needs--both internally and externally. DISA provided documentation that described how the balanced score card was used to communicate the drivers of performance by which DISA was to achieve its goals to all DISA personnel and customers. DISA also reported that this approach involved coordination of its major business units and cross coordination meetings, along with metrics to reflect how well the groups work together.
Defense Information Systems Agency To strengthen management of DISA's effort to develop, implement, and maintain an enterprise architecture, the DISA director should follow the steps defined in the Chief Information Officer's (CIO) Council's guide on architecture management, as appropriate, including initiating a program.
Closed – Implemented
DISA initiated an enterprise architecture program that includes gaining executive management buy-in and support for an architecture development strategy. DISA developed an enterprise architecture concept of operations (CONOPS) that was approved by the DISA Chief Information Officer. The CONOPS describes roles and responsibilities, the IT governance structure, and detailed phases of IT governance using the enterprise architecture.
Defense Information Systems Agency To strengthen management of DISA's effort to develop, implement, and maintain an enterprise architecture, the DISA director should follow the steps defined in the CIO's Council's guide on architecture management, as appropriate, including defining the architecture process and approach.
Closed – Implemented
DISA developed an enterprise architecture action plan that describes its architecture development approach and process, from initiating the program to developing a transition plan from the as-is to to-be architectures.
Defense Information Systems Agency To strengthen management of DISA's effort to develop, implement, and maintain an enterprise architecture, the DISA director should follow the steps defined in the CIO's Council's guide on architecture management, as appropriate, including developing the architecture, including the baseline and target architectures, and the plan for sequencing from the baseline to the target.
Closed – Implemented
DISA reported that it had finalized its as-is- and to-be architectures and published version 1.0 of a transition plan. The plan includes a description of current systems, planned retirement or modification dates, and replacement systems.
Defense Information Systems Agency To strengthen management of DISA's effort to develop, implement, and maintain an enterprise architecture, the DISA director should follow the steps defined in the CIO's Council's guide on architecture management, as appropriate, including using the architecture in making IT investment decisions.
Closed – Implemented
According to the DISA draft IT portfolio management instruction, DISA's Director of Strategic Planning and Information Directorate is to ensure that DISA's business system IT investments are consistent with its enterprise architecture requirements. Also, the draft instruction requires that each IT investment be ranked numerically using DISA criteria to determine each investment's relative importance within DISA's enterprise architecture.
Defense Information Systems Agency To strengthen management of DISA's effort to develop, implement, and maintain an enterprise architecture, the DISA director should follow the steps defined in the CIO's Council's guide on architecture management, as appropriate, including maintaining the architecture.
Closed – Implemented
The DISA Chief Information Officer established an enterprise architecture program management office that is responsible for maintaining the DISA enterprise architecture. Within this office, the Chief Architect has overall responsibility for the enterprise architecture, including configuration management and annual updates.
Defense Information Systems Agency To strengthen management of DISA's effort to develop, implement, and maintain an enterprise architecture, the DISA director should follow the steps defined in the CIO's Council's guide on architecture management, as appropriate, including continuously controlling and overseeing the program.
Closed – Implemented
The DISA Chief Information Officer (CIO) established an enterprise architecture program management office that is responsible for maintaining the DISA enterprise architecture. Within this office, the Chief Architect has overall responsibility for the enterprise architecture, including configuration management and annual updates. The DISA CIO also established an enterprise architecture working group to develop and maintain the DISA enterprise architecture to ensure synchronization and integration among internal business processes and systems used within DISA.
Defense Information Systems Agency To establish effective IT investment management, the DISA director should follow the steps detailed in GAO's IT investment management guide, including building a foundation for IT investments, including establishing and operating an IT investment board.
Closed – Implemented
DISA established its IT investment board and held its first IT investment board in February 2002. According to meeting minutes, the DISA IT Investment Working Group, which reports to the investment board, is to review IT investments and make recommendations to the IT Board on the investments that would benefit DISA.
Defense Information Systems Agency To establish effective IT investment management, the DISA director should follow the steps detailed in GAO's IT investment management guide, including building a foundation for IT investments, including performing IT project oversight.
Closed – Implemented
The draft DISA IT portfolio management instruction requires regular and systematic evaluation of projects in DISA's IT portfolio, including ensuring that quantitative and qualitative investment data are collected, evaluated, and analyzed during portfolio investment reviews.
Defense Information Systems Agency To establish effective IT investment management, the DISA director should follow the steps detailed in GAO's IT investment management guide, including building a foundation for IT investments, including tracking IT assets.
Closed – Implemented
DISA reported that it completed a 100 percent inventory of its IT assets and recorded the results in the Defense Property Accounting System, its official property accounting system.
Defense Information Systems Agency To establish effective IT investment management, the DISA director should follow the steps detailed in GAO's IT investment management guide, including building a foundation for IT investments, including identifying business needs for IT projects.
Closed – Implemented
DISA's draft IT portfolio management instruction requires that project managers identify and define business needs for each IT project. In addition, according to the instruction, IT investments are reviewed and analyzed using DISA's IT investment criteria, which includes an assessment of business impact.
Defense Information Systems Agency To establish effective IT investment management, the DISA director should follow the steps detailed in GAO's IT investment management guide, including building a foundation for IT investments, including selecting proposals systematically.
Closed – Implemented
According to the draft IT portfolio management instruction, the Director of Strategic Planning and Information Directorate is to conduct mission area portfolio analyses and prioritize the IT investment portfolio using investment criteria and assign each investment to a portfolio category.
Defense Information Systems Agency To establish effective IT investment management, the DISA director should follow the steps detailed in GAO's IT investment management guide, including establishing the capability to manage investments as a complete investment portfolio, including defining portfolio selection criteria.
Closed – Implemented
According to DISA's draft IT portfolio management instruction, all IT investments are to be assessed using the DISA IT investment criteria, which are intended to indicate how well each IT investment supports DISA's vision, corporate strategies, goals, objectives, and required capabilities and priorities. Elements of the IT investment criteria, currently used in DISA's investment selection process, includes criteria on risk factors (investment size, project longevity, and technical risk)and overall factors (business impact or mission effectiveness, customer needs, quantitative analysis, organizational impact, and expected improvement). The draft instruction states that each IT investment will be ranked numerically against all other proposed and existing IT investments to determine its relative prioritized importance with DISA's enterprise architecture and IT funding.
Defense Information Systems Agency To establish effective IT investment management, the DISA director should follow the steps detailed in GAO's IT investment management guide, including establishing the capability to manage investments as a complete investment portfolio, including analyzing investments.
Closed – Implemented
DISA's draft IT portfolio management instruction states that the Director of Strategic and Information Directorate is responsible for periodically analyzing IT investments for selection based on its IT investment criteria (overall risk and return factors), corporate strategies, and enterprise architecture.
Defense Information Systems Agency To establish effective IT investment management, the DISA director should follow the steps detailed in GAO's IT investment management guide, including establishing the capability to manage investments as a complete investment portfolio, including developing an investment portfolio.
Closed – Implemented
DISA's draft IT investment portfolio instruction states that DISA will develop an IT investment portfolio and that each investment within the portfolio is to be periodically assessed to determine whether to continue or terminate the investment.
Defense Information Systems Agency To establish effective IT investment management, the DISA director should follow the steps detailed in GAO's IT investment management guide, including establishing the capability to manage investments as a complete investment portfolio, including overseeing portfolio performance.
Closed – Implemented
DISA's draft IT portfolio management instruction indicates that each investment in the portfolio is to be regularly evaluated.
Defense Information Systems Agency To strengthen customer relations management, the DISA director should build and maintain a supporting customer relations infrastructure that permeates the entire organization.
Closed – Implemented
DISA reported that it had reorganized and created the Customer Advocacy Directorate to foster and sustain strong customer relations throughout the agency. It provided organization charts to show the directorate's position in the agency. DISA also developed processes to identify, address, and resolve customer issues across DOD. DISA also stated that it created a Senior Executive Account Managers (SEAM) program to provide major DISA customers (such as the Defense Logistics Agency) multiple levels of interface and mechanisms to convey their needs to their DISA service provider. DISA also stated that the DISA Director's digital dashboard presents a customer score card that identifies customer action status, issues, concerns, and actions to be taken.
Defense Information Systems Agency To define and implement an organizationally integrated knowledge management function, the DISA director should follow the steps outlined in the CIO Council guide on this subject, including deciding with whom to share organizational knowledge.
Closed – Implemented
DISA's Knowledge Management Capstone Requirements document identifies several users with whom DISA intends to share knowledge. Users identified include, among others, the DISA Operations Directorate, the Chief Financial Executive, and the Chief Information Officer.
Defense Information Systems Agency To define and implement an organizationally integrated knowledge management function, the DISA director should follow the steps outlined in the CIO Council guide on this subject, including deciding what organizational knowledge to share.
Closed – Implemented
DISA's Knowledge Management Capstone Requirements document describes the information that DISA managers need to better perform their mission and how that information is to be shared. According to the requirements document, information to be shared includes, but is not limited to, lessons learned, best practices, transformation initiatives, and customer profiles.
Defense Information Systems Agency To define and implement an organizationally integrated knowledge management function, the DISA director should follow the steps outlined in the CIO Council guide on this subject, including deciding how to share organizational knowledge.
Closed – Implemented
DISA developed a knowledge management requirements framework that established an approach for sharing organizational knowledge. DISA reported that it completed a knowledge management enterprise portal that allows use of the agency's intranet to share organizational information throughout the agency on issues such as tracking the status of DISA programs and sharing DISA issues and their resolution.
Defense Information Systems Agency To define and implement an organizationally integrated knowledge management function, the DISA director should follow the steps outlined in the CIO Council guide on this subject, including institutionalizing and using the knowledge management process.
Closed – Implemented
DISA created a pilot program to establish knowledge communities that initially focused on resource management, contract management, and wireless technology; additional communities were to be added at a later time. The knowledge communities were to use collaboration tools to provide a means for developing specialized information collections and databases for each knowledge community. DISA also developed an enterprise portal project that disseminates organizational information throughout the agency.

Full Report

Office of Public Affairs

Topics

Agency missionsBest practicesEnterprise architectureHuman capital ITInformation systemsInformation technologyInternal controlsIT human capitalPerformance measuresStrategic planningIT investment management