Weather Forecasting:

National Weather Service's Operations Prototype Needs More Rigorous Planning

GAO-07-650: Published: Jun 8, 2007. Publicly Released: Jul 9, 2007.

Additional Materials:

Contact:

David A. Powner
(202) 512-3000
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Using advanced systems and trained specialists located in 122 weather forecast offices throughout the country, the National Weather Service (NWS) provides storm and flood warnings and weather forecasts to protect life and property and to enhance the national economy. To improve the efficiency of its operations, in November 2006, NWS approved an effort to develop a prototype of an alternative way of operating. Under this prototype, weather forecasting offices would share selected responsibilities. GAO (1) determined the status of and plans for the prototype, (2) evaluated whether the prototype's justification was sufficient, (3) determined whether NWS's plans to evaluate the prototype are adequate, (4) evaluated whether NWS is sufficiently involving stakeholders in its prototype plans, and (5) determined how NWS plans to ensure that there will be no degradation of service during and after the prototype. To do so, GAO analyzed agency documentation and interviewed program officials and stakeholders.

NWS's prototype is currently on hold pending a reevaluation of the agency's approach. The agency had just begun the first phase of its three-phased prototype to demonstrate a new concept of operations over a 2-year period when, in late March 2007, the Department of Commerce's Under Secretary suspended the prototype because of concerns about the agency's approach. In the first phase, NWS established a program manager and began planning for the next two phases. During the remaining phases, NWS planned to have 20 weather forecasting offices share responsibilities in 2-office pairs and then in 4-office clusters. NWS then planned to decide whether to implement the new concept of operations on a national basis. The justification for the prototype was not sufficient. Before the prototype was suspended, the agency had approved moving forward with its prototype without conducting a cost-benefit analysis. NWS estimated that the prototype would cost approximately $9.3 million and would offer qualitative benefits, such as increased efficiency and an improved ability to focus on severe weather events, but did not quantify benefits or the expected return on its investment. If NWS were to proceed with the prototype without a cost-benefit analysis, it would lack assurance that its approach would be a cost-effective investment for the agency. NWS identified goals and selected measures to evaluate during its prototype activities, but it did not establish a rigorous evaluation plan. Specifically, NWS did not define a full set of needed measures, how it planned to compare prototype results with baseline performance, or how its selected measures supported the prototype goals. If NWS were to proceed without a rigorous evaluation plan, the agency would run an increased risk of not sufficiently measuring the impact of changes on its performance and could make decisions affecting the nation's weather on the basis of incomplete or flawed data. Although NWS involved internal stakeholders in planning its prototype, it did not (1) involve external stakeholders or (2) establish a plan that identified key stakeholders, the stakeholders' responsibilities, and a time line for involving stakeholders and addressing their comments. If NWS were to proceed with its prototype without such a plan, it could not ensure that stakeholder interests would be identified and addressed. NWS planned to mitigate the risk of degradation during and after the prototype by conducting laboratory exercises to understand the impact of the prototype, using a dedicated workstation in each office so that the office could switch to original systems if warranted, and monitoring its systems and products during the prototype. NWS officials stated that should the agency decide to implement the prototype on a national basis, it has standard procedures for testing and validating systems and software to avoid any degradation of service.

Recommendations for Executive Action

  1. Status: Closed - Implemented

    Comments: The agency agreed with the recommendation, but noted that all efforts to develop a new concept of operations prototype have ceased. Nonetheless, the agency has taken steps to improve how it involves key stakeholders in its initiatives. In November 2010, NWS and FAA issued a joint FAA/NWS Project Plan Roadmap for Traffic Flow Management Weather support. As part of this roadmap, NWS and FAA provide a list of stakeholders to be considered during the duration of their program. This list includes NextGen, the Joint Planning and Development Office, the International Civil Aviation Organization, the aviation industry, as well as internal FAA and NWS processes, and identified the applicable roles and responsibilities. Further, the roadmap states that both FAA and NWS unions are involved in the joint meetings. The plan to involve stakeholders from beginning will help the agencies better ensure that employee and customer interests are better anticipated, identified, and addressed.

    Recommendation: If NWS decides to proceed with its prototype, the Secretary of Commerce should direct the Assistant Administrator for the National Weather Service to develop a plan for internal and external stakeholder involvement, which includes a list of relevant stakeholders, roles and responsibilities for these stakeholders, and a schedule for when stakeholders should be involved before implementing the prototype in any weather forecasting office.

    Agency Affected: Department of Commerce

  2. Status: Closed - Not Implemented

    Comments: The agency agreed with the recommendation, but noted that all efforts to develop a new concept of operations prototype have ceased. The agency has no plans for a new concept of operations to which they can apply an evaluation plan.

    Recommendation: If NWS decides to proceed with its prototype, the Secretary of Commerce should direct the Assistant Administrator for the National Weather Service to develop an evaluation plan that includes a comprehensive set of measures that are linked to prototype goals and identifies the baseline performance that the prototype will be compared with before implementing the prototype in any weather forecasting office.

    Agency Affected: Department of Commerce

  3. Status: Closed - Not Implemented

    Comments: The department agreed with the recommendation, but noted that all efforts to develop a new concept of operations prototype have ceased. The agency has no plans to develop a new concept of operations to which to apply a cost-benefit analysis.

    Recommendation: If NWS decides to proceed with its prototype, the Secretary of Commerce should direct the Assistant Administrator for the National Weather Service to develop a cost-benefit analysis for the clustered peer approach before implementing the prototype in any weather forecasting office.

    Agency Affected: Department of Commerce

  4. Status: Closed - Implemented

    Comments: The department agreed with the recommendation. These system improvements were built into Advanced Weather Interactive Processing System (AWIPS) operational build 8.3. As of November 2008, NWS had installed this improvement at all of its AWIPS sites.

    Recommendation: The Secretary of Commerce should direct the Assistant Administrator for the National Weather Service to evaluate moving forward with technology upgrades to the Advanced Weather Interactive Processing System to allow weather forecast offices to switch to backup service more quickly during high-impact weather situations; these upgrades could be handled as an initiative that is separate and distinct from broader changes in the agency's concept of operations.

    Agency Affected: Department of Commerce

 

Explore the full database of GAO's Open Recommendations »

Sep 10, 2014

Jul 23, 2014

Jun 9, 2014

May 20, 2014

  • science icon, source: National Cancer Institute

    Nanomanufacturing and U.S. Competitiveness:

    Challenges and Opportunities
    GAO-14-618T: Published: May 20, 2014. Publicly Released: May 20, 2014.

Feb 7, 2014

Dec 20, 2013

Nov 4, 2013

  • science icon, source: National Cancer Institute

    Small Business Innovation Research:

    Data Rights Protections
    GAO-14-116R: Published: Nov 4, 2013. Publicly Released: Nov 4, 2013.

Jul 19, 2013

Apr 10, 2013

Feb 15, 2013

Looking for more? Browse all our products here