Customs Service Modernization:

Ineffective Software Development Processes Increase Customs System Development Risks

AIMD-99-35: Published: Feb 11, 1999. Publicly Released: Feb 11, 1999.

Contact:

Randolph C. Hite
(202) 512-6256
contact@gao.gov

 

Office of Public Affairs
(202) 512-4800
youngc1@gao.gov

Pursuant to a congressional request, GAO reviewed the Customs Service's software development maturity and improvement activities, focusing on: (1) the maturity of Customs' software development processes; and (2) whether Customs has an effective software process improvement program.

GAO noted that: (1) because of the number and severity of Customs' software development process weaknesses, Customs did not fully satisfy any of the key process areas (KPA) necessary to achieve the repeatable level of process maturity; (2) as a result, its processes for developing software, a complex and expensive component of Customs' systems, are ad hoc, sometimes chaotic, and not repeatable across projects; (3) Customs had some practice strengths in all but one of the five KPAs evaluated (i.e., requirements management, software project planning, software project tracking and oversight, software quality assurance, and software configuration management); however, GAO also found extensive and significant weaknesses in each of these KPAs; (4) some of these weaknesses were systemic, recurring in each of the KPAs; (5) for example, Customs had no written policy for managing or implementing any of the KPAs; (6) none of the projects had: (a) an approved quality assurance plan; (b) documented procedures for determining the project cost, schedule, or effort; or (c) any outside group reviewing or reporting on the project's compliance with defined processes; (7) these weaknesses are some of the reasons for Customs' limited success, for example, in delivering promised Automated Commercial Environment (ACE) capabilities on time; (8) Customs does not have a software development process improvement program, and it has not taken the basic steps to initiate one; (9) these steps, many of which are described in Software Engineering Institute's initiating, diagnosing, establishing, acting, and leveraging model for process improvement, include assigning responsibility and authority for process improvement, establishing a process improvement management structure, defining a plan of action, and committing needed resources; and (10) until Customs establishes an effective process improvement program, its software processes will remain poorly defined and undisciplined, and its software projects are likely to suffer cost, schedule, and performance shortfalls.

Status Legend:

More Info
  • Review Pending-GAO has not yet assessed implementation status.
  • Open-Actions to satisfy the intent of the recommendation have not been taken or are being planned, or actions that partially satisfy the intent of the recommendation have been taken.
  • Closed-implemented-Actions that satisfy the intent of the recommendation have been taken.
  • Closed-not implemented-While the intent of the recommendation has not been satisfied, time or circumstances have rendered the recommendation invalid.
    • Review Pending
    • Open
    • Closed - implemented
    • Closed - not implemented

    Recommendations for Executive Action

    Recommendation: After ensuring that its mission-critical systems are year 2000 compliant, but before investing in major software development efforts like ACE, the Commissioner of Customs should direct the Customs Chief Information Officer to assign responsibility and authority for software development process improvement.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security

    Status: Closed - Implemented

    Comments: In January 1999, Customs' Office of Information and Technology was reorganized. This reorganization included establishment of the technology and architecture group that is responsible, among other things, for developing and communicating software process improvement policies and plans. Additionally, the software development division was designated as responsible for implementing software process improvement (development and acquisition).

    Recommendation: After ensuring that its mission-critical systems are year 2000 compliant, but before investing in major software development efforts like ACE, the Commissioner of Customs should direct the Customs Chief Information Officer to develop and implement a formal plan for software development process improvement that is based on the software capability evaluation results contained in this report and specifies measurable goals and timeframes, prioritizes initiatives, estimates resource requirements (trained staff and funding), and defines a process improvement management structure.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security

    Status: Closed - Implemented

    Comments: Customs (now known as the Bureau of Customs and Border Protection or CBP) agreed with this recommendation and in mid-1999 prepared a software process improvement plan for achieving Software Engineering Institute Capability Maturity Model (CMM) level 2. Later that year, CBP formed a Software Engineering Process Group (SEPG) that prioritized initiatives, estimated resources, and defined an overall management structure for software process improvement activities. Further, the four individual software development groups in CBP's Software Development Division (SDD) established their own action plans, specifying measurable goals and timeframes for achieving software CMM level 2. CBP's progress toward implementing its software process improvement plans is reflected in the results of an April 2003 assessment in which SDD was rated at software CMM level 2.

    Recommendation: After ensuring that its mission-critical systems are year 2000 compliant, but before investing in major software development efforts like ACE, the Commissioner of Customs should direct the Customs Chief Information Officer to ensure that every new software development effort in Customs adopts processes that satisfy at least software capability maturity model level 2 requirements.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security

    Status: Closed - Implemented

    Comments: Customs (now known as the Bureau of Customs and Border Protection or CBP) agreed with this recommendation and has taken steps to ensure that its new software development efforts satisfy Software Engineering Institute Capability Maturity Model (CMM) level 2. Specifically, CBP revised its system development life-cycle (SDLC) policy to require consistency with level 2 processes. Further, the Chief Information Officer mandated that every new software development project comply with the SDLC. To determine whether individual projects, including all new software development efforts, satisfy level 2 processes, CBP has conducted three assessments of software development process maturity in its Software Development Division (SDD) from March 2001 to April 2003. The assessment in March 2001 showed that one of four SDD groups was CMM level 2 compliant and action plans were developed to bring the remaining three SDD groups to level 2. Another assessment was conducted in July 2002, and new improvement plans were made based on the results of the assessment. The April 2003 assessment found that all of SDD achieved software CMM level 2.

    Recommendation: After ensuring that its mission-critical systems are year 2000 compliant, but before investing in major software development efforts like ACE, the Commissioner of Customs should direct the Customs Chief Information Officer to ensure that process improvement activities are initiated for all ongoing essential software maintenance projects.

    Agency Affected: Department of Homeland Security: Directorate of Border and Transportation Security

    Status: Closed - Implemented

    Comments: Customs agreed with this recommendation. In response, Customs revised its system development life-cycle (SDLC) policy. The revised SDLC policy specifically addresses software maintenance projects and requires Capability Maturity Model (CMM) level 2 compliance for maintenance projects. Assessments conducted in November 2001, July 2002, and April 2003 have confirmed that maintenance projects have initiated and sustained software process improvement activities.

    Jun 10, 2014

    May 22, 2014

    May 12, 2014

    May 8, 2014

    May 7, 2014

    Apr 2, 2014

    Feb 26, 2014

    Feb 12, 2014

    Looking for more? Browse all our products here