This is the accessible text file for GAO report number GAO-03-367 
entitled 'Defense Transportation: Monitoring Costs and Benefits Needed 
While Implementing a New Program for Moving Household Goods' which was 
released on April 18, 2003.


This text file was formatted by the U.S. General Accounting Office 

(GAO) to be accessible to users with visual impairments, as part of a 

longer term project to improve GAO productsí accessibility. Every 

attempt has been made to maintain the structural and data integrity of 

the original printed product. Accessibility features, such as text 

descriptions of tables, consecutively numbered footnotes placed at the 

end of the file, and the text of agency comment letters, are provided 

but may not exactly duplicate the presentation or format of the printed 

version. The portable document format (PDF) file is an exact electronic 

replica of the printed version. We welcome your feedback. Please E-mail 

your comments regarding the contents or accessibility features of this 

document to Webmaster@gao.gov.



Report to Congressional Committees:



United States General Accounting Office:



GAO:



April 2003:



DEFENSE TRANSPORTATION:



Monitoring Costs and Benefits Needed While Implementing a New Program 

for Moving Household Goods:



GAO-03-367:



GAO Highlights:



Highlights of GAO-03-367, a report to Congressional Committees.



Why GAO Did This Study:



The Department of Defense (DOD) spends more than $1.7 billion each year 

to move and store over 600,000 household goods shipments when 

relocating military personnel. It conducted and evaluated several pilot 

program studies aimed at fixing its problem-plagued program and, in 

2002, issued a report to Congress with three recommendations. The 1997 

Defense Appropriations Act Conference Report directed GAO to validate 

the results achieved by the pilot programs. In response, GAO examined 

the extent to which DODís recommendations to Congress (1) offer 

solutions to long-standing problems in the current program and (2) are 

supported by the evaluationís findings and should be implemented. GAO 

also assessed the soundness of methodologies used by DOD to develop 

cost estimates to implement the recommendations.



What GAO Found:



The recommendations in DODís report to Congress have the potential to 

resolve several long-standing problems found in the current personal 

property program, which manages the transportation and storage of 

household goods. The recommendations, if implemented, would

* reengineer the claims process to reduce the length of time it 

currently takes to resolve claims for lost, destroyed, or damaged 

household goods and increase the reimbursement rates that military 

personnel currently receive for their losses;

* use performance-based service contracts to improve the generally low 

quality of service that DOD currently gets from the moving industry; 

and

* put in place new information technology with interface capabilities 

to enable program managers and users to monitor in-transit shipments 

and track the number and cost of shipments processed each year.



The recommendations in DODís report to Congress are supported by the 

Transportation Commandís evaluation of the pilot programsí findings and 

should be implemented within budget constraints. DODís approach in 

conducting the evaluation was methodologically sound: It developed an 

evaluation plan to guide its work and adjusted the plan when necessary 

to address differences in the pilot programsí approaches. While the 

shipments included in the evaluation do not represent all shipment 

types managed annually by DOD, GAO believes that the evaluation results 

provide sufficient information to allow DOD to initiate actions to 

improve its current personal property program. 



GAO found that the soundness of methodologies used to develop DODís 

cost estimates varied. Therefore, DODís ability to implement changes to 

the existing program within the cost estimates DOD reported to Congress 

is uncertain. GAO found that the estimated to implement the information 

technology recommendation was $7 million rather than the $4 million to 

$6 million estimate DOD reported to Congress. In developing cost 

estimates for the remaining recommendations, DOD did not provide the 

same level of evidentiary support for one of the three adjustments it 

used to align the pilot programsí costs to current program costs. As a 

result, GAO questions the extent to which these recommendations can be 

implemented within DODís estimated 13 percent increase over current 

program costs. While DOD believes it used a conservative approach in 

developing this 13 percent estimate, it has not quantified the risk 

associated with the projection, which could provide the military 

services and Congress information needed to develop and review future 

budget requests for this program. Without providing the range of 

possible cost increases and the risk regarding the likelihood of 

achieving this 13 percent projection within that range, DOD may find a 

repeat of what occurred during the pilots, where the military services 

terminated participation in one of the pilot programs due to costs 

exceeding projections. 



GAO also found that without carefully monitoring costs during the 

implementation phase and assessing costs and benefits from a period 

succeeding full implementation of the recommendations, DOD would not 

have the information needed to determine if anticipated improvements in 

the program are being achieved at a reasonable cost. 



What GAO Recommends:



GAO recommends that DOD implement the recommendations within budget 

constraints, quantify the risk associated with achieving its cost 

estimates, monitor costs during the implementation phase, and assess 

the new program to determine if anticipated improvements were achieved 

at a reasonable cost. DOD agreed with three recommendations, but did 

not agree with the need to quantify the risk associated with achieving 

its cost estimates.



www.gao.gov/cgi-bin/getrpt?GAO-03-367.



To view the full report, including the scope

and methodology, click on the link above.

For more information, contact William M. Solis at (202) 512-8365 or 

solisw@gao.gov.



[End of section]



Contents:



Letter:



Results in Brief:



Background:



DODís Recommendations Offer Solutions to Long-Standing Problems:



Transportation Commandís Evaluation of Pilot Programs Supports DODís 

Three Recommendations:



Ability to Implement New Program within Cost Estimates Reported to 

Congress Is Uncertain:



Conclusions:



Recommendations for Executive Action:



Agency Comments and Our Evaluation:



Appendix I: Scope and Methodology:



Appendix II: Overview of Current Personal Property Program 

and Pilot Programs:



Current Program:



Military Traffic Management Commandís Reengineered Personal Property 

Program:



The Department of Defenseís Full Service Moving Project:



Navyís Service Member Arranged Move Pilot Program:



Appendix III: Comments from the Department of Defense:



Related GAO Products:



Tables:



Table 1: Claims-Related Features of the Current Personal Property 

Program and Pilot Programs:



Table 2: Quality of Service-Related Features of the Current Personal 

Property Program and Pilot Programs:



Table 3: Data Reliability-Related Features of the Current Personal 

Property Program and Pilot Programs:



This is a work of the U.S. Government and is not subject to copyright 

protection in the United States. It may be reproduced and distributed 

in its entirety without further permission from GAO. It may contain 

copyrighted graphics, images or other materials. Permission from the 

copyright holder may be necessary should you wish to reproduce 

copyrighted materials separately from GAOís product.



United States General Accounting Office:



Washington, DC 20548:



April 18, 2003:



Congressional Committees:



Military personnel and their families can expect to relocate many times 

during a service memberís career. As the moving industryís single 

largest customer, the Department of Defense (DOD) spends more than $1.7 

billion annually for its personal property program, which provides 

household goods transportation and storage services for military 

personnel and their families when they relocate. The program manages 

more than 600,000 personal property shipments each year. DOD has 

experienced long-standing problems with its current personal property 

program, including excessive loss or damage to property, high claims 

costs incurred by the government, and poor quality of service from 

moving companies. Moreover, the programís data management system does 

not provide reliable information on the status of individual shipments 

or on the types of shipments and their costs.



In an effort to test alternative approaches and improve the quality of 

its personal property program, DOD has carried out several initiatives 

over the past 8 years. In 2000, the U.S. Transportation 

Command[Footnote 1] began to collect data from one of the three pilot 

programs to evaluate alternative approaches for improving the current 

program.[Footnote 2] The Transportation Command compared the features 

of the current program with those of the three pilot programs, and in 

June 2002 it submitted a report to the Deputy Under Secretary of 

Defense (Logistics) outlining its evaluation strategy, findings, and 

recommendations. In its November 12, 2002, report to Congress, DOD 

included the three recommendations resulting from the evaluation and 

the estimated additional costs required to implement program 

improvements based on a subsequent assessment of pilot and current 

programsí operations.



Our involvement in this issue stems from the conference report on the 

1997 Defense Appropriations Act, which directed us to review the data 

collected from the pilot programs and validate the results before DOD 

expands any of the pilot programs, and a subsequent request from the 

Subcommittee on Readiness of the House Committee on Armed Services. 

This report supplements our previous testimony[Footnote 3] on DODís 

efforts to enhance its current program, as well as our status 

report[Footnote 4] and status briefings to Congress on DODís efforts to 

implement a methodologically sound evaluation of the pilot programs. As 

agreed with your offices, for this report we assessed the extent to 

which the recommendations in DODís November 2002 report (1) offer 

solutions to long-standing problems in the personal property program 

and (2) are supported by the evaluationís findings and should be 

implemented. In addition, we assessed the soundness of the methodology 

used by DOD to develop the cost estimates for implementing the 

recommendations.



To conduct our assessment of DODís three recommendations to Congress, 

we reviewed DOD and GAO prior reports on the personal property program, 

interviewed DOD officials and private-sector contractors involved in 

the current and pilot personal property programs and the Transportation 

Commandís evaluation, examined the methodology and findings of the 

Transportation Commandís evaluation of the three pilot programs, 

reviewed the methodology and data generated by each of the three pilot 

programs, and assessed the methodologies used to develop the cost 

estimates for implementing DODís recommendations. We did not make an 

assessment of whether the anticipated benefits to be derived from 

implementing the three recommendations would warrant the additional 

costs DOD projects will be required to fund these improvements. The 

scope and methodology we used in our review are described in further 

detail in appendix I.



Results in Brief:



The three recommendations in DODís report to Congress offer solutions 

to several long-standing problems in the current personal property 

program. Specifically, the recommendations address previously 

identified problems with the liability/claims process (including the 

lengthy claims process, reimbursement rates that may not fully 

compensate service members for loss and damage incurred during their 

moves, and high claims costs to the government), the low quality of 

shipping services, and the inability to track shipments and their 

costs. If implemented, DODís first recommendation--to reengineer the 

claims process--has the potential to reduce the length of time it 

currently takes to resolve claims for lost, destroyed, or damaged 

household property and increase the reimbursement rates that military 

personnel currently receive for their losses. The recommendation also 

has the potential to reduce the high cost of claims incurred by DOD. 

The second recommendation--to use performance-based service contracts-

-has the potential to improve the generally low quality of service that 

DOD currently receives from moving companies. The third recommendation-

-to put in place new information technology with interface 

capabilities--has the potential to enable the personal property program 

managers and other users of the program to reliably monitor in-transit 

shipments and track the number and the cost of shipments processed each 

year so that accurate reporting can be provided to DOD and Congress.



The recommendations in DODís report to Congress are supported by the 

Transportation Commandís evaluation of the pilot programsí findings and 

should be implemented within budget constraints. We found that the 

Transportation Commandís approach in conducting the evaluation was 

methodologically sound: It developed an evaluation plan to guide its 

work and adjusted the plan when necessary to address differences in the 

pilot programsí approaches. While the shipments included in the 

evaluation do not represent all shipment types managed annually by DOD, 

we believe that the evaluation results provide sufficient information 

to allow DOD to initiate actions to improve its current personal 

property program.



Our review showed that the soundness of the methodologies used by DOD 

to develop cost estimates for implementing the three recommendations 

varied. Therefore, DODís ability to implement changes to the existing 

program within the cost estimates reported to Congress is uncertain. 

Our review found that the total initial cost for implementing the 

information technology improvements recommendation would more likely be 

$7 million rather than DODís $4 million to $6 million estimate. We 

agreed that the premise of two of three adjustments DOD used to develop 

the 13 percent cost increase to implement the claims process and 

performance-based service contract recommendations was sound. However, 

we are less assured about the extent to which the projected cost 

savings related to a third adjustment may occur because the adjustment 

was not supported by historical experience or by the same quality of 

data provided for the other adjustments. While DOD believes it can 

incorporate the three recommendations into a new program within its 

proposed 13 percent increase due to the conservative approach it took 

in developing this estimate, we believe that by quantifying the risk 

associated with this projection, DOD could provide the military 

services and Congress information needed to develop and review future 

budget requests for this program. The need for this type of information 

is further supported based on the long-standing problems associated 

with the current program and the large increase in costs contributing 

to the military servicesí decision to terminate participation in one of 

the pilot programs. Further, without carefully monitoring costs during 

the implementation process and assessing costs and benefits from a 

period succeeding full implementation of the recommendations, DOD will 

not have the information needed to determine if anticipated 

improvements in the program are being achieved at a reasonable cost. 

Currently, DOD is beginning planning efforts to implement the 

recommendations. These efforts do not include monitoring and evaluating 

costs and benefits during the implementation phase and post 

implementation of the recommendations in a new program.



We are recommending that DOD initiate actions that will implement the 

recommendations contained in its report to Congress within budget 

constraints, quantify the risk associated with achieving its cost 

estimates, monitor costs during the implementation phase to ensure that 

the proposed changes are being achieved within an acceptable and a 

predefined range, and assess the personal property program after the 

recommendations have been implemented to determine whether anticipated 

improvements are being achieved at a reasonable cost.



In commenting on a draft of this report, DOD concurred with three of 

our four recommendations, including initiating actions to implement the 

recommendations contained in its report to Congress, monitoring costs 

during the implementation phase, and assessing the program after DODís 

recommendations have been implemented. DOD partially concurred with the 

remaining recommendation, i.e., to provide the military services and 

Congress with additional information to quantify the risk associated 

with achieving the projected 13 percent cost estimate. In its response, 

DOD stated that the 5 percent reduction that it made to the pilot 

programsí average costs to adjust for economies of scale/program 

efficiencies was reasonable and very conservative. Further, DOD 

continues to believe that the program can be implemented within the 13 

percent increase and noted that one of the military services validated 

this estimate. Therefore, DOD did not see the value added in providing 

the military services or Congress a formal risk assessment. We find 

that these statements still do not provide a basis for the 5 percent 

reduction and do not indicate the level of risk associated with 

implementing the recommendation within this estimate. We continue to 

believe that this information needs to be developed to help the 

military services prepare their budgets. The military services 

terminated participation in one of the pilot programs because actual 

costs were significantly greater than the projections provided to them 

for budgetary purposes. Providing a measure of risk for the 13 percent 

projection could help prevent a repeat of what occurred during the 

pilot programs. Without this risk information, the military services 

will have to wait until after the moving industry submits bids to find 

out if DODís projection was reliable.



Background:



DODís personal property program is managed centrally by the Military 

Traffic Management Command headquarters and administered locally by 

about 200 military service and DOD transportation offices around the 

world. The program relies on more than 1,200 domestic commercial 

carriers and 150 freight forwarders for international shipments to 

provide household goods transportation and storage services for 

military personnel and their families when they relocate. The military 

services pay shipment and storage-related costs from their military 

personnel accounts and loss and damage claims and personal property 

shipment office expenses through their operations and maintenance 

accounts. The program has remained virtually unchanged for nearly 40 

years. It involves a complex process of qualifying carriers, soliciting 

rates, distributing moves, evaluating transportation providersí 

performance, paying invoices, and settling claims. Among the programís 

many challenges is ensuring that the moving industry provides adequate 

year-round capacity, especially during the summer peak-moving season 

when most service members, as well as the general public, schedule 

their moves.



In prior reports, both DOD and GAO have identified problems related to 

the loss and damage claims process and the low quality of service from 

movers. In designing and implementing its evaluation plan, the 

Transportation Command also noted that weaknesses in the current 

programís data management system precluded DOD from being able to track 

shipments in transit and from being able to extract reliable data on 

the number and types of shipments managed annually and their associated 

costs.



In response to the long-standing problems, DOD has undertaken a number 

of pilot program studies to find ways to improve the process of 

shipping service membersí household goods. In August 1996, the Deputy 

Under Secretary of Defense (Logistics) tasked the Transportation 

Command with evaluating alternative approaches to the current program 

and recommending changes in the program based on the results of its 

evaluation. The Transportation Command identified three ongoing or 

planned pilot programs to include in its evaluation and began to 

collect data for its analysis from one of them in 2000. These three 

pilot programs shared some common features, such as testing 

performance-based service contracts and providing full replacement 

(rather than depreciated) value for loss or damage. Each one also had 

some distinctive features, such as allowing service members to 

participate in selecting their movers and contracting out installation 

personal property shipment office functions to private-sector move 

manager companies.



The three pilot programs are summarized below. Further information on 

each program, as well as DODís current personal property program, is 

provided in appendix II.



* The Military Traffic Management Commandís Reengineered Personal 

Property Program operated from the military servicesí and the Coast 

Guardís installations located in North Carolina, South Carolina, and 

Florida. It used military installation personal property shipment 

office personnel, as the current program does, and developed a new data 

management system that tracked both the movement of individual 

shipments and information on the number and cost of shipments.



* The Department of Defenseís Full Service Moving Project operated from 

the military servicesí and the Coast Guardís installations located in 

the National Capital Region (the Washington, D.C., metropolitan area), 

Georgia, and North Dakota. It contracted the management of shipments to 

private-sector companies and offered optional relocation services, such 

as referrals for rental assistance and purchase and sale of real estate 

services, to personnel participating in the pilot program.



* The Navyís Service Member Arranged Move Pilot Program operated from 

Navy installations located in the states of California, Connecticut, 

Virginia, and Washington. It designated current staff within the 

installation personal property shipment offices as ďmove coordinatorsĒ 

to provide assistance, allowed participants to pre-select 

transportation providers, and paid for moves through government 

purchase cards.



In June 2002, the Transportation Command submitted a report containing 

its evaluation results and proposed three recommendations to the Deputy 

Under Secretary of Defense (Logistics). After reviewing the results and 

receiving comments from the military services, DOD submitted its 

report, dated November 12, 2002, to Congress. DODís report contained 

the same three recommendations contained in the Transportation 

Commandís report. DOD also provided cost estimates for implementing the 

recommendations. The three recommendations were to:



* reengineer the liability/claims process by adopting commercial 

practices of minimum valuation, simplifying the filing of claims, and 

providing direct settlement with the carrier;



* change the acquisition process to implement performance-based service 

contracts; and:



* implement information technology improvements, which could interface 

functions across such areas as personnel, transportation, financial, 

and claims.



In its report to Congress, DOD estimated that reengineering the 

liability/claims process and changing the acquisition process to 

implement performance-based service contracts would increase the 

current programís estimated $1.7 billion cost by 13 percent. 

Implementing the information technology improvements to enhance its 

data management capabilities and to provide training to users was 

estimated at an additional $4 million to $6 million.



DOD also estimated that efforts to implement the changes to the current 

program would be completed by the first quarter of fiscal year 2005. 

DOD has developed a plan of action and milestones for designing the new 

personal property program. This initial effort identifies several 

teams, which are exploring the following issues: the acquisition/

solicitation process, quality assurance, the liability/claims process, 

information systems technology, and electronic billing and payment. 

Four of these issues address the recommendations included in DODís 

November 2002 report to Congress. The plan identifies a list of 

essential activities needed to carry out the responsibilities required 

to build the future personal property program. It also includes time 

lines and identifies a process to monitor problems and delays. However, 

it does not include monitoring costs and benefits during the 

implementation phase and the extent the proposed changes are being 

achieved within an acceptable and a predefined range. Further, it does 

not include evaluating the extent the benefits from the pilot programs 

will be achieved after the new program is implemented to determine 

whether the anticipated improvements were achieved at a reasonable 

cost.



DODís Recommendations Offer Solutions to Long-Standing Problems:



The three recommendations in DODís report to Congress offer solutions 

to several of the current programís long-standing problems, such as the 

liability/claims process and the low quality of service. These problems 

have been identified in DOD and GAOís prior reports, as well as in 

surveys conducted as part of the pilot program evaluation. The 

inability to monitor shipments and shipping information has been long 

recognized and was highlighted as an additional problem during DODís 

evaluation. If implemented, the first recommendation (i.e., 

reengineering the liability/claims process by adopting commercial 

practices of minimum valuation, simplifying the filing of claims, and 

providing direct settlement with the carrier) has the potential to help 

reduce the length of time it currently takes to resolve claims for 

lost, destroyed, or damaged household goods because the carrier 

recovery time would be eliminated for most moves, increase the 

reimbursement rates military personnel receive for their losses, and 

reduce DODís claims-related costs. The second recommendation (i.e., 

changing the acquisition process to implement performance-based service 

contracts) has the potential to help improve the generally low quality 

of service that DOD currently receives from the moving industry. The 

third recommendation (i.e., implementing information technology 

improvements, which could interface functions across areas such as 

personnel, transportation, financial, and claims) has the potential to 

improve the programís ability to reliably monitor and collect data on 

the status and costs of shipments so that accurate reporting can be 

provided to DOD and Congress.



Reengineered Liability/Claims Process Has the Potential to Shorten 

Time, Increase Reimbursement Rates, and Reduce Claims-Related Costs to 

DOD:



As part of its evaluation, the Transportation Command cited that one of 

the long-standing problems with military household goods shipments is 

the liability/claims process, including the (1) length of time it takes 

to resolve claims, (2) low reimbursement rates, and (3) high cost of 

claims that DOD must pay.



Length of Time to Resolve Claims:



In a study conducted in 1999, the Military Traffic Management Command 

reported that 146 days are expended between the time a claim is filed 

by a service member to recovery of costs from the carrier by the 

government. During this period, military personnel file their claims 

for lost, destroyed, or damaged household goods with their respective 

military serviceís claims offices and receive settlements (this occurs, 

on average, within 23 days), and then these offices file the claims 

against the carriers to recover the costs (this step is completed 

within the 146 day period).[Footnote 5] In the Transportation Commandís 

pre-evaluation survey completed in 2000, responses from military 

personnel who had recently moved indicated that one of the lowest 

performance ratings involved the time required to settle a claim.



Based on the Transportation Commandís evaluation of the claims process 

under the pilot programs, one of the results from implementing the 

pilot programs was the 146 day average required under the current 

program to settle claims and recover costs was reduced to an average of 

30 days since the service member filed directly with the carrier and 

the military services did not have to recover costs. Under each pilot 

program, military personnel settled claims directly with the carriers. 

Service members who were not satisfied with offers made by the carriers 

could file their unresolved claims directly with DOD. Military services 

worked these claims with the carriers and if a claim was justified, the 

service member received just settlement under the pilot programs (i.e., 

if the items were lost or destroyed, the member received full 

replacement value, while damaged items were repaired).



Low Reimbursement Rates for Loss and Damage:



In its pre-evaluation survey, the Transportation Command found that 

military personnel cited low reimbursement amounts that typically do 

not cover the loss or damage of household goods as a major concern 

during their moves. Under the current program, a carrierís liability is 

limited to $1.25 per pound multiplied by the shipment weight. Personnel 

receive only the depreciated value of lost, destroyed, or damaged 

items, up to a maximum of $40,000 per move.[Footnote 6] When arranging 

their moves under the current program, military personnel can buy 

increased insurance coverage from their carrier, up to a full 

replacement value limit of $3.50 per pound times the shipment weight, 

at a cost of 85 cents per $100 of the stated value of the shipment. 

However, only military personnel making moves within the continental 

United States can buy this additional coverage; it is not available to 

those moving to or from overseas posts. Another option that military 

personnel have to increase their protection for loss and damage is to 

buy additional coverage from private-sector sources.



Unlike the current program, the pilot programs provided full 

replacement value for lost and destroyed goods, with maximum amounts 

ranging from $63,000 to $75,000. Damaged items were repaired. Two of 

the three pilot programs reported that their cost per pound times the 

shipment weight rates were $3.50 for $63,000 maximum coverage and $6.00 

for $75,000 maximum coverage, respectively. The remaining pilot program 

did not give a cost per pound, stating only that its maximum coverage 

rate was $72,000.



High Costs to Government for Claims:



DOD has reported that, historically, approximately 35 percent of all 

moves result in loss or damage claims. A 1997 Military Traffic 

Management Command survey of 3,000 moves revealed that while 65 percent 

of shipments had loss or damage, only 35 percent resulted in claims 

being filed. DOD pays approximately $100 million a year in claims but 

recovers only 60 to 65 percent of the amounts paid to military 

personnel from the moving industry. These figures understate the actual 

loss and damage, since all military personnel do not file claims, 

apparently because the process takes a long time and reimbursement 

rates do not always cover the losses. DOD incurs these losses due to 

the structure of its current program. The military claims offices 

assist service members by arranging to pay their claims and then 

submitting the claims to the respective movers for reimbursement. As 

indicated above, DOD receives only partial reimbursement from the 

moving industry.



If the recommendation is implemented, DOD expects to reduce a 

substantial portion of the estimated $100 million it currently pays in 

claims each year to service members and eliminate much of the 35 to 40 

percent in losses it incurs from settling claims with the moving 

industry because service members will be settling claims directly with 

their carriers. DOD also expects additional savings because fewer 

demands would be placed on military claims officials to manage the 

claims process. DOD believes that these savings will help offset the 

higher costs of providing full replacement value to service members for 

any loss and damage incurred during the shipment and storage of their 

personal property.



Change to Performance-Based Service Contracts Could Improve Quality of 

Moving Services:



Our work has shown that another long-standing problem with the current 

personal property program is the poor quality of moving services 

provided to military personnel. The high number of loss and damage 

claims that military personnel file underscores this problem. According 

to the two pre-evaluation surveys cited in the Transportation Commandís 

evaluation, around 55 to 65 percent of respondents reported suffering 

some loss or damage of household goods during a recent move. Moreover, 

in the 

pre-evaluation survey conducted by the Transportation Command, the top 

four factors identified by service members as being of greatest 

importance to them in the moving process were the quality of packing, 

the care in handling personal property, the condition of their property 

upon receipt at the end of the move, and the receipt of fair payment 

for any losses or damages they suffered. In individual comments 

obtained during the 

pre-evaluation survey, the Transportation Command reported that some 

service members also cited the lack of professionalism and quality of 

customer service on behalf of moving crews as a concern.



The problem stems primarily from the current program placing greater 

emphasis on costs (i.e., the lowest bids) than on the quality of 

service that carriers provide when moving shipments of military 

household goods. While the current program established its Total 

Quality Assurance Program to measure quality, data collected to develop 

scores for each carrier includes three measures (timeliness of pickup, 

timeliness of delivery, and reported loss and damage), which are not 

collected for all household goods shipments. The best indication of 

quality, customer satisfaction, is not measured in the current program. 

The problem of quality is further exacerbated by the programís use of a 

20 year-old tariff schedule that carriers use in developing their bids. 

This tariff contains lower rates than the current commercial tariff 

used during the pilot programs.



Unlike the current program, the pilot programs screened carriers that 

wanted to participate in their programs by emphasizing the quality of 

carriersí prior performance rather than the amount of their bids. For 

example, the Full Service Moving Project contracted a financial 

services company to conduct a financial and performance assessment of 

potential movers. The pilot program emphasized best value and placed 

more emphasis on performance (70 percent) than cost (30 percent) in 

determining which providers were awarded shipments. The pilot programs 

showed that these types of contracts could allow the government to 

pre-screen carriers for financial viability and, more importantly, to 

institute and maintain a quality assurance process to reduce losses and 

improve service.



In addition to prescreening carriers for quality control purposes, the 

pilot programs also surveyed military personnel who participated in the 

programs and used the results to distribute future shipments to 

carriers that received the best performance scores. To address concerns 

about the obsolete tariff schedule, the pilot programs adopted current 

commercial tariffs for carriers to use in establishing their bids.



The pilot programs also showed that the solicitation process could be 

streamlined by eliminating detailed statements of work and that the 

pilot programs could place responsibility for successful performance on 

carriers, allowing the government to focus on outcomes, rather than 

processes. Finally, the pilot programs demonstrated that using these 

types of performance-based service contracts did not have an adverse 

effect on small business participation, a major concern of the moving 

industry. On the basis of the total dollar value of shipments, each 

pilot program exceeded the Small Business Administrationís goal of 23 

percent participation for the industry. Specifically, 48 percent of the 

Military Traffic Management Commandís Reengineered Personal Property 

Programís revenues, 74 percent of the Full Service Moving Projectís 

revenues, and 100 percent of the Navyís Service Member Arranged Moveís 

revenues went to small businesses.



Implementing Information Technology Improvements to Address Data 

Reliability Problems:



Another ongoing problem with the current personal property program is 

its inability to provide reliable data on the status of in-transit 

shipments or on the number and associated costs of shipments managed by 

DOD each year. Because of the lack of reliable data on shipments and 

costs, program managers have no way of knowing the actual costs of 

moving military personnelís household goods. In addition, they have no 

access to real-time tracking data that they could use to manage 

transportation and storage costs and to help cut down on the need for 

temporary storage by reducing the number of failed deliveries.



Two of the pilot programs included features to address the problems 

associated with the current programís stand-alone data management 

system. The pilot programs each developed a Web-based data management 

system to enhance the visibility of individual shipments and provide 

more reliable data on shipments and costs. For example, the Military 

Traffic Management Commandís Reengineered Personal Property Programís 

data management system provided in-transit visibility. This made it 

possible to track the status of individual shipments and gave real-time 

access to those sections of the shipment records that various parties 

involved in the relocation process needed for data entry or status 

review. The pilot programís data management system provided a complete 

picture of the service memberís move from start (the move application 

process) to finish (the claims submission and resolution process). In 

addition, the data management system demonstrated the potential to 

provide information to personnel in various functional areas involved 

in the service membersí relocation process (such as personnel, 

transportation, financial, and claims). Finally, the data management 

system demonstrated the potential to provide data for planning and 

budgeting purposes on the types of shipments made annually across DOD 

and their costs. The Full Service Moving Projectís data management 

system was developed but not fully implemented because the military 

services terminated their participation in the pilot program due to its 

high costs. While the Navyís pilot program developed a database near 

the end of the Transportation Commandís evaluation, the database was 

not fully implemented nor assessed as part of the evaluation.



Transportation Commandís Evaluation of Pilot Programs Supports DODís 

Three Recommendations:



Our analysis indicated that DODís three recommendations are supported 

by the results of the Transportation Commandís evaluation of the three 

pilot programs. The Transportation Command adopted a sound methodology 

to conduct its evaluation, and it adjusted this methodology when 

circumstances warranted. The results of the Transportation Commandís 

evaluation are based on data collected from a limited number of 

geographical areas. While the shipments included in the evaluation do 

not represent all shipment types managed annually by DOD, we believe 

that the evaluation results provide sufficient information to allow DOD 

to initiate actions to improve its current personal property program 

within budget constraints.



Transportation Command Implemented a Methodologically Sound 

Evaluation:



We found that the Transportation Command used a methodologically sound 

approach to evaluate the results of the three pilot programs and make 

its recommendations. Before it started the evaluation process, the 

Transportation Command considered some lessons learned that had emerged 

from our review of the Hunter Pilot Program in 1999,[Footnote 7] and it 

followed through with several of them. For example, it obtained 

assistance from a contractor to design an evaluation plan that met 

professional standards. The Transportation Command identified four 

aspects, or factors, of the property program that served as the focus 

of its evaluation (i.e., quality of life, total costs, small business 

participation, and process improvements). The evaluation plan also 

prescribed that only one quality of life survey be administered to each 

participating service member in order to avoid survey ďfatigueĒ that 

can result from subjecting a person to multiple surveys, and thus avoid 

the resulting potential for questionable results. In designing the 

evaluation plan, the Transportation Command incorporated a number of 

important evaluation features. These features included assessing the 

four factors consistently across all three pilot programs, ensuring 

that the evaluation received data from the pilot programs during the 

same time period to avoid the need to make adjustments due to potential 

changes in carrier operations and costs, conducting a survey of service 

members using the current program to establish a baseline from which to 

measure the pilot program results, and developing a method to provide 

estimates of what DOD would have paid for comparable shipments under 

the current program for those shipments completed under the pilot 

programs.



The Transportation Command made appropriate adjustments to the 

evaluation plan when it learned that the three pilot programs would not 

be underway at the same time and that they would not provide all of the 

information originally outlined in the plan. For example, the Full 

Service Moving Project began later and terminated earlier than was 

expected, the Navyís Service Member Arranged Move Pilot Program failed 

to conduct a quality of life survey and collect cost data as outlined 

in the evaluation plan, and none of the three pilot programs provided 

costs associated with individual process improvements. The 

Transportation Command included qualitative analytical techniques so 

that it could include as much information on each pilot program as 

possible in its evaluation while also dealing appropriately with data 

limitations. The Transportation Command also shifted the evaluation 

focus from the individual pilot programs to specific features from the 

three programs, such as full replacement value for loss and damage and 

the screening process for carrier participation.



Survey and Analytical Data Support Commandís Recommendations:



Our work indicated that the Transportation Commandís analysis of data 

collected from the three pilot programs supports the three 

recommendations that DOD included in its report to Congress. The 

Transportation Commandís analysis of household goods shipment data from 

the pilot programs showed that the average amount of time that service 

members and DOD spend to settle claims and recover costs from carriers 

fell dramatically in all three pilot programs. In comparison with the 

current programís 146-day average, it took only 30 days, on average, to 

settle a claim under the Reengineered Personal Property Program and the 

Full Service Moving Project and fewer than 14 days under the Navyís 

program. Survey results indicated that full replacement (rather than 

depreciated) value, direct claims settlements, and anticipated 

improvements in the claims process accounted for the highest increases 

in satisfaction. Based on experiences during the pilot programs, DOD 

believes that direct claims settlement between service members and 

carriers should reduce claims costs DOD currently incurs. Under the 

current program, DOD must collect from the carriers after it has paid 

the service membersí claims. DOD expects that this step will be 

eliminated in most instances because it is anticipated that service 

members will be resolving most of their claims directly with their 

carriers.



The Transportation Commandís analysis of process improvement data, 

interviews and observations during site visits, and survey results from 

the pilot programs supported DODís recommendation to use performance-

based service contracts to improve the quality of services that the 

moving industry provides to the military. The process of prescreening 

carriers desiring to participate in the pilot programs on the basis of 

their financial viability and past performance helped to eliminate poor 

performers. Furthermore, the pilot programsí use of post-move surveys 

allowed them to get immediate and continuous feedback on the carriersí 

performance and to use this information to distribute future work to 

those carriers with the highest performance ratings and best value. In 

addition, two of the pilot programs reduced the amount of paperwork 

associated with soliciting proposals and approving carriers.



Finally, the Transportation Commandís review and observations of two of 

the pilot programsí Web-based data management systems supported DODís 

recommendation to overhaul the current personal property programís 

computer system (the Transportation Operational Personal Property 

Standard System). The Transportation Command found that the 

Reengineered Personal Property Programís data management system 

significantly improved communications between the various DOD offices 

and the moving industry. The system gave real-time access to shipment 

records to DODís personal property shipment offices, certifying 

officers, prepayment auditors, military service headquarters, and 

military service claims offices and finance centers, as well as moving 

industry participants. Similar results occurred with the Full Service 

Moving Projectís Best Value Distribution Database system, but the 

military services terminated their participation in this pilot program 

before the systemís full potential could be demonstrated.



While the shipments included in the evaluation do not represent all the 

shipment types managed annually by DOD, we believe that the evaluation 

results provide sufficient information to allow DOD to initiate actions 

to improve its current personal property program.



Ability to Implement New Program within Cost Estimates Reported to 

Congress Is Uncertain:



Our review found that the estimates DOD reported to Congress might 

understate the total initial cost for implementing the information 

technology improvements recommendation and contain a questionable 

adjustment for costs associated with the claims and contracting process 

recommendations. Also, DOD did not quantify the risk associated with 

implementing these latter recommendations within its projected 13 

percent increase over the current programís cost. Therefore, the 

ability to implement changes to the existing program within the cost 

estimates reported to Congress is uncertain.



Based on our discussions with Military Traffic Management Command 

officials and review of available documents, we concluded that the 

total initial cost to implement the information technology improvements 

recommendation will more likely be $7 million rather than the $4 

million to $6 million estimate that DOD previously reported to 

Congress. In its response to a draft of this report, DOD maintained 

that the costs to implement a new Web-based data management system 

would fall within its initial cost estimate of $4 million to $6 

million. DODís projected cost estimate includes $5 million for 

development and implementation of the new system and $500,000 each for 

user training and system verification and validation testing. At a 

minimum, based on these projected cost estimates, the initial cost to 

implement the information technology improvements recommendation would 

more likely be $6 million.



While we concur with the premise of two of the three adjustments used 

to develop the 13 percent cost increase to implement the remaining 

recommendations, we are less assured in the extent to which the 

projected savings related to the third adjustment may occur. We found 

that the first two adjustments were based on historical data. However, 

we question the rationale DOD used to develop the third adjustment, as 

the savings associated with this adjustment are based on assumed cost 

reductions resulting from changes in program operations. Also, these 

reductions lack the same quality of evidentiary support as DOD provided 

for the other two adjustments.



DOD believes it took a conservative approach in developing the savings 

in each of the three adjustments; therefore, it assumes that the 

proposed changes to claims and the contracting process can be achieved 

within the 13 percent increase over the current programís costs. Due to 

the long-standing problems with this program and the high pilot program 

costs that contributed to the military servicesí early termination of 

participating in one of the pilot programs, we believe that by 

quantifying the risk associated with this projection, DOD could provide 

the military services and Congress information needed to develop and 

review future budget requests for this program. Further, without 

carefully monitoring costs during the implementation process and 

assessing costs and benefits from a period succeeding full 

implementation of the recommendations, DOD will not have the 

information needed to determine if anticipated improvements in the 

program are being achieved at a reasonable cost. Currently, DOD is 

beginning planning efforts to implement the recommendations. These 

efforts do not include monitoring and evaluating costs and benefits 

during the implementation phase and post implementation of the 

recommendations in a new program.



Costs to Implement Information Technology Improvements Vary:



The information DOD has provided on costs to implement the information 

technology improvements recommendation varies. Information provided 

during our review indicated that the total initial cost to improve the 

current data management system would be higher than the $4 million to 

$6 million DOD included in its report to Congress. DOD worked with the 

contractor who developed the Reengineered Personal Property Programís 

Web-based data management system to develop an estimate of the cost to 

expand the capabilities tested during the pilot program. Also included 

in this estimate were funds to provide training for users of the new 

system. Based on our discussions with officials from the Military 

Traffic Management Command and our review of available documents, we 

concluded that these costs would more likely be $6 million, as the data 

management system development cost was projected to be $5 million with 

an additional $1 million for user training. The need for this training 

as part of a new personal property program was identified during DODís 

evaluation of the pilot programs. We increased our overall projections 

for the cost of the new system to $7 million when we learned that DOD 

planned to continue spending at least another $1 million annually for 

independent verification and validation testing and contractor support. 

This latter expense was identified to us during discussions following 

DODís submission of its report to Congress.



In its response to a draft of this report, DOD maintained that the 

costs to implement a new Web-based data management system would fall 

within its initial cost estimate of $4 million to $6 million. It 

projected a cost of $5 million for system development and 

implementation and $500,000 each for user training and initial system 

validation. At a minimum, based on these projected cost estimates, the 

initial cost to implement the information technology improvements 

recommendation would more likely be $6 million. Because we did not 

assess the sufficiency of DODís original estimates of $1 million each 

for training and validation testing, we are unable to assess the impact 

of the reduction on the improvements in information technology across 

DOD.



Based on our discussion with DOD officials, we learned that the plan is 

to implement this recommendation regardless of the status of the other 

two recommendations because managers and users of the program need more 

reliable information to manage the programís shipments and their costs. 

Funds to implement this recommendation would come from the military 

servicesí operations and maintenance accounts.



Soundness of Estimates to Develop Cost for Changing Claims and 

Contracting Processes Varies:



The soundness of the three adjustments the Military Traffic Management 

Command used to develop its estimated 13 percent increase over the 

current program costs to implement the remaining recommendations--the 

claims process and performance-based service contracts--varies. We 

found that two of these adjustments are based on reasonable assumptions 

and are supported by historical experience and by data. The savings 

associated with the third adjustment are based on assumed cost 

reductions resulting from changes in program operations and lack the 

same quality of evidentiary support as DOD provided for the other two 

adjustments. Therefore, we are less assured in the extent to which the 

savings associated with this adjustment may occur. Finally, we found 

that in its report to Congress, DOD did not quantify the risk of 

achieving these recommendations within the projected 13 percent 

increase. This information is important to the military services as 

they develop their military personnel and operations and maintenance 

budget requests and to Congress as it assesses the reasonableness of 

these requests.



Estimate Includes a Questionable Cost Adjustment:



In developing the 13 percent estimate, the Military Traffic Management 

Command determined that three adjustments to the average costs for the 

pilot programs were required to develop the cost for the full rollout 

of a new personal property program. The first two adjustments (i.e., 

reducing the average weight of shipments and reducing costs to adjust 

for a mix of small and large businesses) were made to offset 

differences between the pilot programsí shipments and those more 

typically managed across DOD. The third adjustment was made to reduce 

the pilot programsí costs to reflect anticipated savings based on 

economies of scale.[Footnote 8] In developing these adjustments, the 

Military Traffic Management Command worked with a contractor and 

consulted with officials from the military services and moving industry 

associations.



While we believe that the shipment weight and small business mix 

adjustments are reasonable, we question the extent to which the 

economies of scale or program efficiencies adjustment may be achieved. 

For the weight adjustment, the Military Traffic Management Command 

determined that the average weights of moves in the two pilot program 

areas were higher than those experienced in typical departmentwide 

moves. As a result, the Military Traffic Management Command reduced the 

pilot programsí average weights to reflect the lower, more typical 

weights to be used in calculating a total cost for a departmentwide 

program. This adjustment resulted in a 12 percent drop in average 

costs. We found the approach of using historical data to more 

accurately reflect the typical shipment weights to be reasonable.



Next, the Military Traffic Management Command further lowered the pilot 

programsí average costs because the pilot programs had higher small 

business participation rates than the departmentwide average, and small 

businesses are typically more expensive than large businesses. Small 

businesses accounted for 48 percent of the cost of all moves under the 

Reengineered Personal Property Program and 73 percent under the Full 

Service Moving Project. In addition, small businesses were 14 percent 

more expensive per shipment in the Reengineered Personal Property 

Program and 74 percent more expensive in the Full Service Moving 

Project than what each pilot program paid to large businesses. In 

developing its departmentwide estimate, the Military Traffic Management 

Command used a small business participation target rate of 30 percent. 

This 30 percent target rate is higher than the Small Business 

Administrationís 23 percent goal for government agencies conducting 

business with this industry. On the basis of this lower participation 

rate, the Military Traffic Management Command reduced the pilot 

programsí average costs further by 8 percent. We agree that this 

adjustment in costs based on differences in the pilot programsí small 

business participation rate and the new 30 percent goal is a reasonable 

way to reflect the differences between the pilot programsí costs and 

the departmentwide-projected costs.



We found that the third adjustment that the Military Traffic Management 

Command made--to reduce the cost of departmentwide shipments because of 

economies of scale or program efficiencies--was not adequately 

supported based on either historical experience or data that DOD later 

provided. The Military Traffic Management Command reduced the pilot 

programsí average costs by 5 percent on the assumption that:



* the pilot programsí shipments involved only a limited number of 

providers;

:



* the pilot programs only included a limited number of shipments while 

the current program manages over 600,000 shipments annually;

:



* more accurate and timely management data that includes service member 

counseling, reduced losses, and storage and indirect costs will result 

in a more efficient program; and 

:



* overhead and operating costs will be spread due to a larger volume of 

shipments.

:



While recognizing that some changes may result from these anticipated 

program efficiencies, the effect of these changes on potential cost 

savings is uncertain at this time. The Military Traffic Management 

Command did not provide the same level of evidentiary support that it 

provided on the other two adjustments. Further, we believe that only 

time will determine if DODís assumption for this adjustment, in 

particular, proves to be correct.



DOD Did Not Quantify the Risk for Its Cost Estimate:



We found that DOD has not provided a level of assurance to the military 

services and Congress that its projected 13 percent increase over the 

current programís cost can be achieved. Quantifying the risk associated 

with this projection could provide the military services assurance of 

the viability of the projected 13 percent increase as they prepare 

budgets to support the increased cost for this program. Congress could 

also use this information as it reviews DODís requests for additional 

funds to implement changes in this program. The need for this type of 

information is further supported based on the long-standing problems 

associated with the current program and the fact that shipment and 

storage costs under the pilot programs were significantly higher than 

those that DOD estimated it would have paid under its current program 

in the same geographical areas. These costs ranged from 31 to 32 

percent higher under the Reengineered Personal Property Program and 

from 51 to 54 percent higher under the Full Service Moving 

Project.[Footnote 9] These higher-than-anticipated costs contributed 

to the military servicesí decision to terminate their participation in 

the Full Service Moving Project before its test period ended.



While DOD did not quantify the risk, per se, it believes a conservative 

approach was taken in developing the savings in each of the three 

adjustments. As a result, DOD assumes that the proposed changes to the 

claims and contracting processes can be achieved with its projected 

increase of 13 percent over the current programís budget. We still 

believe that the Military Traffic Management Command could have 

quantified the risk and provided this additional information to the 

military services and Congress as additional assurance of the 

likelihood of implementing the two recommendations within its projected 

13 percent increase. The need for this information is further supported 

based on the long-standing problems DOD has experienced in this 

program, the fact that the military services terminated participation 

in one of the pilot programs due to the high cost increases, and the 

need to determine whether the proposed additional funds from military 

personnel and operations and maintenance accounts will be sufficient to 

implement the recommendations.



In addition to the information that could be gained from quantifying 

the risk of its cost projection, we believe that only by careful 

monitoring during the implementation phase will DOD be able to ensure 

that the proposed changes are being achieved within an acceptable and a 

predefined range. Further, while we believe that the evaluation results 

support implementing plans to enhance the current program, it should be 

noted that the pilot programsí shipments included in the evaluation 

were not typical of all types of shipments managed annually. Therefore, 

DOD was precluded from projecting the extent to which the recommended 

improvements can be achieved DOD-wide. Unless a subsequent evaluation 

is undertaken after the recommendations have been implemented, DOD will 

not be able to assess the extent to which the projected benefits are 

being achieved for military personnel, their families, and DOD, and 

whether the benefits are being achieved at a reasonable cost. Selecting 

an evaluation period to include the peak-moving season would also 

provide DOD with the information its needs to determine if the proposed 

changes can be achieved during the summer, when the demand for moving 

services by DOD and the private sector is at its highest.



Conclusions:



The three recommendations DOD developed from its evaluation of the 

current and pilot programs, if implemented successfully, could enhance 

the quality of life for relocating service members and their families; 

reduce claims-related costs to DOD; and resolve problems related to the 

reliability of management information on the status of shipments and on 

the quantity, types, and costs of shipments that DOD and the military 

services manage annually. Delaying implementation of the 

recommendations only prolongs problems military personnel, their 

families, and DOD experience under the current program.



DOD has not quantified the risk associated with achieving its projected 

13 percent increase over the current programís costs to implement the 

claims process and performance-based service contract recommendations. 

Without quantifying the risk, the military services and Congress cannot 

be assured that these recommendations can be achieved within this 

estimate or whether additional funding or trade-offs may be needed. 

Further, without careful monitoring during the implementation phase, 

DOD will not be able to ensure that the proposed changes are being 

achieved within an acceptable and a predefined range.



Because the pilot programsí shipments included in the evaluation were 

not typical of all types of shipments managed annually, it was not 

possible for DOD to project the extent to which the recommended 

improvements can be achieved departmentwide. Without evaluating the 

program following implementation of the recommendations, DOD will be 

unable to assess the extent to which the projected benefits for 

military personnel, their families, and DOD are being achieved and, if 

so, whether they are being achieved within a reasonable cost. Also, if 

DOD does not select an evaluation period that includes the peak-moving 

season, it will not have the information needed to determine if the 

proposed changes can be achieved in the summer, when the demand for 

moving services is at its highest.



Recommendations for Executive Action:



To improve the personal property program for military personnel, their 

families, and program administrators, we recommend that the Secretary 

of Defense direct the Commander, U.S. Transportation Command, to:



* initiate actions to implement the three recommendations contained in 

DODís report to Congress within budget constraints, 

:



* provide the military services and Congress additional information to 

quantify the risk associated with achieving the projected 13 percent 

cost estimate before the claims process and performance-based service 

contracts recommendations are implemented to provide the military 

services with information needed for budgeting purposes,

:



* monitor costs for all recommendations during the implementation phase 

to ensure that the proposed changes are being achieved within an 

acceptable and a predefined range, and

:



* assess the effects of the three recommendations on the personal 

property program (to be carried out after the summertime peak-moving 

season once the recommendations have been implemented) to determine 

whether the anticipated improvements in the program are being achieved 

at a reasonable cost.



Agency Comments and Our Evaluation:



In commenting on a draft of this report, DOD concurred with three of 

our four recommendations. For the first of these recommendations, DOD 

stated that it is developing a plan to implement those recommendations 

it reported to Congress and anticipates its recommendations will be 

implemented by the end of the first quarter of fiscal year 2006, 

assuming the military services receive the additional funds needed to 

fund program enhancements. In response to our recommendation to monitor 

costs during the implementation phase, DOD stated that rolling out the 

new program will require monitoring of costs to determine if the moving 

industry partners are submitting bids that will allow DOD to enhance 

this program within the projected 13 percent cost increase. Further, 

DOD plans to include a process to conduct a rate reasonableness 

analysis upon receipt of the rates. For rates found to be outside the 

range of reasonableness, carriers will be given one opportunity to 

resubmit their rates. DOD plans to only use those rates determined to 

be reasonable in the new program. DOD also plans to include metrics, 

target/benchmark performance indicators, and a methodology for data 

collection in an updated program of action and milestone plan. For our 

recommendation, i.e., assess the effects of the three DOD 

recommendations on the personal property program to determine whether 

the anticipated improvements in the program are being achieved at a 

reasonable cost, DOD plans to collect data needed to determine if 

anticipated improvements have been achieved on a continuing basis. DOD 

plans to use customer satisfaction surveys in developing carrier 

performance ratings, which will be established quarterly, with the 

exception of the peak season, when performance ratings will be 

established monthly. If properly implemented, we believe the proposed 

DOD actions will sufficiently address these recommendations.



DOD partially concurred with the remaining recommendation, i.e., 

provide the military services and Congress with additional information 

to quantify the risk associated with achieving the projected 13 percent 

cost estimate to provide the military services with information needed 

for budgeting purposes. DOD continues to believe that the 5 percent 

reduction it made to pilot programsí average costs to adjust for 

economies of scale/program efficiencies was reasonable and very 

conservative and that the program can be implemented within the 

projected 13 percent increase over current program costs. DOD also 

reported that one of the military services validated the 13 percent 

cost increase following our audit. Further, DOD stated that it did not 

see value added in providing the military services or Congress a formal 

risk assessment but will continue to work with the military services as 

execution progresses to make sure they have all information required 

for budget purposes. Additionally, while not part of this 

recommendation, DOD also said it did not concur with our finding that 

the cost estimate for implementing its information technology 

improvements recommendation would be $7 million.



In reviewing the response, we found that DOD still did not provide any 

data to support its assumption of a 5 percent cost savings from 

economies of scale/program efficiencies. DOD stated that the new 

program will be about 200 times larger than the pilot programs and that 

the resulting increase in volume will lower the cost per unit, a 

standard and accepted law of economics. While we agree that the cost 

may decrease, it may also increase or remain unchanged. Moreover, the 

cost may decrease by less than 5 percent. Without specific data showing 

the per move costs will decrease as the scale of operations increase, 

we continue to question the basis for DODís assumption of a 5 percent 

reduction.



We believe that the validation effort completed by one of the military 

services, along with the calculations and assumptions DOD used in 

developing the 13 percent cost estimate, does not provide the military 

services and Congress with information needed to reliably develop and 

review budget requests to fund enhancements to the current program. We 

continue to believe that DOD needs to qualify this estimate with a 

measure of the risk associated with implementing its recommendations. 

Without providing the range of possible cost increases and the risk 

regarding the likelihood of achieving this 13 percent projection within 

that range, DOD may encounter a repetition of its experience with one 

of the pilot programs, which had to be terminated because actual costs 

exceeded projected costs. Absent this risk information, the military 

services will have to wait until after the transportation providers 

submit their bids in order to learn whether the recommendations can be 

implemented within the 13 percent projection. Should the bids result in 

costs that exceed this estimate, DOD and the military services will 

need to make adjustments to ensure that the recommendations are 

implemented within funding limits. Therefore, we continue to believe 

that our recommendation has merit.



Our finding that the implementation of the information technology 

improvements recommendation would likely cost $7 million rather than 

the $4 million to $6 million that DOD projected was based on 

information we received from DOD during the audit. Specifically, we 

calculated that the costs to develop and implement the new system would 

be about $5 million and that training for users of the enhanced system 

would cost an additional $1 million. DOD had identified the need for 

this training during its evaluation of the pilot programs. After DOD 

submitted its report to Congress, it identified another potential cost-

-an additional $1 million for independent verification and validation 

testing of the system. Our $7 million estimated included all three of 

these cost elements. In its response to a draft of this report, DOD 

maintained that its costs estimate would fall within its initial cost 

estimate of $4 million to $6 million, including $5 million for system 

development and implementation and an additional $500,000 each for user 

training and system verification and validation testing. At a minimum, 

based on these projected cost estimates, the initial cost to implement 

the information technology improvements recommendation would more 

likely be $6 million. However, since we did not originally assess the 

sufficiency of the $1 million estimates for training and testing, we 

are unable to assess what impact DODís reduction for these costs to 

$500,000 would have on the implementation of the system across DOD. We 

have reflected DODís changes in the body of our report.



DODís comments are reprinted in appendix III. DOD also provided 

technical comments, and we revised our report to reflect them where 

appropriate.



We performed our review from April 2002 through February 2003 in 

accordance with generally accepted government auditing standards. 

Appendix I contains the scope and methodology for this report. DODís 

comments are reprinted in their entirety in appendix III.



We are sending copies of this report to the appropriate congressional 

committees; the Secretary of Defense; the Commander, U.S. 

Transportation Command; and the Director, Office of Management and 

Budget. We will also make copies available to others upon request. In 

addition, the report will be made available at no charge on the GAO Web 

site at http://www.gao.gov.



Please contact me at (202) 512-8365 or Lawson Gist, Jr., at (202) 512-

4478 if you or your staff have any questions concerning this report. 

Key contributors to this assignment were Robert L. Self, Jacqueline S. 

McColl, Arthur L. James, Jr., Charles W. Perdue, and Nancy L. Benco.



William M. Solis

Director

Defense Capabilities and Management:



Signed by William M. Solis:



List of Congressional Committees:



The Honorable John Warner

Chairman

The Honorable Carl Levin

Ranking Minority Member

Committee on Armed Services

United States Senate:



The Honorable Ted Stevens

Chairman

The Honorable Daniel K. Inouye

Ranking Minority Member

Subcommittee on Defense

Committee on Appropriations

United States Senate:



The Honorable Duncan Hunter

Chairman

The Honorable Ike Skelton

Ranking Minority Member

Committee on Armed Services

House of Representatives:



The Honorable Joel Hefley

Chairman

The Honorable Solomon P. Ortiz

Ranking Minority Member

Subcommittee on Readiness

Committee on Armed Services

House of Representatives:



The Honorable Jerry Lewis

Chairman

The Honorable John P. Murtha

Ranking Minority Member

Subcommittee on Defense

Committee on Appropriations

House of Representatives:



[End of section]



Appendix I: Scope and Methodology:



To assess the extent to which the recommendations in the Department of 

Defenseís (DOD) November 2002 report to Congress addressed major 

problems in the personal property program, we took the following steps:



* To identify the major problems facing the current personal property 

program, we reviewed DOD and GAO reports addressing this program. These 

reports identified problems associated with quality of service and 

claims. We also conducted interviews with personal property program 

officials and their contractors to gain an understanding of the current 

data management systemís limitations and the long-standing problems 

involving the lack of reliable information on shipments and their 

costs.

:



* To determine whether the proposed recommendations in DODís report to 

Congress addressed the major problems of the current program, we 

tracked the recommendations back to the U.S. Transportation Commandís 

report on its evaluation results and assessed the extent to which the 

recommendations are linked to and have the potential to address 

problems.

:



To assess whether the recommendations in DODís report to Congress were 

supported by DODís evaluation findings and should be implemented, we 

took the following steps:



* To determine if the Transportation Command developed a 

methodologically sound evaluation plan, we assessed the commandís 

efforts against the findings and recommendations contained in our 

report[Footnote 10] on the Armyís Hunter Pilot Program results and 

against professional standards[Footnote 11] we would use if conducting 

a comparable evaluation. These sources addressed issues such as (1) 

seeking advice in designing a methodologically sound evaluation plan, 

(2) developing the evaluation plan prior to testing, (3) identifying 

factors to be assessed and the data required for analyses to develop 

findings and recommendations, (4) limiting quality of life surveys to 

only one for each participant to preclude survey ďfatigue,Ē and (5) 

conducting simultaneous testing of the pilot and current programs.



* To determine if the Transportation Command implemented an effective 

evaluation strategy during the data collection phase of its evaluation, 

we reviewed the pilot programsí efforts to collect data for the four 

factors as prescribed in the Transportation Commandís evaluation plan. 

We also assessed the adjustments the Transportation Command made in its 

evaluation strategy to address issues that could affect the soundness 

of the results. An example of the issues addressed included developing 

a constructed cost methodology to provide better estimates of what DOD 

would have paid under the current program for shipments made by the 

pilot programs.

:



* To assess the Transportation Commandís development of findings and 

recommendations to improve the current personal property program, we 

reviewed the evaluation techniques (quantitative and qualitative 

analyses) used to analyze data collected for the four factors. Further, 

we assessed the extent to which the Transportation Command adjusted the 

evaluation techniques to make up for differences in the way that the 

pilot programs provided data for the evaluation.

:



To assess the methodology that DOD used to develop cost estimates for 

implementing the recommendations, we took the following steps:



* To determine the reliability of the cost estimates for the pilot 

programs and for the proposed recommendations, we reviewed the cost 

projection methodologies used by the Transportation Command and by the 

Military Traffic Management Command. 

:



* To determine the reliability of pilot program shipment-related costs 

used in the report, we reviewed the data collection efforts used by 

each pilot program for the transportation and storage of household 

goods included in the Transportation Commandís evaluation. Further, we 

reviewed the constructed cost methodology used to develop the estimates 

of what DOD would have paid to make comparable shipments under the 

current program in the pilot programsí test areas.

:



* To determine the reasonableness of the assumptions and sources of 

data used to develop cost estimates for implementing recommendations 

for the personal property program, we met with officials from the 

Military Traffic Management Command and their contractor to discuss the 

methodology. We also reviewed the contents of their briefing on the 

cost estimate work for implementing changes to the claims process and 

performance-based service contracts and 





additional information the Military Traffic Management Command provided 

on the costs to implement information technology improvements.

:



We did not make an assessment of whether the anticipated benefits to be 

derived from implementing the three recommendations would warrant the 

additional costs DOD projects will be required to fund these 

improvements. Furthermore, we did not independently test the 

reliability of data DOD extracted from its data system to develop 

costs. We found that the department placed proper caveats on their use 

of such data, and in the case of comparing pilot programsí shipment 

costs to current program costs, developed a constructed cost 

methodology to address current program data management system 

weaknesses.



During this and prior[Footnote 12] reviews of DODís evaluation efforts, 

we met with officials and obtained documents from the Office of the 

Assistant Deputy Under Secretary of Defense (Transportation Policy), 

Washington, D.C.; the U.S. Transportation Command, Scott Air Force 

Base, Illinois; the Military Traffic Management Command, Alexandria, 

Virginia; the Department of Defense Inspector General, Full Service 

Moving Project, and Hay Group (Transportation Command Contractor), 

Arlington, Virginia; American Management Systems (Transportation 

Command contractor), PricewaterhouseCoopers (Military Traffic 

Management Command contractor), and Systems Research and Applications 

(Military Traffic Management Command contractor), Fairfax, Virginia; 

Logistics Management Institute (Military Traffic Management Command 

contractor), McLean, Virginia; the Navyís Service Member Arranged Move 

Pilot Program, Mechanicsburg, Pennsylvania; The Gallup Organization 

(Full Service Moving Project contractor), Omaha and Lincoln, Nebraska; 

and Parsifal Corporation (Military Management Traffic Command 

contractor), Palm Bay, Florida. In addition to these agency meetings 

and documents, we drew upon information contained in a testimony 

statement, in reports, and in status briefings resulting from our prior 

reviews of this program.



Our work for this review was performed from April 2002 through February 

2003 in accordance with generally accepted government auditing 

standards.

:



[End of section]



Appendix II: Overview of Current Personal Property Program and Pilot 

Programs:



The Transportation Command evaluated three pilot programs to assess 

alternative approaches that might address long-standing problems with 

its current personal property program. The following tables provide 

features of the current program and the three pilot programs. As the 

tables show, the pilot programs had several features that provided 

enhancements to military personnel and their families and to DOD that 

are not offered by the current program.



Table 1 compares claims-related features. Specifically, the pilot 

programs provided full replacement value rather than depreciated value 

for loss and damage and guaranteed claims settlement with 45 to 60 days 

of filing the claims.



Table 1: Claims-Related Features of the Current Personal Property 

Program and Pilot Programs:



Program features: Loss and damage claims.



Program features: Reimbursements: Current personal property program: 

[Empty]; Military Traffic Management Commandís Reengineered Personal 

Property Program: [Empty]; Department of Defenseís Full Service Moving 

Project: [Empty]; Navyís Service Member Arranged Move Pilot Program: 

[Empty].



Program features: --Basis for valuing property; Current personal 

property program: Depreciated value; Military Traffic Management 

Commandís Reengineered Personal Property Program: Full replacement 

value; Department of Defenseís Full Service Moving Project: Full 

replacement value; Navyís Service Member Arranged Move Pilot Program: 

Full replacement value.



Program features: --Maximum dollar value per move; Current personal 

property program: $40,000; Military Traffic Management Commandís 

Reengineered Personal Property Program: $63,000; Department of 

Defenseís Full Service Moving Project: $75,000; Navyís Service Member 

Arranged Move Pilot Program: $72,000.



Program features: Guaranteed claims settlement; Current personal 

property program: Not specified; Military Traffic Management Commandís 

Reengineered Personal Property Program: Within 60 days; Department of 

Defenseís Full Service Moving Project: Within 45 days; Navyís Service 

Member Arranged Move Pilot Program: Within 60 days.



Sources: DOD (data); GAO (analysis).



[End of table]



Table 2 compares the quality of service-related features. Some of the 

comparable features included emphasizing performance over cost in 

selecting transportation providers and prescreening of transportation 

providers.



Table 2: Quality of Service-Related Features of the Current Personal 

Property Program and Pilot Programs:



Program features: Counseling and arranging shipment services.



Program features: Single relocation coordinator; Current personal 

property program: Counseling and arranging shipment services: No; 

Military Traffic Management Commandís Reengineered Personal Property 

Program: Counseling and arranging shipment services: No; Department of 

Defenseís Full Service Moving Project: Counseling and arranging 

shipment services: Yes; Navyís Service Member Arranged Move Pilot 

Program: Counseling and arranging shipment services: Yes.



Program features: Program management and counseling services provider; 

Current personal property program: Counseling and arranging shipment 

services: Installation personal property shipping office; Military 

Traffic Management Commandís Reengineered Personal Property Program: 

Counseling and arranging shipment services: Installation personal 

property shipping office; Department of Defenseís Full Service Moving 

Project: Counseling and arranging shipment services: Private-sector 

move managers; Navyís Service Member Arranged Move Pilot Program: 

Counseling and arranging shipment services: Installation personal 

property shipping office.



Program features: Screening process for transportation providers.



Program features: Program emphasis in selecting transportation 

providers; Current personal property program: Counseling and arranging 

shipment services: Lowest cost; Military Traffic Management Commandís 

Reengineered Personal Property Program: Counseling and arranging 

shipment services: Performance; Department of Defenseís Full Service 

Moving Project: Counseling and arranging shipment services: 

Performance; Navyís Service Member Arranged Move Pilot Program: 

Counseling and arranging shipment services: Performance.



Program features: Prescreening of transportation providers; Current 

personal property program: Counseling and arranging shipment services: 

Very limited; Military Traffic Management Commandís Reengineered 

Personal Property Program: Counseling and arranging shipment services: 

Yes; Department of Defenseís Full Service Moving Project: Counseling 

and arranging shipment services: Yes; Navyís Service Member Arranged 

Move Pilot Program: Counseling and arranging shipment services: Yes.



Program features: Customer satisfaction surveys conducted and method; 

Current personal property program: Counseling and arranging shipment 

services: Surveys are not conducted; Military Traffic Management 

Commandís Reengineered Personal Property Program: Counseling and 

arranging shipment services: Survey company calls member; Department of 

Defenseís Full Service Moving Project: Counseling and arranging 

shipment services: Survey company calls member; Navyís Service Member 

Arranged Move Pilot Program: Counseling and arranging shipment 

services: Service member returns survey via mail.



Program features: Visibility of shipments during relocation process.



Program features: Methods used to increase service membersí visibility 

of shipments during move process; Current personal property program: 

Counseling and arranging shipment services: None; Military Traffic 

Management Commandís Reengineered Personal Property Program: 

Counseling and arranging shipment services: Toll-free number to 

transportation provider; Department of Defenseís Full Service Moving 

Project: Counseling and arranging shipment services: Toll-free number 

to move manager and transportation provider; Navyís Service Member 

Arranged Move Pilot Program: Counseling and arranging shipment 

services: Toll-free number to move coordinator and transportation 

provider, and member has pager.



Sources: DOD (data); GAO (analysis).



[End of table]



Table 3 compares data reliability-related features. As noted, only one 

of the pilot programs had a data management system that provided 

reliable information to track individual shipments in transit and 

provide overall data on shipments and their associated costs.



Table 3: Data Reliability-Related Features of the Current Personal 

Property Program and Pilot Programs:



Program features: Availability and reliability of data on household 

goods shipments.



Program features: Reliable data management system to track individual 

shipments in transit and to provide overall data on shipments and 

associated costs; Current personal property program: Availability and 

reliability of data on household goods shipments: No (current system is 

not designed to track shipments nor provide reliable shipment and cost 

data); Military Traffic Management Commandís Reengineered Personal 

Property Program: Availability and reliability of data on household 

goods shipments: Yes; Department of Defenseís Full Service Moving 

Project: Availability and reliability of data on household goods 

shipments: Not determined (developed but not fully operational-data 

management system needed additional refinement); Navyís Service Member 

Arranged Move Pilot Program: Availability and reliability of data on 

household goods shipments: Not determined (developed a database by end 

of pilot program but was not fully implemented or evaluated).



Sources: DOD (data); GAO (analysis).



[End of table]



Additional information on the current program and on each pilot program 

and its unique features follows.



Current Program:



The current DOD personal property program, valued at over $1.7 billion 

annually, moves more than 600,000 shipments each year for military 

personnel and their families from the military services, Defense 

agencies, and the Coast Guard. DOD is the moving industryís single 

largest customer. Managed centrally by the headquarters office of the 

Military Traffic Management Command and administered locally by about 

200 military and DOD transportation offices around the world, this 

program relies on over 1,200 domestic commercial carriers and more than 

150 forwarders for international traffic to provide moving and storage 

services.



Loss and Damage Claims:



When loss and damage occur, military personnel can submit claims to 

their respective military service claims office. Based on depreciated 

values, the reimbursement rate is $1.25 per pound multiplied by the 

shipment weight, with a maximum amount of $40,000 per move. Military 

personnel have up to 2 years after receiving their shipments to file 

claims but must submit notice of loss and damage within 70 days of 

delivery. The current program does not have a specified time period in 

which the claims are to be settled.



Counseling and Arranging Shipment Services:



The current program provides counseling services and arranges the 

shipment and storage of household goods and unaccompanied baggage 

through government representatives, who are available to assist 

military personnel and their families at the origin and destination 

points of their moves. The current program does not have a real-time 

tracking system for shipments nor does it provide a single point of 

contact to manage the entire moving process; therefore, military 

personnel may interact with several people at the origin and 

destination offices during their relocation.



Screening and Shipment Distribution Process for Transportation 

Providers:



The current system is not designed to select transportation providers 

on the basis of quality service; rather, transportation providers 

offering a minimally acceptable level of quality are generally selected 

based on the lowest rates. The program uses the Total Quality Assurance 

Program to develop quality scores for each transportation provider. 

Each local military installation distributes its traffic using a 

traffic distribution roster. Transportation providers are placed on the 

rosters for each channel (origin and destination areas) by order of 

rate level and quality score. Transportation providers who participate 

in the domestic part of the current program submit their rates as a 

percentage of the government tariff, which is nearly 20 years old. The 

providers who participate in the international part of the current 

program submit single factor or fixed rates per hundredweight of the 

shipments.



The current program does not use customer satisfaction surveys as a 

means to evaluate transportation provider performance. To remain in the 

program, a provider must maintain a minimally acceptable level of 

quality-a 90 percent score. Three factors are measured: on-time pickup, 

on-time delivery, and reported loss and damage to determine if points 

should be deducted from transportation providers and allocation of 

shipments should be reduced or terminated.



Visibility of Shipments during the Relocation Process:



The current program does not provide service members with real-time 

visibility of shipments during the relocation process.

:



Availability and Reliability of Data on Household Goods Shipments:



Personnel at origin and destination personal property shipping offices 

enter information on shipments to their respective Transportation 

Operational Personal Property Standard Systems. However, data in these 

individual systems does not include all shipments that occur during the 

year, and the systems are not accessible to all parties involved in the 

relocation process. Destination personal property shipment offices are 

forwarded information on shipments via the current system; however, 

payment data on these shipments is maintained in a separate system.



In addition to not providing information on all aspects of individual 

shipments, the current programís data management system does not 

provide DOD and the military services information about the types of 

shipments and related costs for planning and budgeting purposes. The 

following are examples of the current systemís limitations:



* the format of the system makes compiling data from multiple sites 

difficult;



* not all data is captured promptly; and:



* not all data and costs are captured/updated in the system.



Funding Sources:



Under the current program, the military services reimburse carriers and 

forwarders for shipment-related costs from military personnel accounts. 

Personal property shipment office expenses and claims filed with the 

government are funded from the military servicesí operations and 

maintenance accounts.



Military Traffic Management Commandís Reengineered Personal Property 

Program:



Sponsored by the Military Traffic Management Command, the Reengineered 

Personal Property Program included outbound shipments for military and 

Coast Guard personnel departing from installations located in North 

Carolina, South Carolina, and Florida (excluding Tyndall Air Force 

Base). The pilot program ran concurrently with the existing program at 

these installations. The pilot programís goal was to include 50 percent 

of eligible moves from the above installations to continental United 

States and European locations. The remaining shipments were to be moved 

under the existing program. The Reengineered Personal Property Program 

was initiated in January 1999 and operated for 12 months before data 

was submitted to the Transportation Command for evaluation.



Loss and Damage Claims:



Reimbursement for loss and damage claims was increased from depreciated 

value to full replacement value, and the dollar amounts per move 

increased from $40,000 under the current program to $63,000 under the 

Reengineered Personal Property Program. Additionally, the pilot program 

provided direct claims settlement between military personnel and their 

transportation providers and a requirement that transportation 

providers settle claims within 60 days of receiving claims forms from 

military personnel.



Counseling and Arranging Shipment Services:



Like the current program, the Reengineered Personal Property Program 

relied on personnel in the personal property shipping offices to 

provide counseling services and arrange for shipment and storage of 

household goods and unaccompanied baggage. A central contact point in 

these offices was not designated to manage the entire moving process; 

therefore, military personnel may have interacted with several people 

at the origin and destination offices during the relocation process. 

However, to improve customer service, the programís Pilot 

Transportation Operational Personal Property Standard System provided 

real-time worldwide tracing capability.



Screening and Shipment Distribution Process for Transportation 

Providers:



Greater emphasis was placed on performance in awarding shipments to 

transportation providers. Evaluation of financial status, elimination 

of high-risk companies, and consideration of providersí past 

performance, rather than lowest bid, played the dominant role in 

selecting initial transportation providers to participate in this pilot 

program. Transportation providers who participated in the pilot program 

submitted their bids for various origin and destination routes as a 

discount from the commercial tariff. Prices were fixed for a year, with 

no provision for rate increases during the contract period. Awards were 

made only to transportation providers whose offers conformed to the 

solicitation and represented the best overall value to the government.



The Military Traffic Management Command evaluated company performance 

quarterly and compliance with terms and conditions of the contracts 

annually. Subsequent performance reviews were conducted based on 

customer satisfaction surveys and claims data. After transportation 

providers received their minimum guarantee of business for the year 

($25,000), future awards were offered to the best performers. Feedback 

was provided monthly to transportation providers, and those that became 

poor performers were no longer offered household goods and 

unaccompanied baggage shipments.



Visibility of Shipments during the Relocation Process:



The Reengineered Personal Property Program provided the transportation 

providerís toll-free number to military personnel to enhance visibility 

over their shipments throughout the relocation process.



Availability and Reliability of Data on Household Goods Shipments:



The Reengineered Personal Property Program implemented its central, 

Web-based Pilot Transportation Operational Personal Property Standard 

System in part to address problems associated with visibility of and 

availability of information on shipments during the relocation process. 

The pilot programís data management system provided real-time access to 

both shipment and payment records. Access to the various modules of the 

system was granted to personal property shipment office personal at 

origin and destination locations, transportation providers, invoice 

certifying officers, prepayment auditors, military service 

headquarters, and military service claims offices and finance centers, 

based on each partyís need for the information.



The systemís design allowed for entry of current address and telephone 

numbers of military personnel to improve the process of delivering 

household goods to a new residence. Data reliability was enhanced under 

the Reengineered Personal Property Program, but one problem noted 

during the evaluation was the need for military personnel to ensure 

that their contact information (phone number and address) was current 

during the relocation process. This had an effect on deliveries of 

household goods and the quality of life survey contractorís ability to 

reach military personnel to ascertain their opinions about their 

relocation experience.



In addition to providing information on all aspects of individual 

shipments, the Reengineered Personal Property Programís data management 

system demonstrated the potential to provide DOD and the military 

services with information about the types of shipments and related 

costs managed annually for planning and budgeting purposes.



Other Reengineered Personal Property Program Features:



The Reengineered Personal Property Program achieved stronger 

transportation provider commitment with long-term contracts, and it 

used contractor support to conduct quality of life surveys with 

military personnel moving under the pilot program and to perform audits 

of each invoice submitted by transportation providers.



Funding Sources:



Like the current program, the military services reimbursed carriers and 

forwarders for shipment-related costs from their military personnel 

accounts. Transportation office expenses and any claims filed with the 

government were funded from the servicesí operations and maintenance 

accounts.



The Department of Defenseís Full Service Moving Project:



Sponsored by the Office of the Assistant Deputy Under Secretary of 

Defense (Transportation Policy), the Full Service Moving Project 

included outbound shipments for military and Coast Guard personnel and 

DOD civilian departing from locations in the National Capital Region, 

Georgia (excluding Robins Air Force Base), and Minot Air Force Base, 

North Dakota. The pilot programís goal was to include 90 percent of the 

moves from these locations to continental United States and to European 

and Asian-Pacific locations. The remaining shipments were to be moved 

under the current program. The Full Service Moving Project began in 

January 2001 and continued until its early termination in September 

2001. Due to continuing delays in implementing this pilot program and 

DODís decision to terminate the pilot program in September 2001, the 

Full Service Moving Project had limited operational experience before 

submitting data to the Transportation Command.



Loss and Damage Claims:



Reimbursement for loss and damage claims was increased from depreciated 

value to full replacement value, with the dollar amounts increasing 

from $40,000 per move under the current program to $75,000 per move 

under the Full Service Moving Project. Additionally, the pilot program 

provided for direct claims settlement between military personnel and 

their transportation providers and a requirement that the responsible 

party (transportation providers or move managers) settle claims within 

45 days of receiving claim forms from military personnel.



Counseling and Arranging Shipment Services:



Unlike the current program and other pilot programs, the Full Service 

Moving Project tested the use of private-sector relocation companies 

(move managers) for outsourcing traditional transportation services 

(counseling and arranging for the shipment and storage of household 

goods and unaccompanied baggage) performed by origin and destination 

personal property shipping offices. The pilot programís goal was to 

provide a single point of contact (move manager) for military personnel 

and transportation providers to contact throughout the relocation 

process.



Screening and Shipment Distribution Process for Transportation 

Providers:



The Full Service Moving Project made major changes to the existing 

transportation provider approval, rate solicitation, and traffic 

distribution processes. The pilot program emphasized best value and 

placed more weight on performance (70 percent) than cost (30 percent) 

in determining which transportation providers would be awarded 

shipments. The pilot program contracted with a financial services 

company to conduct financial and performance assessments of 

transportation providers and move manager companies that wanted to 

participate in the pilot program. For approved transportation 

providers, rates were established for a 1-year cycle. The providers 

submitted their rates as a discount from the commercial tariff for 

domestic shipments and negotiated single rate factors for the overseas 

locations. Approved move managers were awarded 2-year contracts with 1-

year options. The move management companies competitively bid their 

fees as flat rates, depending on whether they were responsible for 

claims settlement or the transportation provider carried this 

liability. Also, different fees were established for domestic and 

international shipments.



The Full Service Moving Project used survey data from all personnel 

participating in the pilot program to determine future percentages of 

shipments that would be allocated to the transportation providers. The 

pilot program also planned to use survey data on move manager 

performance to determine future participation in the pilot program and 

incentive payments. The Full Service Moving Projectís Web-based Best 

Value Distribution Database maintained the transportation providersí 

quality and cost scores based on survey information and costs 

associated with prior shipments. Move managers used this data to assign 

future shipments. However, in some instances (i.e., for group moves, 

when meeting small business requirements, when there was a lack of 

transportation provider capacity to handle shipments offered, for 

multiple shipments to a single transportation provider, and for 

international shipments to areas without an established rate), move 

managers were told to deviate from the information provided by the data 

management system.



Visibility of Shipments during the Relocation Process:



One of the goals of incorporating move managers into the relocation 

process was to provide real-time information to military personnel and 

to transportation providers regarding the status of household goods 

shipments. The move managers, unlike the current programís personal 

property shipping office personnel, were responsible for the entire 

relocation process from the point of origin in establishing 

entitlements, arranging for transportation providers, and handling 

other personnel-related issues, to the destination in overseeing 

deliveries, approving storage, and either settling claims or assisting 

military personnel with issues involving settling claims with the 

transportation providers if the liability fell with the providers. 

Working with both military personnel and transportation providers, the 

move managers used contact information to keep military personnel 

informed of their shipmentsí status and to coordinate the delivery of 

the shipments at the destination. Additionally, as part of the pilot 

program, all participants were provided a toll-free number to maintain 

visibility over their shipments throughout the process.



Availability and Reliability of Data on Household Goods Shipments:



In addition to move managers, the Full Service Moving Projectís Web-

based Best Value Distribution Database was implemented to address 

problems associated with visibility of shipments during the relocation 

process. The pilot programís data management system had access to both 

shipment and payment records via interface with US Bankís PowerTrack 

and the move managersí systems. Access to the pilot programís data 

management system was granted to move managers, invoice certifying 

officers, military service headquarters, and military service claims 

offices and finance centers, based on each partyís requirements.



Move managers were responsible for keeping the status of the shipments 

current in the pilot programís data management system. However, the 

move managers did not always update this information in the system. 

Further, the ability of the move manager to contact the service member 

was directly affected by the information provided by the member.



The Full Service Moving Projectís Web-based Best Value Distribution 

Database was anticipated to provide DOD and the military services 

information on the types of shipments and related costs managed 

annually for planning and budgeting purposes. Unlike the Reengineered 

Personal Property Program where various parties in the relocation 

process entered data into that pilot programís data management system, 

the majority of the data in the Full Service Moving Projectís data 

management system was predicated on the move managers gathering and 

entering the information.



Other Full Service Moving Project Features:



The Full Service Moving Project achieved stronger transportation 

provider commitment with long-term contracts and faster payment of 

invoices; it offered binding cost estimates for shipments; it used 

contractor support to conduct quality of life surveys with military 

personnel moving under the pilot program and to perform audits of each 

invoice submitted by the transportation providers; and it offered 

optional relocation referral assistance for activities such as the 

purchase and sale of service membersí residences.



Move managers were required to perform prepayment audits and business 

rules were established for an automatic payment method. Payment 

methodology was predicated on the move manager entering the expected 

invoice into PowerTrack and the transportation provider submitting a 

notice of delivery and invoice. Payment timeliness was also driven by 

the timeliness of documentation submitted by the transportation 

providers. On some invoices, the contracting representative had to 

review and certify payment in PowerTrack. This occurred when the match 

showed a difference of more than $1.00.



Funding Sources:



For this pilot program, the military services reimbursed carriers and 

forwarders for shipment-related costs from military personnel accounts. 

These accounts were also used to fund move manager expenses. Any claims 

that might have been filed with the government would have been funded 

from the military servicesí operations and maintenance accounts.



Navyís Service Member Arranged Move Pilot Program:



Sponsored by the Navy, the Service Member Arranged Move Pilot Program 

included only domestic outbound intrastate and interstate shipments for 

Navy personnel moving from its installations located at Puget Sound, 

Washington; San Diego, California; Norfolk, Virginia; New London, 

Connecticut; and Whidbey Island, Washington. One of this programís 

objectives was to offer Navy military personnel a set of moving choices 

to meet their specific needs. This pilot program was one of three 

choices offered. Military personnel moving from the above locations 

could choose to move under the current personal property program, move 

their own household goods, or participate in the pilot program. The 

pilot program was initiated in April 1997 and began operations in 

January 1998. Because the Navy decided not to scope the Service Member 

Arranged Move Pilot Program comparable to other pilot programs (i.e., 

operational at multiple military services) and the pilot program did 

not provide data as outlined by the Transportation Commandís evaluation 

plan, its inclusion in the Transportation Commandís evaluation was 

limited to a qualitative assessment.



Loss and Damage Claims:



Reimbursement for loss and damage claims was increased from depreciated 

value to full replacement value, with the dollar amounts per move 

increasing from $40,000 under the current program to $72,000 under the 

Service Member Arranged Move Pilot Program. Additionally, the pilot 

program provided direct claims settlement between military personnel 

and their transportation providers and a requirement that 

transportation providers settle claims within 60 days of receiving 

claims forms from military personnel.



Counseling and Arranging Shipment Services:



Like the current program, the Service Member Arranged Move Pilot 

Program also relied on personnel in the personal property shipping 

offices to provide counseling services and arrange shipment and storage 

of household goods and unaccompanied baggage. However, the shipping 

office personnel at the origin installations participating in this 

pilot program served as the single point of contact coordinating the 

service membersí moves and remained available throughout the move to 

handle all issues, including claims. Unlike those participating in the 

current program and other pilot programs, service members participating 

in this pilot program identified the transportation provider they 

desired to handle their household goods shipments after they completed 

their reviews of participating providersí vendor quality books 

(containing provider information and marketing materials) and of 

surveys completed by previous pilot program participants. The personal 

property office coordinator assigned to the service member had to 

concur with the memberís request, and the coordinator made actual 

arrangements with the carrier.



Screening and Shipment Distribution Process for Transportation 

Providers:



Staff in the program management office and personal property shipping 

offices participating in the Service Member Arranged Move Pilot Program 

initially screened transportation providers that wished to participate 

in the pilot program based on providersí performance rather than low 

cost. Letters of agreement were adopted to streamline the contracting 

process and improve the quality of the move for Navy personnel. 

According to pilot program officials, these letters of agreement 

provided commercial best practices and enabled lessons learned from 

prior pilot program efforts and industry to be incorporated into the 

Navy pilot program. Actual contract awards were made on a case-by-case 

basis based on the best value decision for each move. Transportation 

providers used commercial tariffs in developing their bids for each 

move.



Transportation providers approved to participate in the pilot program 

submitted their bids for various origin and destination channels using 

commercial tariffs. Bids were rejected if they did not fall within 

acceptable percentage discounts. Feedback was provided monthly to 

transportation providers, and those that became poor performers were no 

longer offered household goods and unaccompanied baggage shipments. 

Subsequently, service members who were planning their upcoming moves 

relied on information contained in a nine-question survey that other 

service members had completed after their moves and claims process 

ended. Service members who volunteered to participate in this pilot 

program had to manually review carrier books, which included documents 

provided by the carriers and prior surveys completed by service members 

who had been moved by the carriers. According to pilot program 

officials, six carriers were terminated or canceled from the pilot 

program-four for providing poor service and two for price gouging.



Visibility of Shipments during Relocation Process:



The Service Member Arranged Move Pilot Program relied upon the shipping 

office personnel, who served as the single point of contact 

coordinating the service membersí moves, to maintain visibility of 

shipments during the relocation process. In addition, the pilot program 

provided both the personal property shipping officeís and 

transportation providerís toll-free numbers, as well as a pager to 

service members to enhance the membersí visibility of their shipments 

during the relocation process.



Availability and Reliability of Data on Household Goods Shipments:



The Service Member Arranged Move Pilot Program did not initially 

develop an alternative data management system to capture data on 

shipments and payment records. By the end of the pilot program, the 

Navy had developed a database to capture shipment data; however, the 

system was not fully implemented or evaluated.



Unlike the other pilot programs, the Navy pilot program used local 

personal property program personnel rather than third parties to review 

all invoices for payment. Navy personnel who participated in the pilot 

program completed their own surveys, mailing the paper forms to their 

respective personal property program offices.



Other Service Member Arranged Move Pilot Program Features:



The Service Member Arranged Move Pilot Program was designed to offer 

all shipments to small businesses, to provide direct claims settlements 

between Navy personnel and the transportation providers, to make faster 

payments to transportation providers through government purchase cards, 

and to establish a stronger commitment from transportation providers by 

offering long-term contracts.



Funding Sources:



Like the current program, the Navy reimbursed carriers and forwarders 

for shipment-related costs from its military personnel account. 

Personal property shipment office expenses were funded from the Navyís 

operations and maintenance account. While information on claims filed 

with the government was not provided, under this pilot program such 

expenses would also be funded from the operations and maintenance 

account.



[End of section]



Appendix III: Comments from the Department of Defense:



DEPUTY UNDER SECRETARY OF DEFENSE FOR LOGISTICS AND MATERIEL READINESS 

3500 DEFENSE PENTAGON WASHINGTON, DC 20301-3500:



MAR 25 2003:



Mr. William M. Solis:



Director, Defense Capabilities and Management U.S. General Accounting 

Office:



Washington, D.C. 20548:



Dear Mr. Solis:



This is the Department of Defense (DoD) response to the GAO draft 

report, GAO-03-367, ďDEFENSE TRANSPORTATATION: Monitoring Costs and 

Benefits Needed While Implementing a New Program for Moving Household 

Goodsí, dated February 24, 2003 (GAO Code 350161).



The Department has reviewed the subject GAO draft report and offers the 

attached comments.



Sincerely,



Allen W. Beckett,

Principal Assistant:



Signed by Allen W. Beckett:



GAO-03-367/GAO CODE 350161:



ďDEFENSE TRANSPORTATION: MONITORING COSTS AND BENEFITS NEEDED WHILE 

IMPLEMENTING A NEW PROGRAM FOR MOVING HOUSEHOLD GOODSĒ:



DEPARTMENT OF DEFENSE COMMENTS TO THE RECOMMENDATIONS:



RECOMMENDATION 1: The GAO recommended that the Secretary of Defense 

direct the Commander, U.S. TRANSCOM, to initiate actions to implement 

the three recommendations contained in DODís report to Congress within 

budget constraints.



(Page 21/Draft Report):



DoD RESPONSE: DoD concurs with this recommendation. DoD is currently 

developing a plan to implement the recommendations. The Department 

expects new program implementation beginning with electronic payment 

process in 4QFY03, with remaining changes beginning in 1QFY05, 

continuing through 1QFY06 assuming the Military Services receive the 

additional funds needed to fund program enhancements.



RECOMMENDATION 2: The GAO recommended that the Secretary of Defense 

direct the Commander, U.S. TRANSCOM, to provide the Military Services 

and Congress additional information to quantify the risk associated 

with achieving the 13 percent projected cost estimate before the claims 

process and performance-based service contracts recommendations are 

implemented to provide the Military Services with information needed 

for budgeting purposes. (Page 21/Draft Report):



DoD RESPONSE: DoD partially concurs with this recommendation. The 

Department continues to believe that the factors used in developing its 

projected cost estimate for the claims process and performance-based 

service contract recommendations are supportable and the new program 

can be implemented within the projected 13 percent increase over 

current program costs. The cost analysis was based on historical data 

and assumptions only where data was not available to provide analytical 

information. The GAO report emphasizes a lack of confidence in the 

adjustment made to the averaged pilot costs for economies of scale/

program efficiencies. The 5 percent reduction to the averaged pilot 

costs for economies of scale is not only reasonable, but also very 

conservative. The shipment costs in the pilots were based on a new set 

of business practices with only 3,000-5,000 shipments annually spread 

among the participating transportation providers. The current and new 

program encompasses approximately 600,000 shipments annually, about 200 

times larger scale of operations than the pilots. The larger volume 

resulting in a lower cost per unit concept is a standard and accepted 

law of economics. While the GAO reports the basis of the 5 percent 

reduction for economies of scale was not based on either historical 

experience or data provided by DoD, they did not declare the 

methodology flawed.



GAO concurs with the methodology used for estimating costs for the 

pilot programs and for two of the three cost reductions used to 

estimate the future program cost increase. We are confident in all 

three. We have presented the methodology to several of the Military 

Services. One

Military Service performed their own analysis and validated the 13 

percent. The methodology evaluated the ďunproductiveĒ cost drivers 

(shipment weight, small/large carrier mix, economies of scale) in the 

pilots and compensated for them using very conservative estimates.



DoD does not see value added in providing the Military Services or 

Congress a formal risk assessment but will continue to work with the 

Military Services as execution progresses to make sure they have all 

information required for budget purposes.



The Department non-concurs with the GAO finding that the cost estimate 

for implementing technology improvements will be $7M. The Department 

maintains that the cost to implement a new web-based system will fall 

between $4-6M. We projected costs of $5 million for development and 

implementation. Training costs and approximately two months of costs 

for independent verification and validation (IV&V) testing are 

estimated to be $500,000 each respectively.



RECOMMENDATION 3: The GAO recommended that the Secretary of Defense 

direct the Commander, U.S. TRANSCOM, to monitor costs for all 

recommendations during the implementation phase to ensure that the 

proposed changes are being achieved within an acceptable and predefined 

range. (Page 21-22/Draft Report):



DoD RESPONSE: DoD concurs with this recommendation. Rolling out the new 

program will require monitoring of costs to determine if the moving 

industry partners are submitting bids that will allow the Department to 

enhance this program within the 13 percent cost increase. The 

Department intends to establish program requirements that will focus 

more on quality of services (weighted best value score: 70 percent 

performance and 30 percent cost). To address the issue of monitoring 

costs, the plan includes a process to conduct a rate reasonableness 

analysis upon receipt of the rates. Those rates found to be outside the 

range of reasonableness will be rejected, and the carries will be given 

one opportunity to resubmit their rates-a function of controlling and 

evaluating costs. Only rates determined to be reasonable will be 

available for use in the program. For the first time, the Department 

will have one database with all cost data elements required by the 

Departmentís functional and financial communities for cost evaluation 

at the shipment level. The establishment of metrics, target/benchmark 

performance indicators, and methodology for data collection will be 

included in the updated program plan of action and milestone document 

that was not available at the time of the GAO audit.



RECOMMENDATION 4: The GAO recommended that the Secretary of Defense 

direct the Commander, U.S. TRANSCOM, to assess the effects of the three 

recommendations on the personal property program (to be carried out 

after the summertime peak moving season once the recommendations have 

been implemented) to determine whether the anticipated improvements in 

the program are being achieved at a reasonable cost. (Page 22/Draft 

Report):



DoD RESPONSE: DoD concurs with this recommendation. The Department will 

have the information needed to determine if anticipated improvements 

have been achieved on a continuing basis. A customer satisfaction 

survey will be implemented to assess improvements in the program and 

identify how well the carriers are performing. This approach allows the 

relocating member to access carrier performance and report customer 

satisfaction. Also, since the surveys will be completed upon shipment 

delivery, timely performance data will be

available. Carrier performance ratings will be established quarterly, 

with the exception of the peak season, when performance ratings with be 

established monthly.



[End of section]



Related GAO Products:



Defense Transportation: Final Evaluation Plan Is Needed to Assess 

Alternatives to the Current Personal Property Program. GAO/NSIAD-00-

217R. Washington, D.C.: September 27, 2000.



Defense Transportation: The Armyís Hunter Pilot Project Is Inconclusive 

but Provides Lessons Learned. GAO/NSIAD-99-129. Washington, D.C.: June 

23, 1999.



Defense Transportation: Plan Needed for Evaluating the Navy Personal 

Property Pilot. GAO/NSIAD-99-138. Washington, D.C.: June 23, 1999.



Defense Transportation: Efforts to Improve DODís Personal Property 

Program. GAO/T-NSIAD-99-106. Washington, D.C.: March 18, 1999.



Defense Transportation: The Armyís Hunter Pilot Project to Outsource 

Relocation Services. GAO/NSIAD-98-149. Washington, D.C.: June 10, 1998.



Defense Transportation: Reengineering the DOD Personal Property 

Program. GAO/NSIAD-97-49. Washington, D.C.: November 27, 1996.



FOOTNOTES



[1] The Deputy Under Secretary of Defense (Logistics) tasked the U.S. 

Transportation Command to evaluate alternatives and develop 

recommendations to enhance the departmentís current personal property 

program.



[2] The three pilots included in the evaluation are the Military 

Traffic Management Commandís Reengineered Personal Property Program, 

the Department of Defenseís Full Service Moving Project, and the Navyís 

Service Member Arranged Move Pilot Program.



[3] U.S. General Accounting Office, Defense Transportation: Efforts to 

Improve DODís Personal Property Program, GAO/T-NSIAD-99-106 

(Washington, D.C.: Mar. 18, 1999).



[4] U.S. General Accounting Office, Defense Transportation: Final 

Evaluation Plan Is Needed to Assess Alternatives to the Current 

Personal Property Program, GAO/NSIAD-00-217R (Washington, D.C.: Sept. 

27, 2000). 



[5] DOD reported that the 146 day recovery time is due to the fact that 

all current DOD contracts for shipment and storage of household goods 

give the carriers 120 days from receipt of the governmentís demand to 

pay, deny, or make a final written offer on the claims. While many 

claims are settled in less that 120 days, mailing time and negotiations 

to resolve disputes result in a slightly higher average settlement 

time.



[6] For example, a 10,000-pound shipment would have a maximum carrier 

liability for loss and damage of $12,500. Service members can make 

shipments exceeding their weight allowance by paying the extra shipping 

and storage costs for the overage in weight. However, the governmentís 

liability is limited to $40,000 per move regardless of the excess 

weight shipped and stored.



[7] U.S. General Accounting Office, Defense Transportation: The Armyís 

Hunter Pilot Project Is Inconclusive but Provides Lessons Learned, GAO/

NSIAD-99-129 (Washington, D.C.: June 23, 1999).



[8] In later discussions, department officials stated that ďeconomies 

of scaleĒ should be changed to ďprogram efficienciesĒ to reflect a more 

efficient program with timely and accurate management data; member 

counseling; and reduced loss, storage, and indirect costs. 



[9] We believe that the methodology and data used to develop these cost 

estimates are sound.



[10] GAO/NSIAD-99-129.



[11] U.S. General Accounting Office, Developing and Using 

Questionnaires, GAO/PEMD-10.1.7 (Washington, D.C.: Oct. 1993) and U.S. 

General Accounting Office, Using Structured Interviewing Techniques, 

GAO/PEMD-10.1.5 (Washington, D.C.: June 1991).



[12] Our prior reviews focused on the departmentís efforts (1) to 

develop a methodologically sound evaluation plan and (2) to collect 

data according to the plan for future analysis and development of 

recommendations for an improved departmentwide program.



GAOís Mission:



The General Accounting Office, the investigative arm of Congress, 

exists to support Congress in meeting its constitutional 

responsibilities and to help improve the performance and accountability 

of the federal government for the American people. GAO examines the use 

of public funds; evaluates federal programs and policies; and provides 

analyses, recommendations, and other assistance to help Congress make 

informed oversight, policy, and funding decisions. GAOís commitment to 

good government is reflected in its core values of accountability, 

integrity, and reliability.



Obtaining Copies of GAO Reports and Testimony:



The fastest and easiest way to obtain copies of GAO documents at no 

cost is through the Internet. GAOís Web site ( www.gao.gov ) contains 

abstracts and full-text files of current reports and testimony and an 

expanding archive of older products. The Web site features a search 

engine to help you locate documents using key words and phrases. You 

can print these documents in their entirety, including charts and other 

graphics.



Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as ďTodayís Reports,Ē on its 

Web site daily. The list contains links to the full-text document 

files. To have GAO e-mail this list to you every afternoon, go to 

www.gao.gov and select ďSubscribe to daily E-mail alert for newly 

released productsĒ under the GAO Reports heading.



Order by Mail or Phone:



The first copy of each printed report is free. Additional copies are $2 

each. A check or money order should be made out to the Superintendent 

of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. 

Orders should be sent to:



U.S. General Accounting Office



441 G Street NW,



Room LM Washington,



D.C. 20548:



To order by Phone: 	



	Voice: (202) 512-6000:



	TDD: (202) 512-2537:



	Fax: (202) 512-6061:



To Report Fraud, Waste, and Abuse in Federal Programs:



Contact:



Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov



Automated answering system: (800) 424-5454 or (202) 512-7470:



Public Affairs:



Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.



General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.



20548: