This is the accessible text file for GAO report number GAO-07-208T 
entitled 'Defense Travel System: Estimated Savings Are Questionable and 
Improvements Are Needed to Ensure Functionality and Increase 
Utilization' which was released on November 16, 2006. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

Testimony: 

Before the Permanent Subcommittee on Investigations, Committee on 
Homeland Security and Governmental Affairs, U.S. Senate: 

United States Government Accountability Office: 

GAO: 

For Release on Delivery Expected at 10:00 a.m. EST: 

Thursday, November 16, 2006: 

Defense Travel System: 

Estimated Savings Are Questionable and Improvements Are Needed to 
Ensure Functionality and Increase Utilization: 

Statement of McCoy Williams, Director Financial Management and 
Assurance: 

Keith A. Rhodes, Chief Technologist Applied Research and Methods Center 
for Technology and Engineering: 

GAO-07-208T: 

GAO Highlights: 

Highlights of GAO-07-208T, a testimony to the Permanent Subcommittee on 
Investigations, Committee on Homeland Security and Governmental 
Affairs, U.S. Senate 

Why GAO Did This Study: 

In 1995, the Department of Defense (DOD) began an effort to implement a 
standard departmentwide travel system. The Defense Travel System (DTS) 
is envisioned as DOD’s standard end-to-end travel system. This 
testimony is based on GAO’s September 2006 related report. Today’s 
testimony highlights GAO’s key findings with regard to the following 
objectives: (1) Were the two key assumptions made in the September 2003 
economic analysis reasonable? (2) Was DOD taking action to ensure full 
utilization of DTS and gathering the data needed to monitor DTS 
utilization? and (3) Has DOD resolved several functional problems 
associated with weak system requirements and testing? To address these 
objectives, GAO (1) reviewed the September 2003 DTS economic analysis, 
(2) analyzed DTS utilization data, and (3) analyzed DTS flight 
information. 

What GAO Found: 

GAO’s analysis of the September 2003 DTS economic analysis found that 
the two key assumptions used to estimate annual net savings were not 
based on reliable information. Two cost components represent the 
majority of the over $56 million in estimated net savings—personnel 
savings and reduced commercial travel office (CTO) fees. In regard to 
the personnel savings, GAO’s analysis found that the $24.2 million of 
personnel savings related to the Air Force and the Navy were not 
supported.

* Air Force and Navy DTS program officials stated that they did not 
anticipate a reduction in the number of personnel, but rather the 
shifting of staff from the travel function to other functions.

* The Naval Cost Analysis Division stated that the Navy will not 
realize any tangible personnel cost savings from the implementation of 
DTS. 

In regard to the CTO fees, the economic analysis assumed that 70 
percent of all DTS airline tickets would either require no intervention 
or minimal intervention from the CTOs, resulting in an estimated annual 
net savings of $31 million. However, the sole support provided by the 
DTS program office was an article in a trade industry publication. The 
article was not based on information related to DTS, but rather on the 
experience of one private sector company. Furthermore, the economic 
analysis was not prepared in accordance with guidance prescribed by OMB 
and DOD.

* DOD guidance stated that the life-cycle cost estimates should be 
verified by an independent party, but this did not occur.

* The economic analysis did not undertake an assessment of the effects 
of the uncertainty inherent in the estimates of benefits and costs. 
Because an economic analysis uses estimates and assumptions, it is 
critical that the imprecision in both the underlying data and 
assumptions be understood. Such an assessment is referred to as a 
sensitivity analysis. 

DOD acknowledged that DTS is not being used to the fullest extent 
possible, but lacks comprehensive data to effectively monitor its 
utilization. DOD’s utilization data are based on a model that was 
developed in calendar year 2003. However, the model has not been 
completely updated to reflect actual DTS usage. The lack of accurate 
utilization data hinders management’s ability to monitor progress 
toward the DOD vision of DTS as the standard travel system. GAO also 
found that the military services have initiated actions that are aimed 
at increasing the utilization of DTS. 

Finally, GAO found that DTS still has not addressed the underlying 
problems associated with weak requirements management and system 
testing. While DOD has acted to address concerns GAO previously raised, 
GAO found that DTS’s requirements are still ambiguous and conflicting. 
For example, DTS displaying up to 25 flights for each inquiry is 
questionable because it is unclear whether this is a valid requirement. 
Until DOD improves DTS’s requirements management practices, the 
department will not have reasonable assurance that DTS can provide the 
intended functionality. 

What GAO Recommends: 

GAO made four recommendations aimed at improving the management of DTS, 
including periodic reports on DTS utilization and resolution of 
inconsistencies in DTS’s requirements. DOD generally agreed with the 
recommendations and described efforts to address them. DOD also 
strongly objected to the finding that the reported personnel savings 
were unrealistic. However, DOD provided no new data that were counter 
to GAO’s findings. 

[Hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-07-208T]. 

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact McCoy Williams at (202) 
512-9095 or Keith Rhodes at (202) 512-6412. 

[End of Section] 

Mr. Chairman and Members of the Subcommittee: 

Thank you for the opportunity to discuss our recent report[Footnote 1] 
related to problems encountered by the Department of Defense (DOD) in 
its efforts to successfully implement the Defense Travel System (DTS). 
As you know, DOD envisions DTS as the department's standard end-to-end 
travel system.[Footnote 2] The department estimates that DTS will be 
fully deployed at all 11,000 intended locations during fiscal year 
2007.[Footnote 3] The September 2003 economic analysis noted that DTS, 
when fully implemented, would result in annual net savings of over $56 
million. The economic analysis noted that savings would be realized by 
the department during fiscal years 2009-2016. In December 2003, the 
department's Chief Information Officer approved a DTS funding level of 
approximately $564 million. Of this amount, the contract for the 
design, development, and deployment of DTS was for about $264 million. 
The remaining costs are associated with areas such as the operation and 
maintenance of DTS, operation of the Program Management Office-Defense 
Travel System (PMO-DTS), the voucher payment process, and management 
and oversight of the numerous contracted commercial travel offices 
(CTO). 

My testimony today is based on our September 2006 report,[Footnote 4] 
which followed up on our September 2005 testimony and January 2006 
report.[Footnote 5] One of the major findings in our previous work was 
that DOD did not have reasonable assurance that flight information was 
properly displayed for DOD travelers because the department failed to 
properly test the system interfaces through which the data are accessed 
for display. We further noted that the continued use of the existing 
legacy travel systems at locations where DTS has been deployed results 
in underutilization of DTS and reduces the envisioned savings. 

Today, I will highlight three key findings from our September 2006 
report: 

* Two key assumptions related to the estimated cost savings in the 
September 2003 DTS economic analysis were not reasonable. DOD strongly 
objected to this finding, and I will discuss why we continue to believe 
that our finding is accurate. 

* The department did not have quantitative metrics to measure the 
extent to which DTS is actually being used. 

* DOD still has not addressed several functional problems associated 
with weak requirements management and system testing. 

Finally, I will discuss our recommendations to improve the department's 
management and oversight of DTS. 

Our work focused on the validity of the assumptions that were the 
principal drivers of DOD's net annual estimated savings of over $56 
million. We determined that the data were sufficiently reliable for 
this purpose. We did not review the accuracy and reliability of the 
specific dollar amounts shown in the September 2003 economic analysis. 
To address our objectives we also (1) reviewed the September 2003 
economic analysis and met with cognizant officials, (2) analyzed DTS 
utilization data and obtained an overview of the method and data used 
by DTS program officials to report the rate of DTS utilization for the 
various DOD components, and (3) analyzed DTS flight information. Our 
work was performed from October 2005 through July 2006 in accordance 
with U.S. generally accepted government auditing standards. Further 
details on our scope and methodology are included in our September 2006 
report.[Footnote 6] 

Summary: 

Our analysis of the September 2003 DTS economic analysis found that two 
key assumptions used to estimate cost savings were not based on 
reliable information. Two primary areas represented the majority of the 
over $56 million of estimated annual net savings DTS was expected to 
realize--personnel savings of $24.2 million and reduced CTO fees of $31 
million. The $24.2 million estimated annual personnel savings were 
attributed to the Air Force and Navy.[Footnote 7] However, Air Force 
and Navy DTS officials stated that they did not anticipate a reduction 
in the number of personnel with the full implementation of DTS, but 
rather the shifting of staff to other functions. Further, the Naval 
Cost Analysis Division has stated that the Navy will not realize any 
tangible personnel cost savings from the implementation of DTS. In 
written comments on a draft of our report, the Under Secretary of 
Defense (Personnel and Readiness), strongly objected to our finding 
that the estimated personnel savings are unrealistic and stated that 
recognizing fiscal constraints, the department continues to identify 
efficiencies and eliminate redundancies to help leverage available 
funds. As noted in our report, DOD officials responsible for reviewing 
economic analyses stated that while shifting personnel to other 
functions is considered a benefit, it should be considered an 
intangible benefit rather than tangible dollar savings since the 
shifting of personnel does not result in a reduction of DOD 
expenditures. Because none of the military services could validate an 
actual reduction in the number of personnel as a result of DTS 
implementation, and DOD's comments did not include any additional 
support or documentation for its position, we continue to believe that 
the estimated annual personnel savings of $54.1 million are 
unrealistic. 

In regard to the estimated annual savings of $31 million attributed to 
lower CTO fees, we requested, but the PMO-DTS could not provide, any 
analysis of travel data to support the assumption that 70 percent of 
all airline tickets would be considered "no touch"--meaning that there 
would be no or minimal intervention by the CTO, thereby resulting in 
lower CTO fees. We found that the 70 percent assumption was based 
solely upon an article that appeared in a travel industry trade 
publication. Further, the economic analysis assumed that the Navy would 
save about $7.5 million, almost 25 percent, of the total savings 
related to CTO fees once DTS is fully deployed. Again, this figure was 
based on a reduction in the fees the Navy would pay for "no touch" 
transactions. However, the Navy paid a flat management fee that was the 
same regardless of the involvement of the CTO--therefore, the reduced 
"no touch" fee would not apply. 

In addition, the economic analysis was not prepared in accordance with 
guidance prescribed by the Office of Management and Budget (OMB) and 
DOD. Both sets of guidance require that an economic analysis be based 
on facts and data and be explicit about the underlying assumptions used 
to arrive at future benefits and costs. DOD guidance also states that 
life-cycle cost estimates should be independently validated. An 
independent review is intended, in part, to provide program management 
some degree of assurance that the life-cycle cost estimates are 
reasonable and the cost estimates are built on realistic program 
assumptions. However, an independent validation was not performed. 

Our analysis also found that the department did not have quantitative 
metrics to measure the extent to which DTS is actually being used. The 
reported DTS utilization rates were based on a methodology that was 
developed using estimated data, and PMO-DTS program officials 
acknowledged that the model had not been completely updated with actual 
data as DTS continued to be implemented at the 11,000 sites. As a 
result, the PMO-DTS continues to rely on outdated information in 
calculating DTS utilization rates that are reported to DOD management 
and the Congress. Additionally, while the military services have 
initiated actions to help increase the utilization of DTS, they pointed 
out that ineffective DTS training is a contributing factor to the lower 
than expected usage rate by the military services. 

Finally, DOD still has not addressed several functional problems 
associated with weak requirements management and system testing. 
Requirements represent the blueprint that system developers and program 
managers use to design, develop, test, and implement a system. Because 
requirements provide the foundation for system testing, they must be 
complete, clear, and well documented to design and implement an 
effective testing program. Adequately defined and tested requirements 
are one of the key elements to help reduce a project's risks to 
acceptable levels.[Footnote 8] Our February 2006 analysis disclosed 
that DOD still did not have reasonable assurance that the flight 
information was being properly displayed to DOD travelers. We 
identified 246 unique GSA city pair flights that should have been 
identified on one or more DTS flight displays according to the DOD 
requirements. However, 87 of these flights did not appear on one or 
more of the required listings. While the PMO-DTS has taken action to 
address our concerns, these actions do not fully address the 
fundamental problems we found during this audit and on which we have 
previously reported.[Footnote 9] For example, the DTS requirements we 
reviewed were still ambiguous and conflicting. 

Our September 2006 report includes four recommendations to the 
Secretary of Defense aimed at improving the department's management and 
oversight of DTS. We recommended that the Secretary of Defense (1) 
evaluate the cost effectiveness of the Navy continuing with the CTO 
management fee structure, (2) update the DTS Voucher Analysis Model to 
report DTS actual utilization rates, (3) require the PMO-DTS to provide 
periodic reports on the utilization of DTS, and (4) resolve 
inconsistencies in DTS requirements. DOD generally agreed with the 
recommendations and described its efforts to address them. 

Validity of DTS Economic Analysis Questionable: 

In September 2003, DOD finalized its economic analysis for DTS in 
preparation for a milestone decision review.[Footnote 10] The 
highlights of the economic analysis are shown in table 1. In December 
2003, the DOD Chief Information Officer granted approval for DTS to 
proceed with full implementation throughout the department. 

Table 1: Summary of DTS Estimated Annual Net Savings Reported in the 
September 2003 Economic Analysis: 

Constant fiscal year 2003 dollars in millions. 

Cost Components: Records management; 
Estimated annual net savings: $19.8. 

Cost Components: Centrally billed accounts; 
Estimated annual net savings: 1.7. 

Cost Components: CTO acquisition and administration; 
Estimated annual net savings: 2.4. 

Cost Components: CTO services; 
Estimated annual net savings: 31.0. 

Cost Components: Voucher process and compute; 
Estimated annual net savings: 54.1. 

Cost Components: Voucher pay; 
Estimated annual net savings: 0. 

Cost Components: Legacy systems; 
Estimated annual net savings: 14.5. 

Cost Components: PMO; 
Estimated annual net savings: (8.8). 

Cost Components: Help desk/DTA; 
Estimated annual net savings: (36.8). 

Cost Components: System operations; 
Estimated annual net savings: (21.5). 

Cost Components: Total net savings; 
Estimated annual net savings: $56.4. 

Source: September 2003 economic analysis provided by the PMO-DTS. 

Note: In arriving at the estimated annual net savings of over $56 
million, the economic analysis took into consideration the estimated 
costs of over $2.1 billion, which covers fiscal years 2003-2016. The 
estimated costs included the costs that are estimated to be incurred by 
the PMO-DTS, the Army, the Navy, the Air Force, and the defense 
agencies. 

[End of table] 

Our analysis of the September 2003 DTS economic analysis found that two 
key assumptions used to estimate cost savings were not based on 
reliable information. Consequently, the economic analysis did not serve 
to help ensure that the funds invested in DTS were used in an efficient 
and effective manner. Two primary areas--personnel savings and reduced 
CTO fees--represented the majority of the over $56 million of estimated 
annual net savings DTS was expected to realize. However, the estimates 
used to generate these savings were unreliable. Further, DOD did not 
effectively implement the policies relating to developing economic 
analyses for programs such as DTS. Effective implementation of these 
policies should have highlighted the problems that we found and allowed 
for appropriate adjustments so that the economic analysis could have 
served as a useful management tool in making funding decisions related 
to DTS--which is the primary purpose of this analysis. While the 
department's system acquisition criteria do not require that a new 
economic analysis be prepared, the department's business system 
investment management structure provides an opportunity for DOD 
management to assess whether DTS is meeting its planned cost, schedule, 
and functionality goals. 

Personnel Savings Are Unrealistic: 

The economic analysis estimated that the annual personnel savings was 
over $54 million,[Footnote 11] as shown in table 2. 

Table 2: Summary of Estimated Annual Personnel Savings: 

Constant fiscal year 2003 dollars in millions. 

DOD component: Army; 
Estimated annual savings: $16.0. 

DOD component: Navy; 
Estimated annual savings: 12.9. 

DOD component: Air Force; 
Estimated annual savings: 11.3. 

DOD component: Marine Corps; 
Estimated annual savings: 5.8. 

DOD component: Defense agencies; 
Estimated annual savings: 6.3. 

DOD component: Permanent change of station; 
Estimated annual savings: 1.8. 

DOD component: Total savings; 
Estimated annual savings: $54.1. 

Source: September 2003 economic analysis provided by the PMO-DTS. 

[End of table] 

As shown in table 2, approximately 45 percent of the estimated savings, 
or $24.2 million, was attributable to the Air Force and Navy. The 
assumption behind the personnel savings computation was that there 
would be less manual intervention in the processing of travel vouchers 
for payment, and therefore fewer staff would be needed. However, based 
on our discussions with Air Force and Navy DTS program officials, it is 
questionable as to how the estimated savings will be achieved. Air 
Force and Navy DTS program officials stated that they did not 
anticipate a reduction in the number of personnel with the full 
implementation of DTS, but rather the shifting of staff to other 
functions. According to DOD officials responsible for reviewing 
economic analyses, while shifting personnel to other functions is 
considered a benefit, it should be considered an intangible benefit 
rather than tangible dollar savings since the shifting of personnel 
does not result in a reduction of DOD expenditures. Also, as part of 
the Navy's overall evaluation of the economic analysis, program 
officials stated that "the Navy has not identified, and conceivably 
will not recommend, any personnel billets for reduction." Finally, the 
Naval Cost Analysis Division (NCAD) October 2003 report on the economic 
analysis noted that it could not validate approximately 40 percent of 
the Navy's total costs, including personnel costs, in the DTS life- 
cycle cost estimates because credible supporting documentation was 
lacking. The report also noted that the PMO-DTS used unsound 
methodologies in preparing the DTS economic analysis. 

The extent of personnel savings for the Army and defense agencies, 
which are reported as $16 million and $6.3 million respectively, is 
also unclear. The Army and many defense agencies use the Defense 
Finance and Accounting Service (DFAS) to process their travel vouchers, 
so the personnel savings for the Army and the defense agencies were 
primarily related to reductions in DFAS's costs. In discussions with 
DFAS officials, they were unable to estimate the actual personnel 
savings that would result since they did not know (1) the number of 
personnel, like those at the Air Force and Navy, that would simply be 
transferred to other DFAS functions or (2) the number of personnel that 
could be used to avoid additional hiring. For example, DFAS expects 
that some of the individuals assigned to support the travel function 
could be moved to support its ePayroll program. Since these positions 
would need to be filled regardless of whether the travel function is 
reduced, transferring personnel from travel to ePayroll would reduce 
DOD's overall costs since DFAS would not have to hire additional 
individuals. 

DOD strongly objected to our finding that the personnel savings are 
unrealistic. In its written comments, the department stated that it is 
facing an enormous challenge and continues to identify efficiencies and 
eliminate redundancies to help leverage available funds. We fully 
recognize that the department is attempting to improve the efficiency 
and effectiveness of its business operations. The Comptroller General 
of the United States testified in August 2006 that increased commitment 
by the department to address DOD's numerous challenges represents an 
improvement over past efforts.[Footnote 12] 

The fact remains, however, that the results of an economic analysis are 
intended to help management decide if future investments in a given 
endeavor are worthwhile. In order to provide management with this 
information, it is imperative that the underlying assumptions in an 
economic analysis be supported by valid assumptions. The September 2003 
economic analysis noted that personnel savings of $54.1 million would 
be realized by the department annually for fiscal years 2009 through 
2016. However, based on our review and analysis of documentation and 
discussion with department personnel, we found that the underlying 
assumptions in support of the $54.1 million were not valid, 
particularly in regard to the amounts estimated for the Navy and Air 
Force. For example, we agree with the statements of DOD officials who 
indicated that the shifting of personnel to other functions cannot be 
counted towards tangible dollar savings, since such actions do not 
result in a reduction of DOD expenditures. Moreover, the department did 
not provide any new data or related documentation in its comments that 
were counter to our finding. As a result of these factors, we continue 
to believe that the estimated annual personnel savings of $54.1 million 
are unrealistic. 

Savings Associated with Reduction of CTO Fees Are Unknown: 

According to the September 2003 economic analysis, DOD expected to 
realize annual net savings of $31 million through reduced fees paid to 
the CTOs because the successful implementation of DTS would enable the 
majority of airline tickets to be acquired with either no or minimal 
intervention by the CTOs. These are commonly referred to as "no touch" 
transactions. However, DOD did not have a sufficient basis to estimate 
the number of transactions that would be considered "no touch" since 
the (1) estimated percentage of transactions that can be processed 
using the "no touch" was not supported and (2) analysis did not 
properly consider the effects of components that use management fees, 
rather than transaction fees, to compensate the CTOs for services 
provided. The weaknesses we identified with the estimating process 
raise serious questions as to whether DOD will realize substantial 
portions of the estimated annual net savings of $31 million. 

"No Touch" Transaction Volume Estimates Are Not Supported: 

DOD arrived at the $31 million of annual savings in CTO fees by 
estimating that 70 percent of all DTS airline tickets would be 
considered "no touch" and then multiplying these tickets by the savings 
per ticket in CTO fees. However, a fundamental flaw in this analysis 
was that the 70 percent assumption had no solid basis. We requested, 
but the PMO-DTS could not provide, any analysis of travel data to 
support the assertion. Rather, the sole support provided by the PMO-DTS 
was an article in a travel industry trade publication.[Footnote 13] The 
article was not based on information related to DTS, but rather on the 
experience of one private sector company. 

The economic analysis assumed that DOD could save about $13.50 per "no 
touch" ticket. Since that analysis, DOD has awarded one contract that 
specifically prices transactions using the same model as that 
envisioned by the economic analysis. This contract applies to the 
Defense Travel Region 6 travel area.[Footnote 14] During calendar year 
2005, the difference in fees for "no touch" transactions and the 
transactions supported by the current process averaged between $10 and 
$12, depending on when the fees were incurred because the contract 
rates changed during 2005.[Footnote 15] In analyzing travel voucher 
data for Region 6 for calendar year 2005, we found that the reported 
"no touch" rate was, at best 47 percent--far less than the 70 percent 
envisioned in the economic analysis. 

PMO-DTS program officials stated they are uncertain as to why the 
anticipated 70 percent "no touch" was not being achieved. According to 
PMO-DTS program officials, this could be attributed, in part, to the 
DOD travelers being uncomfortable with the system and with making 
reservations without using a CTO. Although this may be one reason, 
other factors may also affect the expected "no touch" fee. For example, 
we were informed that determining the airline availability and making 
the associated reservation can be accomplished, in most cases, rather 
easily. However, obtaining information related to hotels and rental 
cars and making the associated reservation can be more problematic 
because of the limitations in the data that DTS is able to obtain from 
its commercial sources. Accordingly, while a traveler may be able to 
make a "no touch" reservation for the airline portion of the trip, the 
individual may need to contact the CTO in order to make hotel or rental 
car reservations. When this occurs, rather than paying a "no touch" fee 
to the CTO, DOD ends up paying a higher fee, which eliminates the 
savings estimated in the economic analysis. 

The economic analysis assumed that (1) DOD would be able to modify the 
existing CTO contracts to achieve a substantial reduction in fees paid 
to a CTO when DTS was fully implemented across the department and (2) 
all services would use the fee structure called for in the new CTO 
contracts. The first part of the assumption is supported by results of 
the CTO contract for DOD Region 6 travel. The fees for the DTS "no 
touch" transactions were at least $10 less than if a CTO was involved 
in the transactions. However, to date, the department has experienced 
difficulty in awarding new contracts with the lower fee structure. On 
May 10, 2006, the department announced the cancellation of the 
solicitation for a new contract. According to the department, it 
decided that the solicitation needed to be rewritten based on feedback 
from travel industry representatives at a March 28, 2006, conference. 
The department acknowledged that the "DTS office realized its 
solicitation didn't reflect what travel agency services it actually 
needed."[Footnote 16] The department would not say how the solicitation 
would be refined, citing the sensitivity of the procurement process. 
The department also noted that the new solicitation would be released 
soon, but provided no specific date. 

Navy Impact of CTO Management Fees Not Adequately Considered: 

The economic analysis assumed that the Navy would save about $7.5 
million, almost 25 percent, of the total savings related to CTO fees 
once DTS is fully deployed. The economic analysis averaged the CTO fees 
paid by the Army, the Air Force, and the Marine Corps--which amounted 
to about $18.71 per transaction--to compute the savings in Navy CTO 
fees. Using these data, the assumption was made in the economic 
analysis that a fee of $5.25 would be assessed for each ticket, 
resulting in an average savings of $13.46 per ticket for the Navy 
($18.71 minus $5.25).[Footnote 17] While this approach may be valid for 
the organizations that pay individual CTO fees, it may not be 
representative for organizations such as the Navy that pay a management 
fee. The management fee charged the Navy is the same regardless of the 
involvement of the CTO--therefore, the reduced "no touch" fee would not 
apply. 

We were informed by Navy DTS program officials that they were 
considering continuing the use of management fees after DTS is fully 
implemented. According to Navy DTS program officials, they paid about 
$14.5 million during fiscal year 2005 for CTO management fees, almost 
$19 per ticket for approximately 762,700 tickets issued. Accordingly, 
even if the department arrives at a new CTO contract containing the new 
fee structure or fees similar to those of Region 6, the estimated 
savings related to CTO fees for the Navy will not be realized if the 
Navy continues to use the management fee concept. 

Effective Implementation of Existing Policies Should Have Identified 
Problems with the Economic Analysis: 

Effective implementation of DOD guidance would have detected the types 
of problems discussed above and resulted in an economic analysis that 
would have accomplished the stated objective of the process--to help 
ensure that the funds invested in DTS were used efficiently and 
effectively. DOD policy[Footnote 18] and OMB guidance[Footnote 19] 
require that an economic analysis be based on facts and data and be 
explicit about the underlying assumptions used to arrive at estimates 
of future benefits and costs. Since an economic analysis deals with 
costs and benefits occurring in the future, assumptions must be made to 
account for uncertainties. DOD policy recognizes this and provides a 
systematic approach to the problem of choosing the best method of 
allocating scarce resources to achieve a given objective. 

A sound economic analysis recognizes that there are alternative ways to 
meet a given objective and that each alternative requires certain 
resources and produces certain results. The purpose of the economic 
analysis is to give the decision maker insight into economic factors 
bearing on accomplishing the objectives. Therefore, it is important to 
identify factors, such as cost and performance risks and drivers, that 
can be used to establish and defend priorities and resource 
allocations. The DTS economic analysis did not comply with the DOD 
policy, and the weaknesses we found should have been detected had the 
DOD policy been effectively implemented. The PMO-DTS had adequate 
warning signs of the potential problems associated with not following 
the OMB and DOD guidance for developing an effective economic analysis. 
For example, as noted earlier, the Air Force and Navy provided comments 
when the economic analysis was being developed that the expected 
benefits being claimed were unrealistic. Just removing the benefits 
associated with personnel savings from the Air Force and Navy would 
have reduced the overall estimated program cost savings by almost 45 
percent. This would have put increased pressure on the credibility of 
using a 70 percent "no touch" utilization rate. 

Specific examples of failures to effectively implement the DOD policy 
on conducting economic analyses include the (1) DTS life-cycle cost 
estimates portion of the economic analysis was not independently 
validated as specified in DOD's guidance[Footnote 20] and (2) September 
2003 DTS economic analysis did not undertake an assessment of the 
effects of the uncertainty inherent in the estimates of benefits and 
costs, as required by DOD and OMB guidance.[Footnote 21] Because an 
economic analysis uses estimates and assumptions, it is critical that a 
sensitivity analysis[Footnote 22] be performed to understand the 
effects of the imprecision in both underlying data and modeling 
assumptions. 

DTS Remains Underutilized by the Military Services: 

Our September 2005 testimony and January 2006 report[Footnote 23] noted 
the challenge facing the department in attaining the anticipated DTS 
utilization. While DOD has acknowledged the underutilization, we found 
that, across DOD, the department does not have reasonable quantitative 
metrics to measure the extent to which DTS is actually being used. 
Presently, the reported DTS utilization is based on a DTS Voucher 
Analysis Model[Footnote 24] that was developed in calendar year 2003 
using estimated data, but over the years has not been completely 
updated with actual data. While the military services have initiated 
actions to help increase the utilization of DTS, they pointed out that 
ineffective DTS training is a contributing factor to the lower than 
expected usage rate by the military services. 

Metrics to Measure DTS Utilization Are Inadequate: 

The DTS Voucher Analysis Model was prepared in calendar year 2003 and 
based on airline ticket and voucher count data that were reported by 
the military services and defense agencies, but the data were not 
verified or validated. Furthermore, PMO-DTS officials acknowledged that 
the model has not been completely updated with actual data as DTS 
continues to be implemented at the 11,000 sites. We found that the Air 
Force is the only military service that submits monthly metrics to the 
PMO-DTS officials for their use in updating the DTS Voucher Analysis 
Model. Rather than reporting utilization based on individual site 
system utilization data, the PMO-DTS continues to rely on outdated 
information in the reporting of DTS utilization to DOD management and 
the Congress. We have previously reported[Footnote 25] that best 
business practices indicate that a key factor of project management and 
oversight is the ability to effectively monitor and evaluate a 
project's actual performance against what was planned. 

In order to perform this critical task, best business practices require 
the adoption of quantitative metrics to help measure the effectiveness 
of a business system implementation and to continually measure and 
monitor results, such as system utilization. This lack of accurate and 
pertinent utilization data hinders management's ability to monitor its 
progress toward the DOD vision of DTS as the standard travel system, as 
well as to provide consistent and accurate data to Congress. With the 
shift of the DTS program to the Business Transformation Agency 
(BTA),[Footnote 26] which now makes DTS an enterprisewide endeavor, 
improved metrics and training are essential if DTS is to be DOD's 
standard, integrated, end-to-end travel system for business travel. 

DTS's reported utilization rates for the period October 2005 through 
April 2006 averaged 53 percent for Army, 30 percent for Navy, and 39 
percent for Air Force. Because the PMO-DTS was not able to identify the 
total number of travel vouchers that should have been processed through 
DTS (total universe of travel vouchers), these utilization rates may be 
over-or understated. PMO-DTS program officials confirmed that the 
reported utilization data were not based on complete data because the 
department did not have comprehensive information to identify the 
universe or the total number of travel vouchers that should be 
processed through DTS. PMO-DTS program and DTS military service 
officials agreed that the actual DTS utilization rate should be 
calculated by comparing actual vouchers being processed in DTS to the 
total universe of vouchers that should be processed in DTS. The 
universe would exclude those travel vouchers that cannot be processed 
through DTS, such as those related to permanent change of station 
travel. 

The Air Force was the only military service that attempted to obtain 
data on (1) the actual travel vouchers processed through DTS and (2) 
those travel vouchers that were eligible to be processed through DTS, 
but were not. These data were site-specific. For example, during the 
month of December 2005, the PMO-DTS reported that at Wright-Patterson 
Air Force Base, 2,880 travel vouchers were processed by DTS, and the 
Air Force reported that another 2,307 vouchers were processed through 
the legacy system--the Reserve Travel System (RTS). Of those processed 
through RTS, Air Force DTS program officials stated that 338 travel 
vouchers should have been processed through DTS. DTS Air Force program 
officials further stated that they submitted to the PMO-DTS the number 
of travel vouchers processed through RTS each month. These data are 
used by the PMO-DTS to update the DTS Voucher Analysis Model. However, 
neither the Air Force nor the PMO-DTS have verified the accuracy and 
reliability of the data. Therefore, the accuracy of the utilization 
rates reported for the Air Force by the PMO-DTS is not known. 

Because Army and Navy DTS program officials did not have the 
information to identify the travel transactions that should have been 
processed through DTS, the Army and Navy did not have a basis for 
evaluating DTS utilization at their respective military locations and 
activities. Furthermore, Navy DTS program officials indicated that the 
utilization data that the PMO-DTS program officials reported for the 
Navy were not accurate. According to Navy DTS program officials, the 
Navy's primary source of utilization data was the monthly metrics 
reports provided by the PMO-DTS, but Navy DTS program officials 
questioned the accuracy of the Navy utilization reports provided by the 
PMO-DTS. 

DOD Has Taken Steps to Improve DTS Utilization, but Further Action Is 
Needed: 

Although the military services have issued various memorandums aimed at 
increasing the utilization of DTS, the military service DTS program 
officials all pointed to ineffective training as a primary cause of DTS 
not being utilized to a far greater extent. The following examples 
highlight the concerns raised by the military service officials: 

* Army DTS program officials emphasized that the DTS system is complex 
and the design presents usability challenges for users--especially for 
first-time or infrequent users. They added that a major concern is that 
there is no PMO-DTS training for existing DTS users as new 
functionality is added to DTS. These officials stated that the PMO-DTS 
does not do a good job of informing users about functionality changes 
made to the system. We inquired if the Help Desk was able to resolve 
the users' problems, and the Army DTS officials simply stated "no." The 
Army officials further pointed out that it would be beneficial if the 
PMO-DTS improved the electronic training on the DTS Web site and made 
the training documentation easier to understand. Also, improved 
training would help infrequent users adapt to system changes. The Army 
officials noted that without some of these improvements to resolve 
usability concerns, DTS will continue to be extremely frustrating and 
cumbersome for travelers. 

* Navy DTS program officials stated that DTS lacks adequate user/ 
traveler training. The train-the-trainer concept of training system 
administrators who could then effectively train all their travelers has 
been largely unsuccessful. According to Navy officials, this has 
resulted in many travelers and users attempting to use DTS with no or 
insufficient training. The effect has frustrated users at each step of 
the travel process and has discouraged use of DTS. 

* Air Force officials stated that new DTS system releases are 
implemented with known problems, but the sites are not informed of the 
problems. Workarounds are not provided until after the sites begin 
encountering problems. Air Force DTS program officials stated that DTS 
releases did not appear to be well tested prior to implementation. Air 
Force officials also stated that there was insufficient training on new 
functionality. PMO-DTS and DTS contractor program officials believed 
that conference calls to discuss new functionality with the sites were 
acceptable training, but Air Force officials did not agree. The Air 
Force finance office was expected to fully comprehend the information 
received from those conference calls and provide training on the new 
functionality to users/approvers, but these officials stated that this 
was an unrealistic expectation. 

As discussed in our September 2005 testimony and January 2006 
report,[Footnote 27] the unnecessary continued use of the legacy travel 
systems results in the inefficient use of funds because the department 
is paying to operate and maintain duplicative systems that perform the 
same function--travel. 

Previously Reported DTS Requirements Management and Testing 
Deficiencies Have Not Been Resolved: 

Our September 2005 testimony and January 2006 report noted problems 
with DTS's ability to properly display flight information and traced 
those problems to inadequate requirements management and testing. DOD 
stated that it had addressed those deficiencies, and in February 2006, 
we again tested the system to determine whether the stated weaknesses 
had been addressed. We found that similar problems continue to exist. 
Once again, these problems can be traced to ineffective requirements 
management and testing processes. Properly defined requirements are a 
key element in systems that meet their cost, schedule, and performance 
goals since the requirements define the (1) functionality that is 
expected to be provided by the system and (2) quantitative measures by 
which to determine through testing whether that functionality is 
operating as expected. 

We briefed PMO-DTS officials on the results of our tests and in May 
2006 the officials agreed that our continued concerns about the proper 
display of flight information were valid. PMO-DTS officials stated that 
the DTS technology refresh, which was to be completed in September 
2006, should address some of our concerns. While these actions are a 
positive step forward, they do not address the fundamental problem that 
DTS's requirements are still ambiguous and conflicting--a primary cause 
of the previous problems. Until a viable requirements management 
process is developed and effectively implemented, the department (1) 
cannot develop an effective testing process and (2) will not have 
reasonable assurance the project risks have been reduced to acceptable 
levels. 

Providing Complete Flight Information Has Been a Continuing Problem: 

In our earlier testimony and report,[Footnote 28] we noted that DOD did 
not have reasonable assurance that the flights displayed met the stated 
DOD requirements. Although DOD stated in each case that our concerns 
had been addressed, subsequent tests found that the problems had not 
been corrected. Requirements represent the blueprint that system 
developers and program managers use to design, develop, and acquire a 
system. Requirements should be consistent with one another, verifiable, 
and directly traceable[Footnote 29] to higher-level business or 
functional requirements. It is critical that requirements be carefully 
defined and that they flow directly from the organization's concept of 
operations (how the organization's day-to-day operations are or will be 
carried out to meet mission needs). Improperly defined or incomplete 
requirements have been commonly identified as a cause of system failure 
and systems that do not meet their cost, schedule, or performance 
goals. 

Requirements represent the foundation on which the system should be 
developed and implemented. As we have noted in previous 
reports,[Footnote 30] because requirements provide the foundation for 
system testing, significant defects in the requirements management 
process preclude an entity from implementing a disciplined testing 
process. That is, requirements must be complete, clear, and well 
documented to design and implement an effective testing program. Absent 
this, an organization is taking a significant risk that its testing 
efforts will not detect significant defects until after the system is 
placed into production. Our February 2006 analysis of selected flight 
information disclosed that DOD still did not have reasonable assurance 
that DTS displayed flights in accordance with its stated requirements. 
We analyzed 15 U.S. General Services Administration (GSA) city 
pairs,[Footnote 31] which should have translated into 246 GSA city pair 
flights for the departure times selected. However, we identified 87 
flights that did not appear on one or more of the required listings 
based on the DTS requirements. For instance, our analysis identified 44 
flights appearing on other DTS listings or airline sites that did not 
appear on the 9:00 am DTS listing even though those flights (1) met the 
12-hour flight window[Footnote 32] and (2) were considered GSA city 
pair flights--two of the key DTS requirements the system was expected 
to meet. 

After briefing PMO officials on the results of our analysis in February 
2006, the PMO-DTS employed the services of a contractor to review DTS 
to determine the specific cause of the problems and recommend 
solutions. In a March 2006 briefing, the PMO-DTS acknowledged the 
existence of the problems, and identified two primary causes. First, 
part of the problem was attributed to the methodology used by DTS to 
obtain flights from the Global Distribution System (GDS). The PMO-DTS 
stated that DTS was programmed to obtain a "limited" amount of data 
from GDS in order to reduce the costs associated with accessing GDS. 
This helps to explain why flight queries we reviewed did not produce 
the expected results. To resolve this particular problem, the PMO-DTS 
proposed increasing the amount of data obtained from GDS. Second, the 
PMO-DTS acknowledged that the system testing performed by the 
contractor responsible for developing and operating DTS was inadequate 
and, therefore, there was no assurance that DTS would provide the data 
in conformance with the stated requirements. This weakness was not new, 
but rather reconfirms the concerns discussed in our September 2005 
testimony and January 2006 report[Footnote 33] related to the testing 
of DTS. 

DOD's Planned Corrective Actions Will Not Address Fundamental 
Requirements Management Problems: 

While DOD's planned actions, including a recent technology upgrade, 
should address several of the specific weaknesses we identified related 
to flight displays, they fall short of addressing the fundamental 
problems that caused those weaknesses--inadequate requirements 
management. DTS's requirements continue to be ambiguous. For example, 
DOD has retained a requirement to display 25 flights for each inquiry. 
However, it has not determined (1) whether the rationale for that 
requirement is valid and (2) under what conditions flights that are not 
part of the GSA city pair program should be displayed. For example, we 
found that several DTS flights displayed to the user "overlap"[Footnote 
34] other flights. Properly validating the requirements would allow DOD 
to obtain reasonable assurance that its requirements properly define 
the functionality needed and the business rules necessary to properly 
implement that functionality. As previously noted, requirements that 
are unambiguous and consistent are fundamental to providing reasonable 
assurance that a system will provide the desired functionality. Until 
DOD improves DTS requirement management practices, it will not have 
this assurance. 

Recommendations to Improve DTS Management and Oversight: 

Our recent report[Footnote 35] included four recommendations to improve 
the department's management and oversight of DTS. We recommended that 
DOD (1) evaluate the cost effectiveness of the Navy continuing with the 
CTO management fee structure versus adopting the revised CTO fee 
structure, once the new contracts have been awarded, (2) develop a 
process by which the military services develop and use quantitative 
data from DTS and their individual legacy systems to clearly identify 
the total universe of DTS-eligible transactions on a monthly basis, (3) 
require the PMO-DTS to provide periodic reports on the utilization of 
DTS, once accurate data are available, and (4) resolve inconsistencies 
in DTS requirements by properly defining the functionality needed and 
business rules necessary to properly implement the needed 
functionality. DOD concurred with three and partially concurred with 
one of the recommendations. In regard to the recommendations with which 
the department concurred, it briefly outlined the actions it planned to 
take in addressing two of the three recommendations. For example, the 
department noted the difficulties in obtaining accurate utilization 
data from the existing legacy systems, but stated that the Office of 
the Under Secretary of Defense (Personnel and Readiness) and BTA will 
evaluate methods for reporting actual DTS utilization. 

Additionally, DOD noted that the Defense Travel Management Office 
developed and implemented a requirements change management process on 
May 1, 2006. In commenting on the report, the department stated that 
this process is intended to define requirements and track the entire 
life cycle of the requirements development process. While we fully 
support the department's efforts to improve its management oversight of 
DTS's requirements, we continue to believe that the department needs to 
have in place a process that provides DOD reasonable assurance that (1) 
requirements are properly documented and (2) requirements are 
adequately tested as recommended in our January 2006 report.[Footnote 
36] This process should apply to all existing requirements as well as 
any new requirements. As discussed in this report, we reviewed in May 
2006 some of the requirements that were to have followed the new 
requirements management process and found problems similar to those 
noted in our January 2006 report. Although we did not specifically 
review the new process, if it does not include an evaluation of 
existing requirements, the department may continue to experience 
problems similar to those we previously identified. 

DOD partially concurred with our recommendation to evaluate the cost 
effectiveness of the Navy continuing with the CTO management fee 
structure. However, DOD's response indicated that the Defense Travel 
Management Office is currently procuring commercial travel services for 
DOD worldwide in a manner that will ensure evaluation of cost 
effectiveness for all services. If DOD proceeds with the actions 
outlined in its comments, it will meet the intent of our 
recommendation. 

Effective implementation of these recommendations as well as those 
included in our January 2006 report[Footnote 37] will go a long way 
towards improving DTS functionality and increasing utilization. 
Furthermore, the shift of DTS to the BTA, which makes DTS an 
enterprisewide endeavor, should help in making DTS the standard 
integrated, end-to-end travel system for business travel. Management 
oversight is essential for this to become a reality. As I stated 
previously, in written comments on a draft of our report, the Under 
Secretary of Defense (Personnel and Readiness), strongly objected to 
our finding that the estimated personnel savings included in the 
economic analysis are unrealistic. Because none of the military 
services could validate an actual reduction in the number of personnel 
as a result of DTS implementation, and DOD's comments did not include 
any additional support or documentation for its position, we continue 
to believe that the estimated annual personnel savings of $54.1 million 
are unrealistic. Although the department's criteria do not require that 
a new economic analysis be prepared, the fiscal year 2005 defense 
authorization act[Footnote 38] requires the periodic review, but not 
less than annually, of every defense business system investment. If 
effectively implemented, this annual review process provides an 
excellent opportunity for DOD management to assess whether DTS is 
meeting its planned cost, schedule, and functionality goals. Going 
forward, such a review could serve as a useful management tool in 
making funding and other management decisions related to DTS. 

In conclusion, overhauling the department's antiquated travel 
management practices and systems has been a daunting challenge for DOD. 
While it was widely recognized that this was a task that needed to be 
accomplished and savings could result, the underlying assumptions in 
support of those savings are not based on reliable data and therefore 
it is questionable whether the anticipated savings will materialize. 
Even though the overall savings are questionable, the successful 
implementation of DTS is critical to reducing the number of stovepiped, 
duplicative travel systems throughout the department. We have reported 
on numerous occasions that reducing the number of business systems 
within DOD can translate into savings that can be used for other 
mission needs. As noted above, management oversight will be an 
important factor in DTS achieving its intended goals. Equally 
important, however, will be the department's ability to resolve the 
long-standing difficulties that DTS has encountered with its 
requirements management and system testing. Until these issues are 
resolved, more complete utilization of DTS will be problematic. 

Mr. Chairman, this concludes my prepared statement. We would be happy 
to answer any questions that you or other members of the Subcommittee 
may have at this time. 

Contacts and Acknowledgments: 

For further information about this testimony, please contact McCoy 
Williams at (202) 512-9095 or williamsm1@gao.gov, or Keith A. Rhodes at 
(202) 512-6412 or rhodesk@gao.gov. 

Contact points for our Offices of Congressional Relations and Public 
Affairs may be found on the last page of this report. In addition to 
the above contacts, the following individuals made key contributions to 
this testimony: Darby Smith, Assistant Director; J. Christopher Martin, 
Senior-Level Technologist; F. Abe Dymond, Assistant General Counsel; 
Beatrice Alff; Harold Brumm, Jr; Francine DelVecchio; and Tarunkant 
Mithani. 

FOOTNOTES 

[1] GAO, Defense Travel System: Reported Savings Questionable and 
Implementation Challenges Remain, GAO-06-980 (Washington, D.C.: Sept. 
26, 2006). 

[2] DOD expects DTS to perform all functions related to travel or 
ensure that other systems are provided with adequate information to 
provide this functionality. For example, obligating funds associated 
with travel is a necessary function, and DTS is expected to (1) make 
sure that adequate funds are available before authorizing travel either 
through information contained in its system or by obtaining the 
necessary information from another system, (2) obligate funds through 
issuance of approved travel orders, and (3) provide DOD's financial 
management systems with the necessary information so that those systems 
can record the obligation. Since DTS is required to ensure that all 
travel-related functionality is properly performed, DOD commonly refers 
to DTS as an "end-to-end travel system." 

[3] As of September 2005, the department had estimated that DTS would 
be fully deployed during fiscal year 2006. 

[4] GAO-06-980. 

[5] GAO, DOD Business Transformation: Defense Travel System Continues 
to Face Implementation Challenges, GAO-06-18 (Washington, D.C.: Jan. 
18, 2006), and DOD Business Transformation: Preliminary Observations on 
the Defense Travel System, GAO-05-998T (Washington, D.C.: Sept. 29, 
2005). 

[6] GAO-06-980. 

[7] The economic analysis identified annual savings of $11.3 million 
and $12.9 million for the Air Force and Navy, respectively. 

[8] Acceptable levels refer to the fact that any systems acquisition 
effort will have risks and will suffer the adverse consequences 
associated with defects in the processes. However, effective 
implementation of disciplined processes, which includes project 
planning and management, requirements management, risk management, 
quality assurance, and testing, reduces the possibility of the 
potential risks actually occurring and prevents significant defects 
from materially affecting the cost, timeliness, and performance of the 
project. 

[9] GAO-05-998T and GAO-06-18. 

[10] The September 2003 economic analysis is an addendum to the July 
2003 DTS economic analysis. 

[11] During fiscal years 2009 through 2016. 

[12] GAO, Department of Defense: Sustained Leadership Is Critical to 
Effective Financial and Business Management Transformation, GAO-06-
1006T (Washington, D.C.: Aug. 3, 2006). 

[13] American Express News Releases: American Express' Interactive 
Travel Update, (New York, N.Y.: Aug. 11, 2003), 
http://corp.americanexpress.com/gcs/cards/us/ni/pr/081303.aspx. 

[14] Defense Travel Region 6 includes the Air Force and defense 
agencies in the states of Kentucky, Illinois, Indiana, Iowa, Michigan, 
Minnesota, Missouri, Nebraska, North Dakota, South Dakota, and 
Wisconsin. The contract also applies to Army activities in 8 of the 11 
states (excluding Kentucky, Missouri, and Nebraska). As discussed 
later, the Navy uses a management fee contract, and is therefore not 
included in the Defense Travel Region 6 contract. 

[15] According to DTS officials, these savings are consistent with the 
DTS contracts that have been awarded to small businesses. The average 
savings per "no touch" ticket under these contracts is about $12.88. 
Because the contractors are paid these fees directly by the traveler, 
they are unable to determine the percentage of transactions that are 
actually paid using the "no touch" rate. 

[16] "DOD Retracts Solicitation for Travel Agency Services," 
FederalTimes.com (May 16, 2006), http://www.federaltimes.com/index.php? 
(downloaded June 14, 2006). 

[17] These savings translate to about 572,000 tickets annually. 

[18] DOD Instruction 7041.3, Economic Analysis for Decisionmaking, 
November 7, 1995. 

[19] Office of Management and Budget, Circular No. A-94, Guidelines and 
Discount Rates for Benefit-Cost Analysis of Federal Programs (Revised 
Jan. 18, 2006). 

[20] Department of Defense Instruction 5000.2, Operation of the Defense 
Acquisition System, May 12, 2003. 

[21] Department of Defense Instruction 7041.3, Economic Analysis of 
Decisionmaking, (Nov. 7, 1995), and Office of Management and Budget 
Revised Circular No. A-94, Guidelines and Discount Rates for Benefit- 
Cost Analysis of Federal Programs (Oct. 29, 1992). 

[22] Sensitivity analysis refers to changing the value of a given 
variable in a model to gauge the effect of change on model results. 

[23] GAO-05-998T and GAO-06-18. 

[24] DOD developed a model in calendar year 2003 that compares the 
expected usage against the actual usage. The expected usage is obtained 
by using historical data, such as ticket counts, to determine the 
expected number of vouchers processed by a given location. For example, 
if a location had 1,000 vouchers as its expected number of vouchers per 
the model, but now processes 750 actual vouchers through DTS, then the 
PMO model considers that that location has achieved a 75 percent 
utilization rate. It then takes the individual computations for each 
DTS location and "rolls them up" to determine the total utilization for 
individual service performance on a monthly basis. 

[25] GAO, Financial Management Systems: Additional Efforts Needed to 
Address Key Causes of Modernization Failures, GAO-06-184 (Washington, 
D.C.: Mar. 15, 2006), and Financial Management Systems: Lack of 
Disciplined Processes Puts Implementation of HHS' Financial System at 
Risk, GAO-04-1008 (Washington, D.C.: Sept. 23, 2004). 

[26] In October 2005, DOD established BTA to advance DOD-wide business 
transformation efforts, particularly with regard to business systems 
modernization. BTA operates under the authority, direction, and control 
of the Under Secretary of Defense for Acquisition, Technology, and 
Logistics, who is the vice chair of the Defense Business Systems 
Management Committee--which serves as the highest ranking governing 
body for business systems modernization activities. 

[27] GAO-05-998T and GAO-06-18. 

[28] GAO-05-998T and GAO-06-18. 

[29] Traceability allows the user to follow the life of the requirement 
both forward and backward through these documents and from origin 
through implementation. Traceability is also critical to understanding 
the parentage, interconnections, and dependencies among the individual 
requirements. This information in turn is critical to understanding the 
impact when a requirement is changed or deleted. 

[30] See, for example, GAO-04-1008 and Army Depot Maintenance: 
Ineffective Oversight of Depot Maintenance Operations and System 
Implementation Efforts, GAO-05-441 (Washington, D.C.: June 30, 2005). 

[31] GSA awards contracts to airlines to provide flight services 
between pairs of cities. This is commonly referred to as the GSA city 
pair program. Under this program (1) no advanced ticket purchases are 
required, (2) no minimum or maximum length of stay is required, (3) 
tickets are fully refundable and no charges are assessed for 
cancellations or changes, (4) seating is not capacity controlled (i.e., 
as long as there is a coach-class seat on the plane, the traveler may 
purchase it), (5) no blackout dates apply, (6) fare savings average 70 
percent over regular walk-up fares, and (7) fares are priced on one-way 
routes permitting agencies to plan for multiple destinations. We 
selected the first 15 city pairs that were provided by DOD to GSA in 
support of a GSA study on accuracy of flight displays and fare 
information by DTS and the GSA eTravel providers. 

[32] A flight window is the amount of time before and after a specified 
time and is used for determining the flights that should be displayed. 
DTS uses a 12-hour flight window for domestic flights and a 24-hour 
flight window for foreign flights. The system is also expected to 
display up to 25 flights for the flight window. 

[33] GAO-05-998T and GAO-06-18. 

[34] For example, DTS displayed a GSA city pair flight between 
Washington, D.C., and Atlanta, Georgia, that departed at 10:05 a.m. and 
arrived at 1:50 p.m. This flight "overlapped" two other GSA city pair 
direct flights that were available and required less travel time. One 
flight left at 10:05 a.m. and arrived at 12:02 p.m. while another left 
at 11:05 a.m. and arrived at 12:56 p.m. Furthermore, DTS displayed a 
non-GSA city pair flight that left at 9:20 a.m. and arrived at 1:05 
p.m. This flight did not meet any of the acceptable criteria for not 
using a GSA city pair flight. 

[35] GAO-06-980. 

[36] GAO-06-18. 

[37] GAO-06-18. 

[38] Ronald W. Reagan National Defense Authorization Act for Fiscal 
Year 2005, Pub. L. No. 108-375, § 332, 118 Stat. 1811, 1851-56 (Oct. 
28, 2004) (codified, in part, at 10 U.S.C. §§ 186, 2222). 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site (www.gao.gov). Each weekday, GAO posts 
newly released reports, testimony, and correspondence on its Web site. 
To have GAO e-mail you a list of newly posted products every afternoon, 
go to www.gao.gov and select "Subscribe to Updates." 

Order by Mail or Phone: 

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to: 

U.S. Government Accountability Office 441 G Street NW, Room LM 
Washington, D.C. 20548: 

To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 
512-6061: 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 U.S. 
Government Accountability Office, 441 G Street NW, Room 7125 
Washington, D.C. 20548: 

Public Affairs: 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 
U.S. Government Accountability Office, 441 G Street NW, Room 7149 
Washington, D.C. 20548: