This is the accessible text file for GAO report number GAO-09-669R 
entitled 'Questions for the Record Related to the Implementation of the 
Department of Defense's National Security Personnel System' which was 
released on May 18, 2009. 

This text file was formatted by the U.S. Government Accountability 
Office (GAO) to be accessible to users with visual impairments, as part 
of a longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov. 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

GAO-09-669R: 

United States Government Accountability Office: 
Washington, DC 20548: 

May 18, 2009: 

The Honorable Solomon Ortiz:
Chairman:
Subcommittee on Readiness:
Committee on Armed Services:
House of Representatives: 

Subject: Questions for the Record Related to the Implementation of the 
Department of Defense's National Security Personnel System: 

It was a pleasure to appear before your subcommittee on April 1, 2009, 
to discuss the Department of Defense's (DOD) implementation of its new 
human capital system for managing civilian personnel--the National 
Security Personnel System (NSPS).[Footnote 1] This letter responds to 
your request that I provide answers to questions for the record from 
the hearing. The questions, along with my responses, follow. 

Questions from Chairman Ortiz: 

1. GAO noted that NSPS was implemented too quickly. 

* What steps could have been taken to roll out NSPS in a more orderly 
and fair fashion? 

As we have previously reported, we support the need to expand broad 
banding approaches and pay-for-performance-based systems in the federal 
government.[Footnote 2] However, moving too quickly or prematurely to 
implement such programs, whether at DOD or elsewhere, can significantly 
raise the risk of doing it incorrectly. Hasty implementation could also 
set back the legitimate need to move to a more performance-and results- 
based system for the federal government as a whole. Thus, while it is 
imperative that we take steps to better link employee pay to 
performance across the federal government, how it is done, when it is 
done, and the basis on which it is done can make all the difference in 
whether or not such efforts are successful. In our view, one key need 
is to modernize performance management systems in executive agencies so 
that they are capable of adequately supporting more performance-based 
pay and other personnel decisions. 

While our previous work does not prescribe a process and time frames 
for rolling out systems such as NSPS, we have stressed that agencies 
should have an institutional infrastructure in place that would 
include, at a minimum, (1) a human capital planning process that 
integrates the agency's human capital policies, strategies, and 
programs with its program goals and mission and desired outcomes; (2) 
the capabilities to effectively develop and implement a new human 
capital system; and (3) the existence of a modern, effective, and 
credible performance management system that includes adequate 
safeguards, including reasonable transparency and appropriate 
accountability mechanisms, to ensure the fair, effective, and 
nondiscriminatory implementation of a new system. Prior to NSPS 
implementation, we cautioned that, while the DOD leadership had the 
intent and the ability to implement the needed infrastructure, it did 
not have the necessary infrastructure in place across the department. 

Further, our work has continued to stress the importance of 
incorporating internal safeguards into the design and implementation of 
large-scale pay-for-performance programs. In 2008, we evaluated DOD's 
efforts to implement nine safeguards and accountability mechanisms. 
[Footnote 3] We found that, while DOD had taken some steps to implement 
internal safeguards to ensure that NSPS is fair, effective, and 
credible, the implementation of some safeguards could be improved. 
First, DOD does not require a third party to analyze rating results for 
anomalies prior to finalizing employee ratings, and therefore it is 
unable to ensure that ratings are fair and nondiscriminatory before 
they are finalized. Second, the process has lacked transparency until 
recently because DOD did not require commands to publish final rating 
distributions, though doing so was recognized as a best practice by 
NSPS program officials at all four components. In 2008, the department 
revised its NSPS regulations and guidance to require commands to 
publish the final overall rating results. Third, NSPS guidance may 
discourage rating officials from making meaningful distinctions in 
employee ratings because it indicated that the majority of employees 
should be rated at the "3" level, on a scale of 1 to 5, resulting in a 
hesitancy to award ratings in other categories. We continue to believe 
that improved implementation of these safeguards will help bolster 
employee confidence in the system and ensure that the system is fair, 
effective, and credible. 

* If NSPS continues, what steps should now be taken to move forward? 

We have previously reported that converting to NSPS was a significant 
transition for the department.[Footnote 4] We have further reported 
that it will take time for employees to accept the system, based on the 
studies conducted by the Office of Personnel Management (OPM) on the 
federal government demonstration projects for performance management. 
First, and foremost, DOD needs to assess and address employee 
engagement in the system. DOD has collected survey data and conducted 
focus groups of employees under NSPS, but it is missing a key piece--an 
action plan. Our 2008 report recommended that DOD develop and implement 
a specific action plan to address employee perceptions of NSPS 
ascertained from feedback avenues such as, but not limited to, DOD's 
survey and DOD's and GAO's employee focus groups. At a minimum, this 
plan should include actions to mitigate employee concerns about the 
potential influence that employees' and supervisors' writing skills 
have on the panels' assessment of employee ratings and the lack of 
transparency and understanding of the pay pool panel process. Such a 
plan would demonstrate to employees that the department is listening to 
their concerns and making plans to address, as appropriate, the 
concerns that are identified. In short, DOD needs to tell the employees 
that they are going to take action on their concerns. In addition, the 
recently announced study by DOD and OPM is an opportunity to assess the 
status of the system. While the review intends to include a thorough 
examination of all NSPS policies, regulations, and practices, we would 
like to see DOD leverage this opportunity to assess for itself how the 
department is implementing internal safeguards. Specifically, we are 
interested in an update of how the safeguards have been incorporated 
into their policies and how the safeguards are working. 

2. The Deputy Secretary of Defense, in cooperation with the Office of 
Personnel Management recently announced that the department would halt 
conversions of DOD civilian employees to NSPS, pending the outcome of a 
review by DOD and OPM. The proposed review will assess whether or not 
NSPS is meeting its objectives of being a fair, transparent, and 
effective personnel system. Finalizing the details of such a review's 
overall framework, scope, timeline, and leadership will take time. 

* As DOD and OPM leadership hold discussions to determine the overall 
framework, scope, and timeline of the review, what guidance or 
suggestions would you give to DOD and OPM to include in the methodology 
of this study? 

As we have previously reported, the extent to which DOD incorporates 
internal safeguards into the design and implementation of NSPS and how 
it addresses employee perceptions of NSPS are key to the success of the 
system.[Footnote 5] Moving forward, as DOD and OPM embark on a study of 
NSPS and review how NSPS operates and its underlying policies, DOD has 
a unique opportunity to consider our previous recommendations, as well 
as all of the other internal safeguards key to ensuring that 
performance management systems in the government are fair, effective, 
and credible. In addition to a review of internal safeguards, this 
study provides DOD the opportunity to look at employee engagement in 
the process and develop an action plan to address employee concerns 
about NSPS. As we approached our work, we used a methodology that 
systematically took into account employee input from all levels. We 
used a combination of survey analysis, interviews, and employee 
discussion groups to obtain information on employee perceptions. In 
general, the combination of employee surveys with interviews or 
discussion groups is helpful because it yields useful information at 
the population level, as well as the individual employee experience 
level. 

3. One concern expressed by employees who have converted from GS to 
NSPS is that there is no real career progression. Under the GS system, 
an employee steadily moves up through the various grades and can 
actually monitor actual career progression. There appears to be no such 
similar movement in NSPS; an employee, while receiving pay increases 
and bonuses, may remain in the same pay band for his/her entire career. 

* If this is a valid concern, how can it be addressed, if NSPS 
continues? 

First, DOD needs to collect more information on what the issues are 
surrounding this employee perception on career progression, including 
the underlying causes and the extent of this concern, so that the 
department can determine if it is indeed a valid concern. For example, 
is there an issue with lack of career progression or are employees 
perceiving that there is an issue as a result of lack of communication 
or education on the new system? In our 2008 report, we recommended that 
the department develop and implement a specific action plan to address 
employee perceptions of NSPS ascertained from feedback avenues such as, 
but not limited to, DOD's survey and DOD's and GAO's employee focus 
groups. We believe that this is another example of how the department 
could use such an action plan to guide its approach for addressing 
employee concerns. Specifically, the plan may incorporate various 
communication and education strategies to help employees understand how 
the shift from pay grades to pay bands still affords them opportunities 
for professional development, as well as movement through the pay band. 
While we acknowledge that change takes time to gain employee acceptance 
and that the implementation of NSPS is a large-scale organizational 
transformation, employee concerns, such as these, must be heard and 
addressed accordingly in order to ensure greater employee acceptance 
and, ultimately, successful implementation of the NSPS performance 
management system. 

Questions from Representative Forbes: 

Based on the GAO testimony, one safeguard GAO believes needs to be 
implemented to increase employee confidence in the pay for performance 
system is for DOD to have a third party analyze the pay-pool 
recommendations for "anomalies" before any final decision is made to 
determine whether an employee's rating accurately reflects the 
employee's performance and whether any non-merit based factors 
contributed to the "anomaly." 

1. Explain how you see this third party analysis working. 

Given that each agency has its own set of unique challenges and its own 
approach for handling those challenges, we believe that the department 
is in the best position to determine how to appropriately design and 
implement a predecisional analysis for NSPS. That said, we believe that 
the third-party analysis should be conducted by an independent 
reviewing office, such as a human capital office, that is able to 
conduct the analysis outside of the chain of command. Taking the 
analysis outside of the chain of command helps to ensure that the 
process remains as independent as possible. Seeing that DOD currently 
has over 200,000 civilian employees under NSPS, the department could 
consider phasing in the third-party analysis by starting with a 
representative sample of employees. A phased implementation approach 
recognizes that different components of agencies will often have 
different levels of readiness and different capabilities to implement 
new authorities. Moreover, a phased approach allows for learning so 
that appropriate adjustments and midcourse corrections can be made 
before new policies and procedures are fully implemented 
organizationwide. 

2. In your view, who would the third party be, a DOD entity or a non-
DOD entity? 

As noted in our response to question 1, we would expect the third party 
to be a DOD entity that is removed from the chain of command--that is, 
the human capital office or an office of opportunity and inclusiveness. 

3. What criteria does GAO see as constituting an anomaly? 

Generally, an anomaly would be characterized as a set of ratings for 
which there is a statistically significant difference in comparison to 
the larger group. For example, if the data indicated that there was a 
statistically significant difference between the ratings of a 
particular subset of the larger workforce compared to the larger group 
at that same level, this could constitute an anomaly. The presence of 
an anomaly is not alone proof that there is a problem. Rather, 
identifying an anomaly in the data prior to finalizing the rating 
decisions would enable management to investigate the situation and 
determine whether the results are justified and merit-based. 

4. In investigating "blatant discrimination" or "egregious decisions" 
would the employee be contacted and interviewed? 

The predecisional reviews are to help achieve consistency in the 
performance management process and provide reasonable assurance that 
the performance decisions are merit-based and fair. Due to the nature 
of the investigation, the employee would not be contacted during an 
investigation. However, information provided by employees, such as the 
self-assessment, can be considered during the review process, as could 
information provided by responsible managers regarding underlying 
reasons for any anomalies. 

5. Would a single third party be evaluating all 1,600 pay pools across 
DOD to get a DOD wide view of anomalies, or would 1,600 third-party 
reviews be conducted at each pay pool without regard for a 
comprehensive DOD look? 

As noted in our response to question 1, each agency has its own set of 
unique challenges and its own approach for handling those challenges. 
As a result, we feel that the department is in the best position to 
determine how to appropriately design and implement a predecisional 
analysis for NSPS. One approach, as noted in our response to question 
1, would be for DOD to phase in the third party analysis by starting 
with a representative sample of employees. Such an approach recognizes 
that different levels of readiness and different capabilities exist 
among agency components and allows for learning so that appropriate 
adjustments and midcourse corrections can be made before full 
implementation. 

6. What effect would the third-party analyses have on the timeliness of 
the pay-pool process? 

It would likely add time to the existing process. However, we believe 
that it is important that DOD take steps to ensure that its employees' 
ratings are perceived as fair reflections of their performance. Taking 
additional time to complete a predecisional analysis is one safeguard 
that DOD can implement to raise employee confidence in the fairness and 
credibility of the system. 

7. Would the pay pool decisions on all the other employees in the pay 
pool be held up until the "anomaly" was resolved? 

Given that the predecisional review is intended to take place prior to 
the ratings being finalized but before they are certified and released 
to employees, all other ratings would not be released until the 
predecisional review was completed and appropriate responses (which 
could include inaction) were determined for any anomalies identified. 

8. How would GAO see the anomaly being corrected - a directive to the 
rater to change the rating, or some disciplinary action against the 
rater, or some other form of corrective action? 

Where managers provide information that explains the merit-based 
factors and reasons for the anomalies, the managers would not change 
the ratings. On the other hand, managers could determine that some 
vital information was not considered that would provide a basis for 
changing the rating. In all cases, it is the unit manager, not the 
third party conducting the predecisional review, that would determine 
whether a change would be warranted. Further, the review is not 
intended to change the results to portray an "ideal" distribution, or 
to alter the outcome of the performance management process. The purpose 
of the predecisional review is to identify if anomalies exist and, if 
found, inform managers of the need for further review to provide 
reasonable assurance that the basis for each rating is fair, credible, 
and merit-based. 

9. Does GAO see any appeal rights for the manager or employee involved 
in the "anomaly"? 

Although the third-party reviewer identifies the anomalies, it is the 
responsible manager that examines the basis underlying the ratings and 
is held accountable for ensuring the ratings are merit based. While it 
is unlikely, should a manager be disciplined, he or she might have 
appeal rights, depending on the nature of the discipline imposed. 

Any employee has a right to appeal his or her final rating. However, 
these appeal rights do not apply to the predecisional review process 
since it is intended to take place prior to the ratings being finalized 
but before they are certified and released to employees. 

For additional information on our work on human capital issues at DOD, 
please contact me on 202-512-3604 or farrellb@gao.gov. 

Signed by: 

Brenda S. Farrell:
Director: 
Defense Capabilities and Management: 

[End of section] 

Footnotes: 

[1] GAO, Human Capital: Improved Implementation of Safeguards and an 
Action Plan to Address Employee Concerns Could Increase Employee 
Acceptance of the National Security Personnel System, [hyperlink, 
http://www.gao.gov/products/GAO-09-464T] (Washington, D.C.: April 1, 
2009). 

[2] GAO, Defense Transformation: DOD's Proposed Civilian Personnel 
System and Governmentwide Human Capital Reform, [hyperlink, 
http://www.gao.gov/products/GAO-03-741T] (Washington, D.C.: May 1, 
2003). 

[3] GAO, Human Capital: DOD Needs to Improve Implementation of and 
Address Employee Concerns about Its National Security Personnel System, 
[hyperlink, http://www.gao.gov/products/GAO-08-773] (Washington, D.C.: 
Sept. 10, 2008). 

[4] [hyperlink, http://www.gao.gov/products/GAO-08-773]. 

[5] [hyperlink, http://www.gao.gov/products/GAO-08-773]. 

[End of section] 

GAO's Mission: 

The Government Accountability Office, the audit, evaluation and 
investigative arm of Congress, exists to support Congress in meeting 
its constitutional responsibilities and to help improve the performance 
and accountability of the federal government for the American people. 
GAO examines the use of public funds; evaluates federal programs and 
policies; and provides analyses, recommendations, and other assistance 
to help Congress make informed oversight, policy, and funding 
decisions. GAO's commitment to good government is reflected in its core 
values of accountability, integrity, and reliability. 

Obtaining Copies of GAO Reports and Testimony: 

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each 
weekday, GAO posts newly released reports, testimony, and 
correspondence on its Web site. To have GAO e-mail you a list of newly 
posted products every afternoon, go to [hyperlink, http://www.gao.gov] 
and select "E-mail Updates." 

Order by Phone: 

The price of each GAO publication reflects GAO’s actual cost of
production and distribution and depends on the number of pages in the
publication and whether the publication is printed in color or black and
white. Pricing and ordering information is posted on GAO’s Web site, 
[hyperlink, http://www.gao.gov/ordering.htm]. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card,
MasterCard, Visa, check, or money order. Call for additional 
information. 

To Report Fraud, Waste, and Abuse in Federal Programs: 

Contact: 

Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: 
E-mail: fraudnet@gao.gov: 
Automated answering system: (800) 424-5454 or (202) 512-7470: 

Congressional Relations: 

Ralph Dawn, Managing Director, dawnr@gao.gov: 
(202) 512-4400: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7125: 
Washington, D.C. 20548: 

Public Affairs: 

Chuck Young, Managing Director, youngc1@gao.gov: 
(202) 512-4800: 
U.S. Government Accountability Office: 
441 G Street NW, Room 7149: 
Washington, D.C. 20548: