Four People Performance Measures: Many Attributes of Successful Measures Met; Opportunities Exist for Further Enhancements
OIG-09-3, Aug 31, 2009
This is a publication by GAO's Inspector General that concerns internal GAO operations. As part of the Office of Inspector General's (OIG) commitment to periodically review the reliability and validity of GAO's performance measures, OIG evaluated four of the agency's people measures--staff development, staff utilization, leadership, and organizational climate. OIG based its evaluation largely on an assessment of whether these measures met nine specific attributes that earlier GAO work cited as key to successful performance measures.
OIG's evaluation showed that, for fiscal year 2007, GAO accurately calculated the four performance measures reviewed. Of the nine attributes of successful performance measures identified by previous GAO work, staff utilization and organizational climate met all of the attributes, and staff development and leadership met many of the attributes. All four measures' scores are derived from GAO's Employee Feedback Survey. For the staff development and leadership measures, OIG found that GAO had not fully disclosed in its annual performance report that (1) it calculates these scores by excluding survey respondents that answered "no basis to judge/not applicable" (no basis/NA) and (2) the exclusion of these respondents has the effect of changing the two measures' scores. A more complete interpretation of the scores would be possible with additional disclosures in the report's tables. In addition, the score for the leadership measure was significantly affected by the large number of no basis/NA respondents for one of its 10 questions. In fiscal year 2007, 45 percent of surveyed respondents answered no basis/NA to the question about supervisors' effectiveness in handling equal employment opportunity (EEO) and discrimination issues. GAO officials stated this large number most likely reflects the relatively few formal discrimination cases and the safeguarding of private information related to these cases. GAO's employee survey has recently added other questions on diversity that could provide more useful information and do not have large numbers of no basis/NA respondents. Further, although GAO's leadership measure captures information about employees' satisfaction with their immediate supervisors, it does not include the employees' survey responses about satisfaction with the strategic leadership of GAO. As a result, the measure's name differs from what is being measured. Finally, GAO was not timely in reporting changes made to its staff development measure that resulted in performance data no longer being comparable. In response to OIG's work, GAO made this disclosure in its fiscal year 2008 annual performance report. However, the agency does not have written procedures that would help ensure the timely reporting of future changes to measures. Please review the full report for a list of Inspector General recommendations.