From the U.S. Government Accountability Office, www.gao.gov Transcript for: A Closer Look at Data-Driven Performance Reviews Description: Audio interview by GAO staff with Chris Mihm, Managing Director, Strategic Issues Related GAO Work: GAO-13-228: Managing for Results: Data-Driven Performance Reviews Show Promise but Agencies Should Explore How to Involve Other Relevant Agencies Released: February 2013 [ Background Music ] [ Narrator: ] Welcome to GAO's Watchdog Report, your source for news and information from the U.S. Government Accountability Office. It's February 2013. The Government Performance and Results Act Modernization Act of 2010 included a provision for agency leaders to conduct quarterly data-driven performance reviews. A team led by Chris Mihm, managing director of GAO's Strategic Issues team, recently reviewed the implementation and impact of these reviews. GAO's Jeremy Cluchey sat down with Chris to learn more. [ Jeremy Cluchey: ] How do data-driven performance reviews differ from what's been used for reviewing performance in the past? [ Chris Mihm: ] Well one of the big pieces of news is that while it's new at the federal level, it's not new at other levels of government. Actually the whole model of the data-driven reviews builds off of something called the STAT reviews that were done in the beginning with something called COMPSTAT in New York in the mid-1990s. It grew from there into being used in the police office--police department in New York City--to other law enforcement agencies, to other programs around the country. Grew into then state STAT in Maryland, in the state of Washington and so what this is, is taking that basic approach of using data to pinpoint improvement opportunities, putting in place improvement strategies, and then following up on those in a very rigorous way. It's bringing that approach to the federal level and that's what's new. [ Jeremy Cluchey: ] Your team identified a few leading practices that can help yield successful reviews of this type. Can you talk a little bit about these practices? [ Chris Mihm: ] Sure. The basic idea behind the STAT reviews is that it's a leadership strategy in which you get the personal involvement of the head of the agency or the head of the organization to actually sit down with his or her senior managers and look for opportunities to improve performance. And what we identified then was based on the literature, experience at local levels, at state levels, certainly the experiences that we're already beginning to see at the federal levels, a set of practices that we have seen that our most directly involved with successful performance review meetings like the quarterly reviews. And they are things like making sure that the top people attend the meetings, making sure other key people are there, making sure that we have the data that we need and that it's disaggregated in such a way that we can pinpoint improvement opportunities, making sure that there's rigorous follow-up, and to make sure that we've actually taken the actions that we're committed to in those meetings. And then finally and you know, perhaps most importantly is making sure that these meetings are regularly scheduled and that they're actually held. Too often in the past especially at the federal level, performance data would be reported once a year or twice a year. It would go into a paper document that nobody would ever read. What these are about is let's get regularly scheduled meetings. Let's make sure that the key players are present and let's follow up and make sure that corrective actions are taken. [ Jeremy Cluchey: ] And what impact did you find that these reviews are having at the agencies that you looked at? [ Chris Mihm: ] Not surprisingly that given the experience at state and local levels, we're certainly seeing that it is having some impact at the federal levels and in fact, one of the key things that we wanted to ask about is this model scalable? That is it can work at a local level where there's a pretty direct connection between an activity you may undertake and then a change in results. And so if you put more cops on the--where you have crime, you can end up reducing crime. The approach that they used in New York was called cops on dots. You know, what we wanted to see in this case, dots being where the crime was. What we wanted to see in this case was is it scalable up to the federal level where you have much more indirect connection between federal programs and the outcomes that are ultimately achieved? And indeed, we've been able to see it. We took some time looking at the STAT meetings that were taking place over at Treasury, over at SBA and Department of Energy. We actually attended the ones, several of them over at Treasury. Found them to just be fascinating discussions that were taking place between the Deputy Secretary and his leadership team including people in the agencies and they were looking for and finding opportunities to improve performance across the Treasury agencies. [ Jeremy Cluchey: ] GAO is making a recommendation in this report; it's related to collaboration among agencies. Can you explain this? [ Chris Mihm: ] Yeah, one of the things that's missing at the federal level or that we've seen thus far is using these STAT meetings to get all the relevant players together and talk about how their individual initiatives are at least intended to combine and work together in such a way to contribute to an outcome. We've seen that when you look at models at the state level where a governor will convene her or his entire management team and program team if it's looking at, you know, for example, children at risk or homelessness or something like that--getting together all the programs that contribute to that. We don't do that yet at the federal level. We think that's a real missed opportunity here, because what it requires then is a parallel structure to be put in place to ensure collaboration takes place. You don't need that. You know, let's use the STAT meetings the way that they have been used at the state and local and even regional level. One of the great models on this is what they have in the state of Maryland. It's a broad thing called Base STAT that brings together a variety of different state programs to look at the health of the Chesapeake Bay. That's exactly the type of cross-cutting and collaborative approach that is needed and how those STAT meetings can be used to foster that collaboration. [ Background Music ] [ Narrator: ] To learn more, visit GAO.gov and be sure to tune in to the next episode of GAO's Watchdog Report for more from the congressional Watchdog, the U.S. Government Accountability Office.