This is the accessible text file for GAO report number GAO-02-641T 
entitled 'Violence Against Women Office: Problems with Grant 
Monitoring and Concerns about Evaluation Studies' which was released 
on April 16, 2002. 

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the 
printed version. The portable document format (PDF) file is an exact 
electronic replica of the printed version. We welcome your feedback. 
Please E-mail your comments regarding the contents or accessibility 
features of this document to 

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately. 

United States General Accounting Office: 


Before the Subcommittee on Crime and Drugs, Committee on the 
Judiciary, U.S. Senate: 

For Release on Delivery: 
Expected at 10:15 a.m., EDT: 
Tuesday, April 16, 2002: 	 

Violence Against Women Office: 

Problems with Grant Monitoring and Concerns about Evaluation Studies: 

Statement of Laurie E. Ekstrand: 
Director, Justice Issues: 


Mr. Chairman and Members of the Subcommittee: 

Thank you for inviting me to testify about our recent work concerning 
the Violence Against Women Office (VAWO). This work has specifically 
focused on monitoring activities and impact evaluations related to 
VAWO's discretionary grant programs and is part of a body of recent 
work concerning monitoring and evaluation of grants by a number of 
Office of Justice Program's (OJP) bureaus and offices. Monitoring and 
evaluation are the activities that identify whether programs are 
operating as intended, whether they are reaching those that should be 
served, and ultimately whether they make a difference. In other words, 
these are major elements of assessing results. Our recent work has 
shown a need for improvement in VAWO grant monitoring and in the 
evaluations that are intended to assess the impacts of VAWO programs. 


VAWO was created in 1995 to carry out certain programs created under 
the Violence Against Women Act of 1994.[Footnote 1] The Victims of 
Trafficking and Violence Prevention Act of 2000 reauthorized most of 
the existing VAWO programs and added new programs.[Footnote 2] VAWO's 
mission is to lead the national effort to end violence against women, 
including domestic violence, sexual assault, and stalking. VAWO 
programs seek to improve criminal justice system responses to these 
crimes by providing support for law enforcement, prosecution, courts, 
and victim advocacy programs across the country. In addition, programs 
are to enhance direct services for victims, including victim advocacy, 
emergency shelter, and legal services. VAWO also addresses violence 
against women issues internationally, including working to prevent 
trafficking in persons. VAWO is one of seven program offices and five 
bureaus in OJP.[Footnote 3] 

VAWO's discretionary grant programs have grown substantially since its 
inception in 1995. Data provided by OJP showed that, between fiscal 
years 1995 and 2000, the yearly number of VAWO discretionary grant 
awards increased about 362 percent—-from 92 in fiscal year 1996, the 
first full year of funding, to 425 in fiscal year 2000. In addition 
the yearly dollar amount of VAWO discretionary grant awards increased 
about 940 percent-—from just over $12 million in fiscal year 1996, the 
first full year of funding, to about $125 million in fiscal year 2000. 
Appendix II shows the number of yearly VAWO discretionary grant awards 
for fiscal year 1995 through fiscal year 2000. Appendix BI shows the 
dollar amount of VAWO discretionary grant awards, adjusted to constant 
fiscal year 2000 dollars, over the same period. 

Problems with VAWO Discretionary Grant Monitoring: 

The monitoring of grant activities is a key management tool to help 
ensure that funds awarded to grantees are being properly spent. In 
November 2001, in response to a request by Senators Grassley and 
Sessions, we reported that grant files for discretionary grants 
awarded by VAWO often lacked the documentation necessary to ensure 
that the required monitoring activities occurred.[Footnote 4] Our 
review of grant files for a representative sample of VAWO 
discretionary grants active in all of fiscal years 1999 and/or 2000 
showed that: 

* VAWO grant files did not always contain requisite grant monitoring 
plans. When monitoring plans were in the files, grant managers did not 
consistently document their monitoring activities, such as site 
visits, according to the plans they developed. 

* A substantial number of VAWO grant files did not contain progress 
and financial reports sufficient to cover the entire grant period, 
contrary to OJP guidelines. Furthermore, VAWO grantee progress and 
financial reports were often submitted late by grantees. These reports 
are an important tool to help managers and grant monitors determine if 
grantees are meeting program objectives and financial commitments. 

* VAWO grant files did not always contain the required closeout 
documents-—key documents by which OJP ensures that, among other 
things, the final accounting of federal funds have been received. 

We also found that, because documentation about monitoring activities 
was not readily available, VAWO was not positioned to systematically 
determine staff compliance with monitoring requirements and assess 
overall performance. Although VAWO officials said that they met with 
grant managers weekly to discuss any grant problems or monitoring 
issues, VAWO did not (1) have an overall system to track monitoring 
activities, other than site visits and (2) appear to be routinely 
using OJP-wide data on late progress reports and financial reports. 
Furthermore, the lack of systematic data associated with program 
monitoring activities and the documentation problems we observed 
raised questions about whether VAWO was positioned to measure its 
performance consistent with the Government Performance and Results Act 
(GPRA) of 1993. Specifically, we pointed out that, in DOJ's Fiscal 
Year 2000 Performance Report and Fiscal Year 2002 Performance Plan, 
DOJ failed to recognize the serious limitations associated with 
inconsistent documentation and the lack of systematic monitoring data 
in measuring whether VAWO was achieving its goals for formula and 
discretionary grants—especially since the Report and Plan stated that 
VAWO would rely on grant monitoring data to measure its performance. 

We concluded that neither OJP, VAWO, nor GAO can determine the level 
of monitoring performed by grant managers as required by OJP and the 
comptroller general's internal control standards, which call for 
documentation of all transactions and other significant events to 
ensure that management directives are being carried out.[Footnote 5] 
We recommended that VAWO review why documentation problems occurred 
and take steps to resolve these problems. 

Too Early to Gauge Efforts to Resolve Grant Monitoring Problems: 

VAWO and OJP officials have acknowledged that they need to take steps 
to resolve some of the problems associated with grant monitoring, but 
it is too early to tell if these steps will be effective. For example, 
in response to our report, the assistant attorney general said that 
VAWO had begun to develop both an internal monitoring manual that 
would include procedures for developing monitoring plans using a risk-
based assessment tool. They also said they have developed a management 
information system that will eventually track the submission of 
progress and financial reports. Furthermore, while we were developing 
our report, VAWO officials said that they were not satisfied with the 
performance measures they used to gauge their performance under GPRA 
because they did not believe they are meaningful for measuring program 
outcomes. They said that they are working with other OJP officials and 
an outside contractor to develop new measures and hope to have them 
available for the fiscal year 2003 performance plan. 

It is also important to note that VAWO's efforts to address grant 
monitoring problems need to be viewed in the context of OJP efforts in 
this area. Our recent related reports discussed grant monitoring 
problems at other OJP organizations, such as the Bureau of Justice 
Assistance (BJA) and the Office of Juvenile Justice and Delinquency 
Programs (OJJDP),[Footnote 6] and pointed out that, over the last few 
years, we and others, including OJP, have identified various grant 
monitoring problems among OJP bureaus and offices. We discussed how 
OJP had begun to work with bureaus and offices to resolve some of the 
problems it and others have identified, including OJP efforts to 
develop an automated grants management system as a way to standardize 
and streamline the grant process. 

Our report concluded that OJP efforts to automate the grant management 
process, particularly in regard to grant monitoring, holds some 
promise if OJP takes steps to ensure that all monitoring activities 
are consistently recorded and maintained in a timely manner. We also 
said that current and future efforts will be futile unless OJP and its 
bureaus and offices, such as VAWO, periodically test grant manager 
compliance with OJP requirements and take corrective action when 
needed to enforce those requirements. We recommended that OJP (1) 
study and recommend ways to establish an approach to systematically 
test or review grant files to ensure consistent documentation across 
OJP and (2) explore ways to electronically compile and maintain 
documentation of monitoring activities to facilitate more consistent 
documentation, more accessible management oversight, and sound 
performance measurement. 

In January 2002, in response to our report, the assistant attorney 
general said that OJP agreed that it needs to develop more consistent 
documentation of monitoring activities. She said that among other 
things, OJP has created a chief information officer position charged 
with planning and implementing an agencywide grant management system. 
According to the assistant attorney general, the new system is 
envisioned to produce reports in response to informational requests, 
provide information pertaining to grantees and all resources provided 
by OJP, and maintain information from the opening to the closing of a 
grant award. Although the assistant attorney general said that OJP 
will consider the comptroller general's internal control standards in 
taking these steps, it is unclear whether the new system will include 
the full range and scope of monitoring activities carried out by grant 
managers in VAWO and other OJP organizations. 

Concerns About Evaluation Studies of VAWO Discretionary Grant Programs: 

We have also recently issued a report on work undertaken for Senators 
Grassley and Sessions that addressed the methodological rigor of 
impact evaluations of three VAWO discretionary grant programs. 
[Footnote 7] During fiscal years 1995 through 2001, the National 
Institute of Justice (NIJ) awarded about $4 million for five VAWO 
discretionary grant program evaluations that were intended to measure 
the impact of the VAWO programs.[Footnote 8] Our in-depth review of 
the three program evaluations that had progressed beyond the formative 
stage showed that all three had methodological problems that raised 
concerns about whether the evaluations will produce definitive results. 

More specifically, our report stated that, although program evaluation 
is an inherently difficult task, in all three VAWO evaluations, the 
effort was particularly arduous because of variations across grantee 
sites in how the programs are implemented. Our concerns about these 
efforts included problems with both evaluation design and 
implementation. In particular, VAWO sites participating in the impact 
evaluations had not been shown to be representative of their programs, 
thereby limiting the evaluators' ability to generalize results. 
Further, the lack of nonprogram participant comparison groups hindered 
evaluators' ability to minimize the effects of factors that are 
external to the program and isolate the impact of the program alone. 
While in some situations, other means (other than comparison groups) 
can be effective in isolating the impact of a program from other 
factors, in these evaluations, effective alternative methods were not 
used. In addition, data collection and analytical problems (e.g., 
related to statistical tests, assessment of change) compromised the 
evaluators' ability to draw appropriate conclusions from the results. 

We concluded that, despite great interest in assessing the results of 
OJP's discretionary grant programs, it can be extremely difficult to 
design and execute evaluations that will provide definitive 
information. We further concluded that, given that NIJ spends millions 
of dollars to evaluate OJP grant programs, including those within 
VAWO, more up-front attention to the methodological rigor of these 
evaluations will increase the likelihood that they will produce 
meaningful results for policymakers. We recommended that the attorney 
general direct the NIJ director to assess the two VAWO impact 
evaluations still in the formative stage to address any design 
methodology and implementation problems and, on the basis of that 
assessment, initiate any needed interventions to help ensure that the 
evaluations produce definitive results. We further recommended that 
the director of NIJ be instructed to assess its evaluation process to 
develop approaches to ensure that future impact evaluation studies are 
designed and implemented to produce definitive results. The assistant 
attorney general commented that she agreed with the substance of our 
recommendations and has begun or plans to take steps to address them. 
It is still too early to tell whether these actions will be effective 
in preventing or resolving the problems we identified, but they appear 
to be steps in the right direction. 

In summary, since its inception, VAWO has grown substantially both in 
terms of the number of discretionary grants awarded and dollars 
awarded for those discretionary grants—increasing the importance of 
ensuring that its grantees are achieving intended results. 
Unfortunately, the lack of good data from monitoring activities and 
impact evaluations leaves us with very little basis to assess program 
results. Both VAWO and OJP have indicated a commitment to making 
improvements, citing reorganization plans and the anticipated 
management information system as the foundation for improved grants 
management, including improvements in monitoring and evaluation. But, 
reorganization and management information systems are only tools and 
are only as good as the management that wields them. Commitment to 
improvement and oversight are needed to ensure progress. 

Mr. Chairman, this concludes my prepared statement. I would be pleased 
to answer any questions that you or other members of the subcommittee 
may have. 

For further information regarding this testimony, please contact 
Laurie E. Ekstrand or John F. Mortin at (202) 512-8777. Individuals 
making key contributions to this testimony included Wendy C. Simkalo, 
Jared A. Hermalin, and Chan My J. Battcher. 

[End of section] 

Appendix I: OJP Organization Chart: 

[Refer to PDF for image: organization chart] 

Top level: 
Office of Assistant Attorney General: 
* American Indian & Alaska Native Affairs Desk; 
* Equal Employment Opportunity Office. 

Second level, reporting to Office of Assistant Attorney General: 
* Violence Against Women Office; 
* Corrections Program Office; 
* Drug Courts Program Office; 
* Executive Office for Weed & Seed; 
* Office for Domestic Preparedness; 
* Office of Police Corps & Law Enforcement Education; 
* Office of Budget, & Management Services; 
* Office of Administration; 
* Office for Civil Rights; 
* Office of the Comptroller; 
* Office of General Counsel; 
* Office of Congressional & Public Affairs; 
* Bureau of Justice Assistance; 
* Bureau of Justice Statistics; 
* National Institute of Justice; 
* Office of Juvenile Justice & Delinquency Prevention; 
* Office for Victims of Crime. 

Note: The organization chart is current as of March 2002. 

Source: Prepared by GAO based on OJP documentation. 

[End of figure] 

[End of section] 

Appendix II: Number of Yearly VAWO Discretionary Awards, Fiscal Years 

[Refer to PDF for image: vertical bar graph] 

Number of Awards (approximated from graph): 

Fiscal Year 1995: 20; 
Fiscal Year 1996: 80; 
Fiscal Year 1997: 245; 
Fiscal Year 1998: 350; 
Fiscal Year 1999: 330; 
Fiscal Year 2000: 425. 

Source: OJP Office of the Comptroller. 

[End of figure] 

[End of section] 

Appendix III: Dollar Amount of VAWO Discretionary Awards, Fiscal Years 

[Refer to PDF for image: vertical bar graph] 

Amount of Awards (approximated from graph): 

Fiscal Year 1995: $1,000,000; 
Fiscal Year 1996: $10,000,000; 
Fiscal Year 1997: $70,000,000; 
Fiscal Year 1998: $115,000,000; 
Fiscal Year 1999: $90,000,000. 
Fiscal Year 2000: $125,000,000. 

Note: The award amounts for each fiscal year are adjusted to constant 
fiscal year 2000 dollars. 

Source: OJP Office of the Comptroller. 

[End of figure] 

[End of section] 


[1] Title IV of the Violent Crime Control and Law Enforcement Act of 
1994 (P.L. 103-322). 

[2] P.L. 106-386. 

[3] OJP’s five bureaus are Bureau of Justice Assistance, Bureau of 
Justice Statistics, National Institute of Justice, Office of Juvenile 
Justice and Delinquency Prevention, and Office for Victims of Crime. 
OJP’s seven program offices are American Indian and Alaska Native 
Affairs Desk, Violence Against Women Office, Executive Office for Weed 
and Seed, Corrections Program Office, Drug Courts Program Office, 
Office for Domestic Preparedness, and Office of Police Corps and Law 
Enforcement Education. Appendix I shows OJP’s current organizational 

[4] U.S. General Accounting Office, Justice Discretionary Grants: 
Byrne Program and Violence Against Women Office Grant Monitoring 
Should Be Better Documented, [hyperlink,] (Washington, D.C.: Nov. 28, 

[5] U.S. General Accounting Office, Internal Control: Standards for 
Internal Control in the Federal Government, [hyperlink,] (Washington, D.C.: 
Nov. 1999). 

[6] U.S. General Accounting Office, Juvenile Justice: OJJDP Reporting 
Requirements for Discretionary and Formula Grantees and Concerns About 
Evaluation Studies, [hyperlink,] 
(Washington, D.C.: Oct. 30, 2001). 

[7] U.S. General Accounting Office, Justice Impact Evaluations: One 
Byrne Evaluation Was Rigorous; AU Reviewed Violence Against Women 
Office Evaluations Were Problematic, [hyperlink,] (Washington, D.C.: Mar. 7, 

[8] Impact evaluations are designed to assess the net effect of a 
program by comparing program outcomes with an estimate of what would 
have happened in the absence of the program. 

[End of section]