This is the accessible text file for GAO report number GAO-02-416 entitled 'Drug Control: DEA Could Improve Its Heroin Signature and Domestic Monitor Programs' Geographic Source Data' which was released on March 29, 2002. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. United States General Accounting Office: GAO: Report to the Co-Chairman, Caucus on International Narcotics Control, U.S. Senate: March 2002: Drug Control: DEA Could Improve Its Heroin Signature and Domestic Monitor Programs' Geographic Source Data: GA0-02-416: Contents: Letter: Results in Brief: Background: Scope and Methodology: Purpose of the HSP and DMP Is To Produce Data That Detect Trends in Heroin Sources: Federal Law Enforcement Used the HSP and DMP Data for Intelligence Purposes and Management: Quantity of Heroin Seized at Ports-of-Entry by Customs but not Sent to DEA for Testing May Be Sufficient To Make a Difference in Results Reported by DEA: The HSP and DMP Data Could Be Improved: Conclusions: Recommendations: Agency Comments: Appendix I: Heroin Signature Program: HSP Methodology: HSP Limitations: HSP Opportunities for Improvements: Appendix II: Domestic Monitor Program: DMP Limitations: DMP Opportunities for Improvements: Appendix III: Comments from the Drug Enforcement Administration: Tables: Table 1: Source and Selection for HSP Heroin: Table 2: U.S. Customs Service Heroin Seizures (by line items): Table 3: U.S. Customs Service Heroin Seizures Weighing at Least 100 grams, Fiscal Years 1998-2000 (by line items): Table 4: Geographic Source and Percentage of Seized Heroin, Calendar Year 1999: Table 5: Stratified HSP Sample Design: Table 6: By Metropolitan Area, the Number of Exhibits from Each Geographic Source, Calendar Year 1999: Figure: Figure 1: DEA Case/Seizure/Exhibit: Abbreviations: DAWN: Drug Abuse Warning Network: DEA: Drug Enforcement Administration: DMP: Domestic Monitor Program: FBI: Federal Bureau of Investigation: HSP: Heroin Signature Program: MOU: Memorandum of Understanding: ONDCP: Office of National Drug Control Policy: SEACATS: Seized Assets and Case Tracking: STRL: Special Testing and Research Laboratory: TECS: Treasury's Enforcement Communications System: [End of section] United States General Accounting Office: Washington, DC 20548: March 29, 2002: The Honorable Charles E. Grassley: Co-Chairman, Caucus on International Narcotics Control: United States Senate: Dear Senator Grassley: The Drug Enforcement Administration (DEA) administers two programs, the Heroin Signature Program (HSP) and the Domestic Monitor Program (DMP), that serve as indicators for assessing trends in the geographic source of heroin supplied to the United States. These programs produce data on the geographic source of heroin through a series of chemical analyses and are the only programs of their kind in the United States. The HSP reports the geographic source of heroin seized at ports-of- entry (primarily by the U.S. Customs Service) and other locations, as well as wholesale level purity. The DMP reports the geographic source of heroin purchased undercover in 23 U.S. metropolitan areas, as well as retail level heroin purity and price. This report does not focus on purity and price. This report responds to your request that we review how DEA captures and reports the data for the HSP and DMP and that we provide, if appropriate, possible alternative approaches for DEA to consider to improve the HSP and DMP data.[Footnote 1] Specifically, this report discusses: * the purpose of the HSP and DMP; * how federal law enforcement uses the data generated by the HSP and DMP; * whether the quantity of heroin seized at ports-of-entry by Customs, but not sent to DEA for testing, is sufficient to make a difference in the results reported by DEA; and; * whether the HSP and DMP data could be improved. To address these areas, we met with officials from DEA, Customs, and the Office of National Drug Control Policy (ONDCP) concerning the purpose and operations of the HSP and DMP and how federal law enforcement uses the data generated by the programs. We obtained and analyzed data from Customs to determine the quantity of heroin that is seized at ports-of-entry by Customs, but not sent to DEA for testing. To determine if improvements could be made in the HSP and DMP data, we met with DEA officials and reviewed the methodology used by DEA in its design and implementation of the programs. Results in Brief: The HSP and DMP produce data for detecting trends in the geographic source of the heroin that is found in the United States.[Footnote 2] DEA officials stressed that the purpose of these data is not to provide overall estimates about the geographic source of heroin. The HSP data are intended to provide law enforcement with a "snapshot" of where heroin at the wholesale level originates, but only for the heroin that is tested; the DMP data are intended to provide law enforcement with a "snapshot" of where heroin at the retail level, in certain metropolitan areas, originates, but only for the heroin that is tested.[Footnote 3] According to DEA officials, federal law enforcement used the data generated by the HSP and DMP for intelligence purposes and as a management tool. The data are used to develop intelligence and investigative reports to inform the DEA and other federal law enforcement agencies about trends in heroin trafficking. According to DEA, federal law enforcement also used the HSP and DMP data as a management tool to make adjustments in enforcement activities. For example, changes in HSP and DMP data, in conjunction with data from other sources, could influence the allocation of federal law enforcement resources from one location to another. DEA cited the emergence of heroin from South America as an example of how the data are used. In 1991, intelligence reports indicated that heroin was entering the United States from South America and that Southwest Asian heroin producers had taught Colombians their methods of processing opium into heroin. DEA subsequently identified, through chemical analyses and this other information, that South America was a new supplier of heroin into the United States. The quantity of heroin seized at ports-of-entry by Customs but not sent to DEA for testing may be sufficient to make a difference in the results reported by DEA. According to DEA officials, all ports-of- entry seizures forwarded to DEA by Customs are tested by DEA for geographic source. However, DEA and Customs officials noted that, in accordance with the Memorandum of Understanding (MOU) between the two agencies, Customs is not required to send all heroin seizures to DEA. Our analysis of Customs' heroin seizure data revealed that, over fiscal years 1998-2000,[Footnote 4] 57 percent of the total weight of heroin seized by Customs was not sent to DEA. The HSP and DMP data could be improved. As currently designed and implemented, there are limitations in the usefulness of the HSP and DMP data, because they were based on nonrepresentative samples of their respective populations. While DEA said that it did not intend to produce any estimates from the HSP and DMP data, our analysis showed that with some modifications to its methodology, DEA could produce estimates. These estimates would provide a stronger basis for law enforcement decision-making. With modifications to the HSP methodology, estimates could be made about the geographic source of all seized wholesale heroin that is sent to DEA for testing. With modifications to the DMP methodology, estimates could be made about the geographic source of retail level purchases within the metropolitan areas where they are made, and the estimates could possibly be combined across the 23 metropolitan areas. Therefore, we are recommending that the Attorney General direct the Administrator of DEA to (1) ensure that the HSP data are based on a probability sample so that all HSP exhibits have a known chance of selection, (2) revise the HSP methodology for reporting testing results to include procedures to adjust for the probability of exhibits being selected for the test sample, (3) take action to ensure that DMP purchases are made according to DEA guidelines, and (4) study the use of alternative data sources for the total number of retail heroin purchases in an area that could allow the DMP data to be combined across metropolitan areas. We are also recommending that the attorney general and the secretary of the treasury direct the administrator of DEA and commissioner of Customs, respectively, to enter into discussions to determine whether additional seized heroin should be forwarded to DEA by Customs. We provided DEA, Customs and ONDCP with a draft of this report. Written comments from DEA are discussed in our Agency Comments section on pages 17-18. Each heroin-producing region has a unique production process,[Footnote 5] or signature, which generally can be determined through chemical analyses. In the HSP, seized or purchased substances are forwarded to one of DEA's regional laboratories,[Footnote 6] which confirms whether the substance is heroin. If the substance is confirmed to be heroin, the laboratory is responsible for preparing a written report for judicial purposes and, in certain circumstances, providing a sample to DEA's Special Testing and Research Laboratory (STRL) for signature analysis. STRL analyzes the heroin samples; in most instances, these analyses result in the identification of the heroin's geographic source. In the DMP, purchased substances are sent directly to STRL for analysis. The heroin that is selected for HSP testing is selected from either "cases" or "exhibits." Figure 1 illustrates a possible case that includes multiple seizures and exhibits. Figure 1: DEA Case/Seizure/Exhibit: [Refer to PDF for image: illustration] DEA Case #1906, opened July 2, 2001: Seizure #1: 9/17/01; Seizure #2: 9/18/01; Seizure #3: 9/19/01: - Exhibit 1: Desk; - Exhibit 2: Closet; - Exhibit 1: Coat Pocket. Source: GAO's analysis of DEA information. [End of figure] As shown in figure 1, a single case can include more than one seizure of heroin. For example, a single case would include more than one seizure if agents seized heroin from the same person (the same case), but on different dates. In turn, a single seizure might include more than one exhibit, if agents find heroin associated with the same person on the same date, but at multiple locations. For example, a seizure would include three exhibits if agents, on the same date, seize heroin from multiple locations such as the person's desk, closet, and coat pocket. DEA initiated the HSP in 1977. Heroin for the HSP is obtained from eight sources and selected for signature analysis as shown in table 1. Table 1: Source and Selection for HSP Heroin: Source: 1. Seizures from passengers, luggage, or cargo on flights that have originated outside the United States; Selected: A sample from each case. Source: 2. Seizures made at nonairport ports-of-entry; Selected: A sample from each case. Source: 3. Seizures from letters, packages, or freight shipped by the U.S. Postal Service or a commercial mail or freight-forwarding company and that originated outside the continental United States and were sent to a DEA laboratory for analysis; Selected: A sample from each case. Source: 4. Nonairport seizures in Hawaii, Alaska, and Puerto Rico; Selected: A sample from each case. Source: 5. Special requests from DEA; Selected: All are to be selected. Source: 6. FBI seizures that have been submitted directly to a DEA laboratory for analysis; Selected: One exhibit from each case. Source: 7. Seizures made by the Washington, D.C., Metropolitan Police Department; Selected: A random sample of exhibits, determined through the use of random numbers provided by DEA to its regional laboratories every 6 months. Source: 8. DEA seizures (includes seizures from flights and mail that originate inside the United States and Federal Bureau of Investigation seizures submitted through a DEA field office); Selected: A random sample of exhibits, determined through the use of random numbers provided by DEA to its regional laboratories every 6 months. Source: DEA. [End of table] DEA initiated the DMP in its New York Field Division in 1979. The program has expanded to include 23 metropolitan areas. The DMP was originally designed to enable DEA to monitor the price and purity of retail-level heroin sold in the United States; it now also provides for the purchase of heroin for signature analysis. In the DMP, DEA provides funding for quarterly purchases by DEA field divisions, that may utilize cooperating sources to make retail-level purchases of heroin.[Footnote 7] Ten purchases are to be made in 22 of the 23 metropolitan areas, each quarter; in New York City, 20 purchases are to be made in each quarter.[Footnote 8] DEA guidelines provide that a certain number of DMP purchases be made each quarter, throughout the 3 months of each quarter, and in various locations in the metropolitan areas. According to the DEA guidelines, it would seldom be necessary to make more than one purchase in any one location, per day. The most important requirement is that exhibits should be purchased from locations within the metropolitan areas that are dissimilar enough to ensure that they come from different suppliers. Each purchase should weigh at least 1 gram net, including diluents and adulterants, to ensure that there is a sufficient amount of pure heroin available to perform a signature analysis. According to DEA, the average time necessary to complete the signature analysis of a heroin sample is about 4 hours, with an associated cost of about $375 per sample. DEA indicated that the STRL performs signature analyses of 3,000 heroin samples annually, for a total annual STRL cost of about $1.13 million. DEA provides its field divisions with a total of $200,000 annually for the DMP purchases. DEA prepares annual reports on the HSP and DMP data The HSP reports display data on geographic source by net weight on a national basis. The DMP reports display data on geographic source by the number of exhibits by metropolitan area. Scope and Methodology: To determine the purpose of the HSP and DMP and how they operate, we interviewed officials at DEA's Office of Intelligence, Forensic Sciences, and the STRL. We reviewed relevant policies, reports, and other documentation. To determine how federal law enforcement uses the data generated by the programs, we interviewed officials at DEA's Office of Intelligence and Office of Forensic Sciences and ONDCP. We also reviewed relevant documentation of the use made of the data, including documentation of instances in which changes have been made in federal law enforcement efforts as a result of the data. To determine if the heroin seized at ports-of-entry, but not tested by DEA, is of sufficient quantity to make a difference in the results reported by DEA, we interviewed officials at DEA's Office of Intelligence, Office of Forensic Sciences, and the STRL, as well as Custom's Smuggling Investigations Division and Office of Intelligence. We also obtained and analyzed data from Customs regarding seized heroin in fiscal years 1998-2000. [Footnote 9] These data were derived from the Department of the Treasury's Enforcement Communications System (TECS), Seized Assets and Case Tracking (SEACATS) subsystem. We did not verify the accuracy of these data. The data included the date and location of the seizure,[Footnote 10] the total weight of the seizure (less the weight of any packaging or container), and the disposition of the heroin. According to Customs, a seizure is recorded in the system when contraband is discovered and physical custody is taken. Under Customs' policy, seizures that involve taking heroin from more than one place, for example, from an individual's pocket and from his suitcase, are counted as one seizure but entered into the system as two "line items." We were provided data by line items. Each line item from the same seizure equates to a DEA exhibit. To determine if the sample-based HSP and DMP data could be improved, we interviewed officials at DEA's Office of Intelligence and Office of Forensic Sciences. We reviewed sample design and sample selection methodology and the formulas and methodology used to develop data on the geographic source of heroin. We obtained and reviewed HSP and DMP data files for calendar years 1999 and 2000. We reviewed earlier reports, analyzed current methodology, and how DEA reports and caveats the figures. In performing our work, we did not talk with officials from all federal law enforcement agencies that may make use of the programs' data. We performed our work from May 2001 to February 2002, in accordance with generally accepted government auditing standards. We requested comments from DEA, Customs, and ONDCP. Comments from DEA are summarized at the end of this report and contained in appendix III. Purpose of the HSP and DMP Is To Produce Data That Detect Trends in Heroin Sources: According to DEA officials, the HSP and DMP produce data for detecting trends in the geographic source of heroin supplied to the United States. Officials stressed that the purpose of these data is not to provide overall estimates of where all heroin supplied to the United States originates. The HSP data are intended to provide law enforcement with a "snapshot" of where heroin at the wholesale level originates. The DMP data are intended to provide law enforcement with a snapshot of where heroin at the retail level, in certain metropolitan areas, originates. Officials explained that they believe that, over time, the snapshots begin to tell a story about what is happening with drug trafficking patterns. When this happens, officials can make their decisions, in conjunction with other investigative and intelligence data. Officials also stressed that direct comparisons should not be made between the geographic source data from the HSP and DMP. For example, the wholesale heroin seized in one market (HSP seizures) may not be intended for retail-level sale (DMP purchases) in the same market. In addition, comparisons should not be made between the HSP data and DMP data because the HSP data reflect law enforcement investigative priorities and techniques, in terms of where and how seizures are made, as well as the difficulties associated with the various concealment techniques used by smugglers. In addition, large quantity seizures of heroin from one geographic source area may boost that geographic source area's representation in the HSP data. This may be especially applicable to heroin from Southeast and Southwest Asia that has been traditionally smuggled in large, multikilogram quantities. The officials also noted that these same factors could influence year- to-year fluctuations in the proportion of heroin from each geographic source area. For example, law enforcement priorities and smuggler concealment techniques are reflected in the numerous small-quantity heroin seizures from Colombian air couriers. Federal Law Enforcement Used the HSP and DMP Data for Intelligence Purposes and Management: According to DEA officials, the HSP and DMP data are used for intelligence purposes and as a management tool by federal law enforcement. Data drawn from a variety of sources, including the HSP and DMP, are used to develop a comprehensive picture of heroin trafficking in the United States.[Footnote 11] For example, HSP and DMP data are frequently included in DEA intelligence and investigative reports to corroborate heroin trafficking trends in the United States and to inform DEA and other federal law enforcement agencies about heroin trafficking. In addition, ONDCP officials noted that the data are used as a confirmation of data from other sources, such as opium production and cultivation estimates provided by the Central Intelligence Agency. ONDCP officials noted that it uses data from the HSP and DMP for drug- flow modeling and that the data,[Footnote 12] which are viewed as one of the better heroin market indicators, are key components of ONDCP's data analysis efforts. ONDCP officials also said that data from the HSP and DMP are used for such purposes as testimony before the Congress and in ONDCP's annual National Drug Control Strategy. According to DEA, federal law enforcement also uses the HSP and DMP data as management tools to make adjustments in enforcement activities. Changes in HSP and DMP data could alert management to changing trafficking patterns. DEA cited the emergence of heroin from South America as an example of how the HSP and DMP geographic source data have been used in law enforcement intelligence and management. Southeast Asian heroin dominated the market on the East Coast until 1991; Southwest Asian heroin was also readily available. In 1991, a high-purity heroin entered the eastern U.S. market and was initially identified by DEA as high-purity but atypical Southwest Asian heroin. Intelligence reports indicated that heroin was entering the United States from South America. Also, reportedly Southwest Asian heroin producers had taught Colombians their methods of processing opium into heroin. DEA's subsequent determination of a signature unique to South America confirmed this intelligence and, as a result, South America was identified as a new supplier of heroin into the United States. Officials said that the data are also used to monitor the success of various initiatives. For example, a decrease in the amount of tested heroin that is found to have originated in a particular geographic source area can be an indicator that law enforcement initiatives against that particular area have been successful. Quantity of Heroin Seized at Ports-of-Entry by Customs but not Sent to DEA for Testing May Be Sufficient To Make a Difference in Results Reported by DEA: According to DEA officials, all ports-of-entry seizures sent to DEA by Customs are tested by DEA for geographic source. However, for several reasons, Customs is not required to send all seized heroin to DEA. DEA and Customs officials noted that Customs is not required to send to DEA abandoned heroin or heroin that is turned over to state or local officials for prosecution. "Abandoned heroin" is heroin that cannot be connected to any individual or defendant. For example, an unmanifested kilogram of heroin found in an aircraft cargo hold is considered abandoned. Under its MOU with DEA, Customs is not required to send abandoned heroin to DEA. Instead, it is to be reported on a Customs Search, Arrest, Seizure report (CF-151) and turned over to the Customs seized property custodian for destruction. Also, under the MOU, Customs does not submit for testing heroin that does not meet local U.S. attorney prosecution guidelines.[Footnote 13] Instead, Customs officials explained that in most instances, this heroin is to be turned over to state or local officials for prosecution. According to Customs officials, there are also instances in which the weight of the seized heroin is so low that it is not turned over to state or local officials for prosecution. In these instances, it is to be destroyed. Our analysis of Customs' heroin seizure data revealed that, for fiscal years 1998-2000, 57 percent of the total weight of the heroin seized by Customs was not sent to DEA. Data on the number and weight of Customs heroin seizures,[Footnote 14] including the number and weight of heroin seizures not sent to DEA for testing, are displayed in table 2. Table 2: U.S. Customs Service Heroin Seizures (by line items): Disposition[A]: Forwarded to DEA[D]; Number of line items: 1998: 244; Number of line items: 1999: 353; Number of line items: 2000: 503; Number of line items: Total: 1,100; Weight of seizures[B]: 1998: 320.1; Weight of seizures[B]: 1999: 425.9; Weight of seizures[B]: 2000: 677.1; Weight of seizures[B]: Total: 1,423.1. Disposition[A]: Not forwarded to DEA; Number of line items: 1998: 1,078; Number of line items: 1999: 708; Number of line items: 2000: 533; Number of line items: Total: 2,319; Weight of seizures[B]: 1998: 932.5; Weight of seizures[B]: 1999: 459.2; Weight of seizures[B]: 2000: 496.7; Weight of seizures[B]: Total: 1,888.4. Disposition[A]: Destroyed; Number of line items: 1998: 794; Number of line items: 1999: 380; Number of line items: 2000: 263; Number of line items: Total: 1,437; Weight of seizures[B]: 1998: 769.6; Weight of seizures[B]: 1999: 326.0; Weight of seizures[B]: 2000: 251.2; Weight of seizures[B]: Total: 1,346.8. Disposition[A]: Turned over to state or local officials or another federal agency; Number of line items: 1998: 223; Number of line items: 1999: 242; Number of line items: 2000: 164; Number of line items: Total: 629; Weight of seizures[B]: 1998: 81.9; Weight of seizures[B]: 1999: 89.0; Weight of seizures[B]: 2000: 136.1; Weight of seizures[B]: Total: 307.1. Disposition[A]: Abandoned and turned over to state or local officials or another federal agency; Number of line items: 1998: 8; Number of line items: 1999: 37; Number of line items: 2000: 5; Number of line items: Total: 50; Weight of seizures[B]: 1998: 3.0; Weight of seizures[B]: 1999: 9.8; Weight of seizures[B]: 2000: 13.5; Weight of seizures[B]: Total: 26.3. Disposition[A]: Abandoned and destroyed; Number of line items: 1998: 15; Number of line items: 1999: 17; Number of line items: 2000: 18; Number of line items: Total: 50; Weight of seizures[B]: 1998: 3.0; Weight of seizures[B]: 1999: 0.6; Weight of seizures[B]: 2000: 2.2; Weight of seizures[B]: Total: 5.8. Disposition[A]: Abandoned and pending final disposition; Number of line items: 1998: 0; Number of line items: 1999: 2; Number of line items: 2000: 0; Number of line items: Total: 2; Weight of seizures[B]: 1998: 0; Weight of seizures[B]: 1999: 3.8; Weight of seizures[B]: 2000: 0; Weight of seizures[B]: Total: 3.8. Disposition[A]: Not abandoned and pending final disposition; Number of line items: 1998: 38; Number of line items: 1999: 30; Number of line items: 2000: 83; Number of line items: Total: 151; Weight of seizures[B]: 1998: 75.0; Weight of seizures[B]: 1999: 30.0; Weight of seizures[B]: 2000: 93.7; Weight of seizures[B]: Total: 198.6. Disposition[A]: Total; Number of line items: 1998: 1,322; Number of line items: 1999: 1,061; Number of line items: 2000: 1,036; Number of line items: Total: 3,419; Weight of seizures[B]: 1998: 1,252.6; Weight of seizures[B]: 1999: 885.1; Weight of seizures[B]: 2000: 1,173.8; Weight of seizures[B]: Total: 3,311.5. [A] Customs seizure disposition categories. [B] Rounded to the nearest tenth of a kilogram. [C] Totals may not equal the individual entries due to rounding. [D] Includes 36 line items, totaling 49.9 kilograms, in which heroin was abandoned but still forwarded to DEA. Source: GAO's analysis of Customs' data. [End of table] According to DEA officials, it is not crucial to test the relatively smaller seizures. Table 3 displays amounts seized by Customs over a 3- year period. Of the total number of line items that were not forwarded to DEA, about 72 percent exceeded 100 grams in weight; these line items accounted for over 99 percent of the total weight of all line items not forwarded to DEA for testing. Table 3: U.S. Customs Service Heroin Seizures Weighing at Least 100 grams, Fiscal Years 1998-2000 (by line items): Disposition[A]: Forwarded to DEA[C]; Total number of line items: 1,100; Total weight of line items: 1,423.1; Number of line items weighing 100 grams or more: 965; Weight of line items weighing 100 grams or more[B]: 1,418.6. Disposition[A]: Not forwarded to DEA; Total number of line items: 2,319; Total weight of line items: 1,888.4; Number of line items weighing 100 grams or more: 1,675; Weight of line items weighing 100 grams or more[B]: 1,876.0. Disposition[A]: Destroyed; Total number of line items: 1,437; Total weight of line items: 1,346.8; Number of line items weighing 100 grams or more: 1,193; Weight of line items weighing 100 grams or more[B]: 1,341.3. Disposition[A]: Turned over to state or local officials or another federal agency; Total number of line items: 629; Total weight of line items: 307.1; Number of line items weighing 100 grams or more: 320; Weight of line items weighing 100 grams or more[B]: 302.0. Disposition[A]: Abandoned and turned over to state or local officials or another federal agency; Total number of line items: 50; Total weight of line items: 26.3; Number of line items weighing 100 grams or more: 15; Weight of line items weighing 100 grams or more[B]: 25.8. Disposition[A]: Abandoned and destroyed; Total number of line items: 50; Total weight of line items: 5.8; Number of line items weighing 100 grams or more: 12; Weight of line items weighing 100 grams or more[B]: 5.3. Disposition[A]: Abandoned and pending final disposition; Total number of line items: 2; Total weight of line items: 3.8; Number of line items weighing 100 grams or more: 2; Weight of line items weighing 100 grams or more[B]: 3.8. Disposition[A]: Not abandoned and pending final disposition; Total number of line items: 151; Total weight of line items: 198.6; Number of line items weighing 100 grams or more: 133; Weight of line items weighing 100 grams or more[B]: 198.0. Disposition[A]: Total; Total number of line items: 3,419; Total weight of line items: 3,311.5; Number of line items weighing 100 grams or more: 2,640; Weight of line items weighing 100 grams or more[B]: 3,294.7. Note: Rounded to the nearest tenth of a kilogram. [A] Customs seizure disposition categories. [B] Totals may not equal the individual entries due to rounding. [C] Includes 36 line items, totaling 49.9 kilograms, in which heroin was abandoned but still forwarded to DEA. The number of line items weighing 100 grams or more was 30, with a total weight of 49.7 kilograms. Source: GAO's analysis of Customs' data. [End of table] The HSP and DMP Data Could Be Improved The HSP and DMP data on the geographic source of heroin could be improved. The HSP data have limitations; appendix I of this report describes the current HSP selection methodology, its limitations, and opportunities for improvements. The DMP data also have limitations; appendix II describes the current DMP selection methodology, its limitations, and opportunities for improvements. HSP Limitations and Opportunities for Improvement: According to DEA, it does not intend that the HSP data be used either to produce estimates as to where all wholesale heroin supplied to the United States originates or as to where all wholesale heroin seized in the United States and forwarded to DEA for testing originates. [Footnote 15 Our analysis showed that the data, with some modifications to DEA's methodology, could be used to produce estimates about the geographic source of all wholesale heroin seized in the United States and forwarded to DEA for testing. To make these estimates, the DEA data must be based on a probability sample. [Footnote 16] The HSP data, however, are not based on a probability sample because not all exhibits have a known chance of being selected for testing. Consequently, there is no way to tell how the HSP sample relates to the universe of all heroin seized in the United States and forwarded to DEA for testing. Our analysis revealed an additional problem. Even if the HSP data were based on a probability sample, DEA's current methodology for reporting testing results does not include procedures to adjust for the probability of exhibits being selected for the test sample. Thus, DEA's current methodology for reporting HSP testing results would not produce valid estimates even if a probability sample were used. With these limitations in mind, opportunities exist for making improvements that would allow DEA to make valid estimates about the geographic source of all seized wholesale heroin that is sent to DEA for testing. These improvements could include modification of sampling procedures and record keeping to ensure that the HSP data are based on a probability sample and revision of its methodology for reporting testing results to include procedures to adjust for the probability of exhibits being selected for the test sample. (See appendix I of this report for detailed information.) DMP Limitations and Opportunities for Improvement: According to DEA, it does not intend that the DMP data be used to either produce estimates about retail heroin markets outside the 23 metropolitan areas covered by the DMP or about the geographic source of all retail level purchases within the 23 metropolitan areas. Our analysis found that the DMP data, with some modifications, could produce estimates about the geographic source of retail level purchases within the 23 metropolitan areas covered by the DMP, and the DMP estimates could possibly be combined across the 23 metropolitan areas. These estimates cannot be made now because of limitations in the DMP sampling and estimation procedures. The DMP data are limited for two reasons. First, our analysis showed that the purchases made by DEA agents were not made in accordance with the DEA guidelines that indicate that a certain number of purchases should be made each quarter and that the purchases should be made throughout the 3 months of each quarter. DEA officials told us that they perform periodic reviews to determine compliance with the guidelines. However, our analysis found that the required number of purchases was not always made and that they tended to occur in certain periods of each quarter and on certain days of the week. Second, the DMP contains no information on the size of the market in each of the metropolitan areas. For example, City A could have 10 out of 10,000 purchases tested by the DMP, but in City B there may be relatively few heroin users and the quarterly DMP sample could be 10 out of 1,000 purchases. The size of the markets is not known. As a result, DMP data reflect only DMP purchases. With these limitations in mind, there are opportunities for making improvements that would allow DEA to produce estimates about the geographic source of heroin purchased in the 23 metropolitan areas and to combine them across the metropolitan areas. These improvements could include taking action to ensure that DEA agents follow DEA guidelines when making the DMP purchases and utilizing alternative data sources for the total number of retail heroin purchases in an area, such as the number of hospital emergency room admissions related to heroin.[Footnote 17] (See appendix II of this report for detailed information.) Conclusions: The HSP and DMP data are used for important purposes by federal law enforcement. For instance, DEA uses the data as an indicator of the geographic source of heroin found in the United States, to measure the success of law enforcement initiatives, and to corroborate trends in heroin trafficking over time. ONDCP uses the data for drug-flow modeling, in testimony before the Congress, and in its annual National Drug Control Strategy. We recognize the challenges and difficulties of the HSP and DMP programs. However, current HSP and DMP data could be providing misleading information about the geographic source of heroin found in the United States because of sampling and statistical analysis problems. Our analysis showed that problems with sampling and statistical analysis in the HSP might lead to misleading information about the geographic source of heroin in the wholesale market. HSP data are derived from a sample of seized heroin. However, our analysis showed that DEA did not obtain a large proportion of the heroin seized by Customs, the seized heroin that was analyzed was not obtained from a random probability sample, and that the reporting methodology did not include procedures to adjust for the probability of exhibits being selected for the sample. If a snapshot of wholesale heroin geographic source is based on HSP data, then this snapshot may be misleading because accurate information about seized heroin cannot be developed from the flawed sample. Our analysis also showed that problems with sampling and statistical analysis in the DMP might lead to misleading information about the geographic source of heroin in the retail market also. DMP data are to be collected from random undercover purchases made in select metropolitan areas. However, our analysis of DMP data showed that these undercover purchases were not spread randomly over the year, as provided by DEA guidelines, but instead were concentrated in certain time periods of the quarter. Therefore, if the retail market characteristics vary over time or vary between midweek and weekend, the data could produce results that would be different from those that might have been obtained had these guidelines been followed. Furthermore, without knowledge of the size of the retail markets in the sampled metropolitan areas there are difficulties in combining DMP results across those metropolitan areas. The quality/validity of data derived from these two programs could be improved by more careful sampling, adhering to existing DEA guidelines, and enhancing data analysis. Recommendations: To help improve the HSP and DMP data on the geographic source of heroin, we recommend that the attorney general direct the administrator of DEA to: * ensure that the HSP data are based on a probability sample so that all HSP exhibits have a known chance of selection, * revise the HSP methodology for reporting testing results to include procedures to adjust for the probability of exhibits being selected for the test sample, * take action to ensure that DMP purchases are made according to DEA guidelines, and, * study the utilization of alternative data sources for an estimate of the total number of retail heroin purchases in an area that could allow the DMP data to be combined across metropolitan areas. To enhance the usefulness of HSP data, we recommend that the attorney general and the secretary of the treasury direct the administrator of DEA and commissioner of Customs, respectively, to enter into discussions to determine whether additional seized heroin should be forwarded to DEA by Customs. Agency Comments: We requested comments on a draft of this report from DEA, Customs, and ONDCP. In its comments, DEA indicated that it strongly disagreed with two of our recommendations and concurred with the remaining three. (See appendix III.) DEA disagreed with our recommendations that it (1) ensure the HSP data are based on a probability sample and (2) revise the HSP methodology for reporting testing results. DEA said that the HSP should remain a program whose data are based solely on the results of signature analysis. This seems to imply that the application of statistical analysis to signature testing results would yield unscientific data. We disagree. Without the use of data based on a probability sample, it is impossible to know how to interpret the HSP data. In addition, DEA seems to imply that we want the signature of untested exhibits to be imputed based on the results of actual analyses of tested exhibits. This is incorrect. We recommended the use of a probability sample and standard weighting procedures that would allow the estimation of the geographic source of all seized heroin. DEA also said that it should implement a stratified sample similar to the one we proposed to ensure that a significant portion of the total weight of heroin seized by DEA is sampled. However, DEA did not want to use the stratified sample to produce estimates with the resulting data. We believe that by implementing a stratified probability sample, DEA could produce estimates of the seized heroin, which would improve the overall data. Furthermore, DEA suggested expanding the number of DEA and Customs exhibits submitted for analysis. However, this method would increase reliability only if a probability sample were used. Expanding the number of exhibits submitted without using a probability sample limits the interpretation of the data to the tested exhibits alone. Finally, DEA said that the estimation model proposed in our report was simplistic. We provided this straightforward model only as an example and strongly endorse any attempt DEA might make to enhance the suggested model. DEA concurred with the remainder of our recommendations, that it take action to ensure that DMP purchases are made according to DMP guidelines; study the utilization of alternative data sources for an estimate of the total number of retail heroin purchases in an area; and that the attorney general and the secretary of the treasury direct DEA and Customs to enter into discussions to determine whether additional seized heroin should be forwarded to DEA by Customs. DEA concurred with our recommendation that it study the utilization of alternative data sources for the DMP. DEA also commented that data sources do not exist that measure either the number of retail heroin purchases or the prevalence of heroin abuse in a metropolitan area While we recognize that these data sources may not exist, we suggested that the DAWN data could provide a useful surrogate measure; without such a measure, the DMP data could be misleading. DEA concurred with our recommendation that DEA and Customs discuss whether additional seized heroin should be forwarded to DEA. However, DEA incorrectly characterized our recommendation by saying we recommended they work to ensure that more seizures are sampled. We did not make this recommendation. We recommended only that DEA and Customs discuss whether additional heroin should be forwarded. DEA officials also provided additional technical comments, which we have incorporated where appropriate. Customs and ONDCP said that they had no comments on the draft report. As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies of this report to the cochairman, Senate Caucus on International Narcotics Control; the attorney general; the secretary of the treasury; the administrator of DEA; the commissioner of Customs; and the director, Office of National Drug Control Policy. We will also make copies available to others upon request. This report will also be available on GAO's home page at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please call me or Weldon McPhail at (202) 512-8777. Other key contributors to this report were Doris Page, Mark Ramage, Anthony Patterson, David Alexander, and Geoffrey Hamilton. Sincerely yours, Signed by: Laurie E. Ekstrand: Director, Justice Issues: [End of section] Appendix I: Heroin Signature Program: The Heroin Signature Program (HSP) sample was selected using sampling procedures for two categories of eligible[Footnote 18] exhibits. The first category included exhibits from seizures made by the Drug Enforcement Administration (DEA) and the Washington D.C., Metropolitan Police Department. This sample is referred to as the "DEA exhibit sample." The second category was comprised of all other exhibits, including those seizures submitted to DEA by the Federal Bureau of Investigation (FBI), and those submitted to DEA by the U.S. Customs Service from passengers, luggage, or cargo on flights that originated outside of the United States. This sample is referred to as the "POE exhibit sample." (See table 1 of this report for additional details on the source and selection of HSP heroin.) HSP Methodology: DEA calculated the results of the HSP signature analyses in the following manner: * First, the total weight of heroin from sampled exhibits from each geographic source was computed.[Footnote 19] * Second, the total weight of heroin from sampled exhibits from all geographic sources was combined. * Last, the total weight of heroin from each geographic source was divided by the total weight of heroin for all areas. The result was expressed as a percentage. We analyzed calendar year 1999 and 2000 data in our review of the HSP. Because 1999 was the most recent year for which finalized reports were available, some analyses are based solely on the 1999 HSP data. According to DEA, there were no methodological changes from 1999 to 2000. HSP analyses results for calendar year 1999 are displayed in table 4. Table 4: Geographic Source and Percentage of Seized Heroin, Calendar Year 1999: Geographic source: South America; Percentage of seized heroin: 60%. Geographic source: Mexico; Percentage of seized heroin: 24%. Geographic source: Southeast Asia; Percentage of seized heroin: 10%. Geographic source: Southwest Asia; Percentage of seized heroin: 6%. Source: DEA. [End of table] HSP Limitations Our analysis of the HSP methodology found that the HSP data, with modifications, could produce valid estimates about the geographic source of all wholesale heroin seized in the United States and forwarded to DEA for testing. These estimates could not currently be made because not all exhibits had a known chance of being selected for testing; therefore, the HSP data were not based on a probability sample. In addition, our analysis revealed that, even if the data were based on a probability sample, DEA's current methodology that is used to report testing results would not produce valid estimates, because it does not include procedures to adjust for the probability of exhibits being selected for the test sample. HSP Data Not Based on a Probability Sample: Our analysis showed that, for the following three reasons, not all exhibits had a known positive chance of being selected for testing. 1. The procedure for selecting the POE exhibit sample resulted in the selection of heavier exhibits. DEA laboratory personnel were responsible for identifying POE sample exhibits for testing and, according to DEA, selected exhibits generally tended to be the heaviest exhibit from a case. For example, if a POE seizure resulted in 3 exhibits weighing 10 gram, 15 grams, and 50 grams, it was likely that the 50-gram exhibit was tested. While not conclusive, the DEA exhibit sample also contained heavier than expected exhibits, suggesting that the heavier exhibits may generally have been selected for testing. Although only about 13 percent of the DEA exhibits were sampled in 1999, these sampled exhibits accounted for about 45 percent of the total weight of seized wholesale heroin sent to DEA for testing.[Footnote 20] 2. Exhibits did not have a known chance of selection because it appears that DEA did not consistently follow its policy for selecting exhibits, for either the POE or the DEA exhibit samples.[Footnote 21] The policy requires that, except for special request exhibits, no more than one exhibit per case be chosen. However, based on our review of 1999 and 2000 HSP data, this policy was apparently not implemented consistently. There were multiple exhibits for individual cases in the HSP sample for both 1999 and 2000. In 1999, about 9.5 percent and 8.4 percent of the DEA and the POE sample exhibits, respectively, were from cases that had multiple exhibits in the sample. For 2000, 10.5 percent and 3.4 percent of the DEA and the POE sample exhibits, respectively, were from cases that had multiple exhibits in the sample. 3. Exhibits for the DEA exhibit sample also did not have a known chance of selection because DEA's sampling procedures are apparently not working as intended. Before the start of each 6-month sampling period, DEA produced a list of random numbers for the DEA exhibit sample. A separate list of random numbers was sent to each DEA regional laboratory. The random numbers were to contain approximately 20 percent of the exhibits expected in the sampling period. However, our analysis showed that the actual percentage of exhibits sampled was 13 percent for 1999 and 8 percent for 2000.[Footnote 22] DEA's Methodology Used to Report HSP Testing Results Did Not Include Procedures to Adjust for the Probability of Exhibits Being Selected for the Test Sample: If DEA developed a probability sample, it would still be unable to produce valid estimates because its current methodology for reporting HSP testing results does not include procedures to adjust for the probability of exhibits being selected for the test sample. In computing estimates using HSP data, DEA did not take into account that some exhibits had a low probability of selection and some had a high probability of selection. As a result, DEA produced only simple tabulations of sample data that could not be used to produce estimates about the geographic source of all seized wholesale heroin sent to DEA for testing. HSP Opportunities for Improvements: Opportunities exist for making improvements that would allow DEA to estimate the geographic source of all seized wholesale heroin that is sent to DEA for testing. These improvements could include (1) modification of sampling procedures and record keeping to ensure that the HSP data are based on a probability sample and (2) revision of its methodology for reporting testing results to include procedures to adjust for the probability of exhibits being selected for the test sample. If these improvements were made, it would also be possible to produce confidence intervals for estimates.[Footnote 23] Modification of Sampling Procedures to Ensure That HSP Data Are Based on a Probability Sample: HSP sampling procedures could be improved to ensure that all exhibits have a known chance of selection. For example, DEA could use a stratified sample in which exhibits are divided into different strata, or categories, and sample selections are made from each stratum. The heaviest exhibits and the exhibits DEA considers to have special intelligence importance could all be selected,[Footnote 24] and a portion of the remaining exhibits could also be chosen. Table 5 shows this alternative sample design. Table 5: Stratified HSP Sample Design: Stratum number: 1. Description of exhibits: POE exhibits weighing at least y grams and those for which DEA makes a special request; Chance of selection: 100. Stratum number: 2. Description of exhibits: DEA exhibits weighing at least x grams and those for which DEA makes a special request; Chance of selection: 100. Stratum number: 3. Description of exhibits: POE exhibits weighing less than y grams and for which DEA does not make a special request; Chance of selection: K. Stratum number: 4. Description of exhibits: DEA exhibits weighing less than x grams and for which DEA does not make a special request; Chance of selection: H. Note: Values for x, y, h, and k could be chosen to achieve the desired sample sizes, and sampling errors and confidence intervals, and to take into account special requests made by DEA to meet intelligence needs. Source: GAO's analysis. [End of table] This suggested design would result in a known chance of selection for all heroin exhibits. All exhibits in strata 1 and 2 would be selected Exhibits in strata 3 and 4 would have a less than 100 percent chance of selection. For example, if k equals 5, then 5 percent of exhibits in stratum 3 would be selected. If h equals 20, then 20 percent of exhibits in stratum 4 would be selected. DEA should also draw the sample and centrally maintain sufficient records in such a way that the resulting sample selection could be verified. For example, * if exhibits are selected manually, with the use of a list of random numbers, the exhibits should first be numbered sequentially, before the random number list is used to identify sample exhibits; * after identifying the sample exhibits, DEA should ensure that sufficient information is maintained to identify each exhibit, indicate whether the exhibit was selected for the sample, and whether it was sent for signature analysis testing; and; * if an exhibit, that is ineligible for testing due to its low weight, were selected by the sampling procedure, it should be recorded as "insufficient for signature analysis." Revision of Current Methodology to Include Procedures to Adjust for the Probability of Exhibits Being Selected for the Test Sample: Several changes to DEA's methodology are possible that would include procedures to adjust for the probability of exhibits being selected for the test sample. For example, DEA could use statistical weights for the sample design to produce estimates about the geographic source of all seized wholesale heroin sent to DEA for testing. These statistical weights would be determined by calculating a value that is the inverse of an exhibit's chance of selection. For example, referring to the example following table 5, all exhibits in stratum 1 would be selected and 5 percent of exhibits in stratum 3 would be selected. Stratum 1 sample exhibits would get a weighting factor of 1/1=1 and stratum 3 sample exhibits would get a weighting factor of 1/.05 = 20. DEA could also use supplemental data to improve the estimates. DEA could use data on the total weight of seized heroin sent to DEA for testing, by regional laboratory. This same quantity could be estimated from the HSP sample. The ratio of these two quantities could then be computed and used as a refinement to the statistical weights described above. For example, if one DEA regional laboratory received a total of 210 kg of heroin, and the sample estimate of heroin for that regional laboratory was 200 kg, then the final statistical weight for sample data from that regional laboratory would be the initial statistical weight multiplied by 210/200=1.05. Using the weighting factor for stratum 3 (from above), the final statistical weight for stratum 3 sample exhibits selected from that laboratory would be 20 x 1.05=21. Estimates of the percentage of seized heroin by geographic source area would then be computed as follows: * Compute the total weighted sum of heroin in grams[Footnote 25] for each of six categories.[Footnote 26] The weighted sum would use the final statistical weighting factors described earlier.[Footnote 27] This would produce six sums, H1, ..., H6; * Compute the total over all six categories, H = H1 + ... + H6; * The ratio of each category to the total (converted to percentages) would then yield estimates of the percentages of seized wholesale heroin, sent to DEA for testing, by geographic region. For example, the percentage from geographic region i would be pi = 100(Hi/H). [End of section] Appendix II: Domestic Monitor Program: DEA produced tables using the Domestic Monitor Program (DMP) data. For example, one table displayed, by metropolitan area, the number of exhibits from each geographic source. Table 6 displays this information for calendar year 1999. Table 6: By Metropolitan Area, the Number of Exhibits from Each Geographic Source, Calendar Year 1999: Metropolitan area: Atlanta, Georgia; Southeast Asia: 9; Southwest Asia: 1; Mexico: 1; South America: 7; Unknown or insufficient: 7. Metropolitan area: Baltimore, Maryland; Southeast Asia: 2; Southwest Asia: 0; Mexico: 0; South America: 33; Unknown or insufficient: 4. Metropolitan area: Boston, Massachusetts; Southeast Asia: 0; Southwest Asia: 0; Mexico: 0; South America: 25; Unknown or insufficient: 12. Metropolitan area: Chicago, Illinois; Southeast Asia: 6; Southwest Asia: 1; Mexico: 0; South America: 14; Unknown or insufficient: 18. Metropolitan area: Dallas, Texas; Southeast Asia: 1; Southwest Asia: 0; Mexico: 29; South America: 0; Unknown or insufficient: 6. Metropolitan area: Denver, Colorado; Southeast Asia: 0; Southwest Asia: 0; Mexico: 23; South America: 0; Unknown or insufficient: 5. Metropolitan area: Detroit, Michigan; Southeast Asia: 3; Southwest Asia: 4; Mexico: 0; South America: 22; Unknown or insufficient: 8. Metropolitan area: El Paso[A], Texas; Southeast Asia: 0; Southwest Asia: 0; Mexico: 6; South America: 0; Unknown or insufficient: 0. Metropolitan area: Houston, Texas; Southeast Asia: 0; Southwest Asia: 0; Mexico: 38; South America: 3; Unknown or insufficient: 8. Metropolitan area: Los Angeles, California; Southeast Asia: 0; Southwest Asia: 0; Mexico: 18; South America: 0; Unknown or insufficient: 7. Metropolitan area: Miami, Florida; Southeast Asia: 0; Southwest Asia: 0; Mexico: 4; South America: 18; Unknown or insufficient: 10. Metropolitan area: Newark, New Jersey; Southeast Asia: 1; Southwest Asia: 0; Mexico: 0; South America: 36; Unknown or insufficient: 7. Metropolitan area: New Orleans, Louisiana; Southeast Asia: 0; Southwest Asia: 0; Mexico: 0; South America: 14; Unknown or insufficient: 7. Metropolitan area: New York, New York; Southeast Asia: 0; Southwest Asia: 0; Mexico: 0; South America: 54; Unknown or insufficient: 4. Metropolitan area: Orlando, Florida; Southeast Asia: 0; Southwest Asia: 0; Mexico: 0; South America: 15; Unknown or insufficient: 4. Metropolitan area: Philadelphia, Pennsylvania; Southeast Asia: 0; Southwest Asia: 1; Mexico: 0; South America: 33; Unknown or insufficient: 5. Metropolitan area: Phoenix, Arizona; Southeast Asia: 0; Southwest Asia: 0; Mexico: 38; South America: 0; Unknown or insufficient: 2. Metropolitan area: San Diego, California; Southeast Asia: 0; Southwest Asia: 0; Mexico: 30v South America: 0; Unknown or insufficient: 3. Metropolitan area: San Francisco, California; Southeast Asia: 0; Southwest Asia: 0; Mexico: 31; South America: 0; Unknown or insufficient: 3. Metropolitan area: San Juan, Puerto Rico; Southeast Asia: 0; Southwest Asia: 0; Mexico: 0; South America: 24; Unknown or insufficient: 6. Metropolitan area: Seattle, Washington; Southeast Asia: 0; Southwest Asia: 0; Mexico: 37; South America: 0; Unknown or insufficient: 3. Metropolitan area: St. Louis, Missouri; Southeast Asia: 0; Southwest Asia: 0; Mexico: 33; South America: 0; Unknown or insufficient: 6. Metropolitan area: Washington, D.C. Southeast Asia: 4; Southwest Asia: 1; Mexico: 0; South America: 20; Unknown or insufficient: 4. Metropolitan area: Total; Southeast Asia: 26; Southwest Asia: 8; Mexico: 288; South America: 318; Unknown or insufficient: 142. [A] El Paso was added to the DMP in mid-1999. Source: DEA. [End of table] DMP Limitations Our analysis of the DMP methodology found that the DMP data, with modifications, could (1) produce estimates about the geographic source of retail heroin purchases in the 23 metropolitan areas covered by the DMP and (2) possibly be combined across the 23 areas. These estimates could not currently be made because the DMP data have the following limitations. First, the purchases were not made by DEA agents in accordance with the DEA guidelines that provide that a certain number of purchases be made each quarter and that the purchases be made throughout each quarter. Second, DMP data analysis did not take into account the size of the heroin market in each of the metropolitan areas. DMP Purchases Are Not Made in Accordance with DEA Guidelines: DEA agents did not make the DMP purchases in accordance with the DEA guidelines that provide that a certain number of purchases be made each quarter and that the purchases be made throughout each quarter. A review of the 1999 DMP data showed that the DEA guidelines, that provide that 10 purchases be made in each quarter (20 in New York City) and that the DMP purchases be spread over the quarter, were not consistently met. For example, in the first quarter there were less than 5 purchases made in 3 of the metropolitan areas. Our analysis also showed that a disproportionate number of DMP purchases occurred in a certain month of a quarter, and on certain days of the week. If the purchases were spread randomly throughout each quarter, approximately a third of the purchases would be expected each month, and approximately one seventh of the purchases would be expected each day of the week. However, based on our analysis of the DMP data, the purchase dates were not random by day of the week or by month of quarter. As a result, it is unlikely that the DMP data for 1999 were derived from a sample in which the days in each quarter selected for purchases were chosen with a known, equal probability of selection.[Footnote 28] In addition, assuming that the characteristics of retail heroin might change over time, biases may have been introduced due to oversampling in certain time periods within the quarter. For example, if the geographic source of heroin supplied to one location changed between the beginning and the end of a quarter, making most of the purchases during one part of the quarter would not reflect the geographic source of the heroin over the entire quarter. DEA Does Not Take into Consideration the Size of the Heroin Markets: The total number of retail sales in each metropolitan area in each quarter is not known. As a result, it is difficult to appropriately combine DMP data across metropolitan areas. For example, in one metropolitan area (City A) there may be 1,000 retail sales in a quarter, but in another area (City B) there may be 5,000 retail sales in a quarter. So, for City A, DMP would sample and analyze 10 purchases representing 1,000 sales, but for City B, the 10 purchases would represent 5,000 sales. Without knowing how many purchases are represented by the DMP purchases, the data should not be combined across metropolitan areas. DMP Opportunities for Improvements: Opportunities exist for making improvements that would allow DEA to estimate the geographic source of heroin purchases in the 23 metropolitan areas and possibly to combine them across the areas. These improvements could include ensuring that DEA agents follow guidelines when making the purchases and using alternative data sources for the total number of retail heroin purchases in an area. Ensure that DEA Agents Follow Guidelines When Making DMP Purchases: DEA could take actions to ensure compliance with its own guidelines. For example, a list of randomly chosen dates on which purchases are to be made each quarter could be sent to each metropolitan area. Additionally, actual purchase dates in each metropolitan area could be more closely monitored, to ensure that purchases are made according to scheduled dates. DEA Should Study the Use of Alternative Data Sources for the Size of the Heroin Markets: To calculate the chance of selecting any one purchase, the total number of retail heroin sales per quarter for each metropolitan area is needed. It is unlikely that this number would ever be known. However, DEA could study using alternative data sources as a substitute for the total number of retail heroin sales. For example, the Drug Abuse Warning Network (DAWN) collects information on hospital emergency room admissions that are drug abuse related. If the number of heroin purchases in a metropolitan area is proportional to emergency room mentions for heroin-related admissions, then DAWN could provide information on the relative number of heroin purchases by metropolitan area. [End of section] Appendix III: Comments from the Drug Enforcement Administration: U.S. Department of Justice: Drug Enforcement Administration: Washington, D.C. 20537: March 18, 2002: Laurie E. Ekstrand, Director: Justice Issues Division: General Accounting Office: 441 G Street, N.W., Room 2A38: Washington, D.C. 20548: Dear Ms. Ekstrand: This is to provide you with the Drug Enforcement Administration's (DEA) formal response to the General Accounting Office's (GAO) recently completed draft report, "Drug Control: DEA Could Improve Its Heroin Signature and Domestic Monitor Programs' Geographic Source Data" (GAO-02-416). Technical comments have previously been provided to the GAO addressing corrections for accuracy and sensitivity concerns. DEA submits the following comments on the facts and findings of this report. DEA has reviewed the draft report and takes strong exception with some of GAO's conclusions as noted within this response. The GAO was directed to review the Heroin Signature Program (HSP) and the Domestic Monitor Program (DMP) to determine the following: * the purpose of the HSP and the DMP; * how federal law enforcement uses the HSP and DMP data; * whether the quantity of heroin seized at ports-of-entry by the U.S. Customs Service (USCS) that is not sent to DEA laboratories for analysis is of sufficient quantity to make a difference in the HSP results; and; * whether the HSP and the DMP could be improved. Based on interviews with officials from the DEA, USCS, and the Office of National Drug Control Policy, as well as an analysis of HSP and DMP data, the GAO made the following recommendations (DEA has provided numbering for easier reference): To help improve the HSP and the DMP data on the geographic source of heroin, we recommend that the Attorney General direct the Administrator of DEA to: 1. Ensure that the HSP data are based on a probability sample so that all HSP exhibits have a known chance of selection. 2. Revise the HSP methodology for reporting testing results to include procedures to adjust for the probability of exhibits being selected for the test sample. DEA Response: DEA strongly disagrees with these first two GAO recommendations regarding the HSP. The intent of those recommendations is to allow the estimation of source areas for heroin seizures that have not been submitted for signature analysis. DEA believes that the HSP should remain a program whose source area figures are based solely on the results of scientifically based signature analysis. The HSP should not be converted to a program that relies on a fairly simplistic probability model to estimate the geographic source of heroin seizures. DEA believes that other measures can and should be taken that would improve the HSP information while retaining the scientific basis of that information. The HSP looks at the wholesale side of the domestic heroin trafficking situation. Included in the program are samples drawn from USCS seizures at ports-of-entry that are submitted to DEA regional laboratories for analysis. These provide insight into the routes and methods used to smuggle heroin into the country. Also sampled are randomly selected DEA seizures and purchases that are submitted to DEA laboratories. These samples provide a glimpse into wholesale distribution patterns within the country. HSP samples are submitted to DEA's Special Testing and Research Laboratory (STRL) for heroin signature analysis. Initiated in 1977, heroin signature analysis is based on an exhaustive chemical profile of authentic samples acquired from each of the four major heroin source areas: Mexico, South America, Southeast Asia, and Southwest Asia. Heroin signature analysis is the only scientifically based source of information currently available on the origins of heroin encountered in the U.S. drug market. Each year, through the HSP, an in-depth chemical analysis is performed on an average of 600 to 900 samples taken from heroin seizures and purchases made in the United States. As a result of heroin signature analysis, DEA chemists at the STRL are able to associate the heroin samples with a heroin production process, or signature, which is indicative of a particular geographic source area. The resultant proportion of heroin associated with each geographic source area is measured in terms of the net weight of heroin seized and analyzed in the program from each area that year. The HSP is not intended to provide U.S. market shares for each of the heroin source areas in any given year. Moreover, fluctuations from year to year in the proportion from each source area may reflect shifting law enforcement priorities, significant seizures, as well as changing trafficking patterns. Rather, data from the HSP is used in conjunction with investigative intelligence and with drug production and seizure data to develop an overall, long-term assessment of the trafficking of heroin to and within the United States. The GAO recommends that the HSP estimate signatures for heroin exhibits submitted to DEA field laboratories that did not undergo signature analysis at the STRL. If implemented, the recommendations would accomplish little while compromising the scientific basis of the HSP results. The HSP would remain a seizure-based program; it still would not provide U.S. market shares for each of the heroin source areas. Moreover, the geographic source statistics would not be based entirely on signature analysis, but would also be generated by a probability model. The probability model proposed by the GAO to estimate signatures are a standard estimation model, but fairly simplistic for this application. To estimate a signature for a seizure, the model only takes into account the DEA regional laboratory where the seizure was submitted. The model considers no other seizure characteristics, such as the location of the seizure, concealment method, transportation mode, or point of origin of the drug shipment. Recent trends indicate the complexity of heroin trafficking. For example, in 2000 HSP signature analysis determined that several seizures were comprised of both Southeast Asian and Southwest Asian heroin. In some instances, the two types were commingled. Instead of estimating heroin signatures, DEA recommends that measures be taken to increase the number of USCS and DEA seizures submitted for HSP signature analysis as noted in GAO's recommendation 5. The goal would be to include in the HSP as much of the total weight of heroin seizures as possible. To that end, DEA recommends other measures that can be taken to increase the portion of DEA heroin exhibits that are sampled for the HSP. At this time, the random sample of DEA exhibits is based on the total number of exhibits, not the weight of those exhibits. DEA recommends that this sample is redesigned from a simple random sample to a stratified sample in which DEA heroin exhibits above a certain threshold weight are selected with certainty and exhibits falling into other weight strata are selected with differing probabilities. This sample design is similar to the one proposed by the GAO, but its goal is different. The purpose of the proposed DEA sample is not to estimate signatures for unsampled seizures, but to ensure that a significant portion of the total weight seized by the DEA is sampled for HSP signature analysis at the STRL. 3. Take action to ensure that DMP purchases are made according to DMP guidelines; and; DEA Response: DEA concurs with this recommendation. The guidelines indicate that a certain number of DMP buys should be made each quarter and those purchases should be made throughout the quarter. Despite periodic reviews by the DEA Intelligence Division (NC) and the Office of Inspections of field compliance with the DMP guidelines, the required number of purchases is not always made. Also, the purchases are sometimes not spread over the entire quarter. DEA concurs that DEA management should continue to emphasize the importance of the DMP and take all possible steps to improve compliance with the guidelines. 4. Study the utilization of alternative data sources for an estimate of the total number of retail heroin purchases in an area that could allow the DMP data to be combined across metropolitan areas. DEA Response: DEA concurs with this recommendation to evaluate using other data sources such as the Drug Abuse Warning Network, which provides drug- related emergency room admissions data, that could be used to weight the metropolitan-area data to overall DMP figures. Although such data sources cannot provide an accurate measure of the size and scope of retail heroin market in an individual location, they can provide a means of weighting the DMP metropolitan data to more meaningful DMP total figures. Nonetheless, both the limitations of the data sources used as weights and the fact that the DMP is not a national sample preclude considering such weighed overall DMP figures as national estimates. In its discussion of DMP limitations and opportunities for improvement, the GAO notes that "DEA does not intend that the DMP data be used to either produce estimates about retail heroin outside the 23 metropolitan areas covered by the DMP or about the geographic source of all retail level purchases within the 23 metropolitan areas." That is correct. It is not possible to determine accurately the scope and size of each metropolitan-area retail market. Consequently, a DMP probability sample for each metropolitan area cannot be designed and estimates cannot be produced from the DMP data. The GAO suggests "utilizing alternative data sources for the total number of retail purchase in an area, such as the number of hospital emergency room admissions related to heroin." Emergency room data only measure the health consequences of drug abuse. Data sources do not exist that measure either the number of retail heroin purchases or the prevalence of heroin abuse in a metropolitan area. Moreover, even if such data sources existed, the DMP metropolitan-area data would have to be weighted to such an extent that the resulting estimates would be very unreliable. Again, it should be noted that like the HSP, the DMP is not intended to provide U.S. market shares for each of the heroin source areas in any given year. Rather than try to artificially convert the DMP to a probability sample, the DMP should continue to be used as it is now, as an indicator of trends in the retail market that is used in conjunction with other information to develop an overall, long-term assessment of the trafficking of heroin within the United States. DEA objects to forcing these programs to serve purposes other than those that DEA has established for these programs. DEA does not agree with GAO's conclusion that "the DM? data, with some modifications, could produce estimates about the geographic source of retail level purchases within the 23 metropolitan areas covered by the DMP." As noted above, there are no accurate measurements of local heroin markets upon which to base such an estimate. 5. The GAO also recommends that the Attorney General direct the Administrator of the DEA, and that the Secretary of the Treasury direct the Commissioner of the USCS, to enter into discussions to determine whether the additional seized heroin should be forwarded to the DEA by USCS [for inclusion in the HSP] DEA Response: In its review, the GAO determined that the USCS is not submitting samples of a substantial portion of its heroin seizures to DEA laboratories. DEA concurs with GAO's recommendation that the DEA and USCS work together to ensure that more USCS seizures are sampled for the HSP signature analysis. This would provide critical additional information from port-of-entry seizures. In sum, DEA concurs with recommendations 3, 4 and 5, but strongly disagrees with recommendations 1 and 2 and the conclusions supporting these recommendations. DEA will follow up with GAO on a course of action to address the recommendations included in this report. If you have any questions regarding this matter, please contact Acting Deputy Chief Inspector, Wayne Nicks or Marjorie Snider, DEA's Audit Liaison, at 202-307-8200. Sincerely, Signed by: Asa Hutchinson: Administrator: cc: Vickie L. Sloan: Director: Audit Liaison Office: Justice Management Division: [End of section] Footnotes: [1] GAO's Office of Special Investigations provided you with information on the two programs. U.S. General Accounting Office, Review of the Drug Enforcement Administration's Heroin Signature and Domestic Monitor Programs, [hyperlink, http://www.gao.gov/products/GAO-01-237R], (Washington, D.C.: 2001). [2] As noted, this report does not focus on the purity and price aspects of the programs. [3] As used in the context of the two programs, "wholesale" represents heroin seizures at ports-of-entry and seizures and purchases from heroin dealers made elsewhere in the United States; "retail" represents individual-use heroin purchases. [4] We chose these years as they represented the most recent data available at the time of our analysis. [5] The regions are: Southeast Asia, Southwest Asia, South America, and Mexico. The South American heroin signature was developed in 1993. This heroin is produced mainly in Colombia. [6] DEA operates the STRL in addition to its regional laboratories. [7] According to DEA, a cooperating source is an individual who performs an investigative activity or provides information regarding drug trafficking, with a reasonable expectation of confidentiality, under the direction and control of DEA personnel. [8] The DMP is a collection of undercover retail purchases in the 23 metropolitan areas. The DEA uses undercover purchases as a substitute for actual retail purchases in these 23 metropolitan areas. [9] DEA maintains signature analysis and HSP and DMP data on a calendar-year basis. Customs maintains seizure data on a fiscal-year basis. [10] Location was defined as airport or other. [11] Other data sources include opium cultivation/production estimates; investigative intelligence; other heroin seizure data such as where, when, and from whom heroin was seized; data from DEA's System To Retrieve Information from Drug Evidence; and drug abuse indicator data. [12] The data are used to develop information on the flow of drugs into the United States. [13] Each of the 93 U.S. attorneys has discretion as to the minimum threshold weight that will trigger prosecution in that federal judicial district. [14] Line items from Customs' database. [15] For our analysis, seized wholesale heroin is that seized heroin that has been sent to DEA for signature analysis. [16] A probability sample means that each exhibit has a known positive chance of being selected, and this chance could be computed. [17] The Drug Abuse Warning Network (DAWN), administered by the U.S. Department of Health and Human Services' Substance Abuse and Mental Health Services Administration, provides these data. [18] In order for an exhibit to be eligible for HSP testing, the exhibit must have contained enough heroin to ensure that a sufficient amount remained after removal of a portion of the heroin for testing. [19] Excluded from these sums were heroin amounts associated with sample exhibits for which a source region could not be determined. [20] In 1999, DEA laboratories received a total of 2,510 exhibits eligible for the random sample. For the same time period, 327 exhibits were selected for the random sample, with a total weight of 151,706 grams. [21] DEA did not provide FBI exhibit data for this analysis. [22] In 1999, the regional laboratory sample consisted of 327 exhibits out of a total of 2,510; in 2000, the sample consisted of 218 exhibits out of a total of 2,598. [23] A particular probability sample is only one of a large number of samples that might have been drawn using the same sampling procedure. Estimates derived from the different samples would differ from each other. Confidence in the precision of a particular sample's results is expressed as a "confidence interval." For example, we may be 95 percent confident that the true population value is within plus or minus 7 percentage points of a sample estimate. [24] If, to meet intelligence needs, DEA continues to make special requests for signature analysis of certain exhibits (not already included in strata 1 or 2 of table 5), the data for these special request exhibits should not be included in the final estimates of the geographic source of seized heroin, because they would already have had a chance of selection. [25] DEA's computations did not account for differences in heroin purity levels. DEA could take this into account by using the amount of pure heroin, if the total amount of pure heroin is of interest. If the substance of interest were grams of various mixtures of heroin and adulterants, then an adjustment for purity would not be necessary. However, if a purity adjustment were not made, it would be difficult to describe what the estimates represented. [26] The four geographic source areas plus "unknown" and "insufficient." DEA's current HSP data tables do not report unknown and insufficient. [27] Statistical weighting is an adjustment to data that takes into account the probabilities of selection. [28] Similar patterns were found in the calendar year 2000 data. [End of section] GAO’s Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site [hyperlink, http://www.gao.gov] contains abstracts and fulltext files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to [hyperlink, http://www.gao.gov] and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office: 441 G Street NW, Room LM: Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov: (202) 512-4800: U.S. General Accounting Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: