The Department of Homeland Security (DHS) is considering upgrades to its current biological threat detection system but the ability to make informed decisions to enhance the Generation-2 BioWatch system is hindered by a lack of information on its ability to detect an attack, a federal audit agency says.
“DHS lacks reliable information about BioWatch Gen-2’s technical capabilities to detect a biological attack and therefore lacks the basis for informed cost-benefit decisions about possible upgrades or enhancements to the system,” the Government Accountability Office (GAO) says in a report released in late November.
GAO says that DHS hasn’t developed technical performance requirements based on operational objectives to assess the Gen-2’s capability to detect a biological attack and instead relies on modeling and simulation studies.
“However,” GAO cautions, “these studies have not directly and comprehensively assessed the capabilities of the Gen-2 system,” adding that “in our review of the tests that have been conducted, we found there are limitations and uncertainties in the test results on the technical performance characteristics of the Gen-2 system.”
The Gen-2 system consists of 600 aerosol collectors in more than 30 major urban areas nationwide. Deployments include outdoor and indoor locations and capacity for major events. The collectors draw in air through filters that are manually retrieved and analyzed at state and local laboratories with actionable results taking between 12 and 36 hours to declare, including the time taken to collect the sample.
The National Academies of Science said in a 2011 report that the Gen-1 and Gen-2 systems were rapidly deployed without sufficient testing, validation and evaluation of their technical capabilities. In April 2014 DHS terminated a program to develop and acquire a Gen-3 version of BioWatch that would have provided autonomous detection and reporting capabilities to dramatically reduce the time between collection of a sample and a threat warning but the effort was cancelled because the department said the costs of the new system outweighed the benefits.
Northrop Grumman [NOC] developed and field tested a Gen-3 system for DHS although the department had planned to reopen the program to competition before it was cancelled.
Some of the equipment in Gen-2 will reach the end of its life-cycle within the next year, which means DHS needs to make decisions about investing in the program, says the GAO report, Biosurveillance: DHS Should Not Pursue BioWatch Upgrades or Enhancements Until System Capabilities are Established (GAO-16-99). The report also says that DHS officials are considering technology upgrades and improvements to the Gen-2 system.
GAO also says that the DHS Office of Health Affairs, which operates BioWatch, and the Science and Technology Directorate, which is responsible for developing new bio-threat detection capabilities, continue “collaborating on next steps for the BioWatch program in an attempt to address the capability gap that Gen-3 was intended to fill.”
One upgrade DHS is considering to the Gen-2 system is autonomous detection, GAO says, but the report cautions that “potential benefits of an autonomous system for BioWatch depend on specific assumptions, some of which are uncertain. For example, reductions in casualties would depend on a rapid, coordinated response from multiple entities at the federal, state, and local levels; whether such a response would materialize is uncertain and partially outside DHS’s control.”
GAO warns that “because of the limitations we have identified, decision makers are not assured of having sufficient information to ensure future investments are actually addressing a capability gap not met by the current system.” The audit agency recommends that DHS not pursue upgrades to Gen-2 until technical performance requirements are established and the system is assessed against these requirements, and that it produce “a full accounting of statistical and other uncertainties and limitations in what is known about the system’s capability to meet its operational objectives.”
Rep. Michael McCaul (R-Texas), chairman of the House Homeland Security Committee, says that GAO’s findings “bring into focus shortcomings in the BioWatch program at a time when concerns about the threat of a bioterrorism event are elevated.” McCaul’s counterpart in the Senate, Ron Johnson (R-Wisc.), chairman of the Homeland Security and Governmental Affairs Committee, state that “I will work with the department, and my colleagues to assess what significant changes are needed in BioWatch going forward to ensure that we are most efficiently utilizing our limited biodefense resources.”
GAO says DHS agrees on the need to establish technical performance requirements for improvements to Gen-2 but the report says the department first needs to establish requirements for the current system.
“Without establishing such performance requirements, the agency does not know what the existing test results mean for the system’s ability to detect attacks, and thus cannot establish the benefits of any future improvements,” GAO says.