It became a mantra after the Sept 11, 2001, terrorist attacks: U.S. ports are vulnerable.
Millions of shipping containers streaming into the country were not being scanned for radiation, the telltale fingerprint of nuclear material. Rogue actors, it was said, could smuggle in an atom bomb and unleash hell.
The Bureau of Customs and Border Protection responded by rolling out detectors that would scan every container for radiation. And while the portals were not perfect, they were a tested, understood technology that would operate reliably. By 2004, many portals had been erected and improvements continued to be made.
Then, in 2005, everything changed. Created by presidential order, an office in the newly created Department of Homeland Security -- called the Domestic Nuclear Detection Office (DNDO) -- began a single-minded pursuit to buy two untested technologies to scan and X-ray shipping containers for radiological devices.
The office's efforts to avoid standard research-and-development procedures sparked congressional inquiries and scientific criticism. And for three years, that quagmire distracted Congress, preventing funding that government auditors say could have quickly improved existing nuclear safety nets.
As a result, little money has gone to research aimed at optimizing the recently installed fleet of radiation portals, auditors say. And many X-ray systems used to complement those scanners -- first put in place to prevent drug smuggling -- are nearing the end of their operational lives. A little-noticed provision in the economic stimulus law sent $100 million to the program.
While most experts agree that DNDO's new generation of portals has scientific merit, auditors have repeatedly questioned whether efforts to upgrade an already functioning system would divert scarce resources away from addressing greater threats.
"This whole situation from 2006 to 2009 is not an example of government at its best," said Gene Aloise, who has led multiple audits of DNDO for the Government Accountability Office. GAO published six reports on DNDO in that period.
"The taxpayer, in our view, has not been served very well," Aloise said. Three years of upgrades that could have been made to existing nuclear detectors, he added, have been lost because of DNDO's "bad tests."
The White House has proposed slashing more than $100 million from DNDO for next year, and the office is now being evaluated by the department's new secretary, Janet Napolitano. The office has been stripped of future acquisition powers and will be purely devoted to research, according to the Obama administration's budget proposal.
Meanwhile, field testing resumed recently on the office's advanced portals, which have returned to the more modest proposals first advanced by the Department of Energy's Pacific Northwest National Laboratory. The lab has pioneered the science of radiation detection and helped roll out the detectors now in use at the nation's ports.
There is little certainty why the DNDO went off the rails in its procurement efforts. There are theories: the lure of a silver-bullet technological solution, fears that the current monitors were deeply flawed, a need for the office to prove its place in the bureaucracy, and the eagerness of defense contractors to expand into a well-funded new field.
But what becomes clear -- through interviews with government officials, investigators and scientists and in hundreds of pages of congressional testimony, letters, reports and procurement requests -- is that for three years, DNDO's highly touted programs failed to meet expectations and, in the words of one senator, put the United States at risk.
After Sept. 11, 2001, the government became acutely aware that shipping containers arriving at U.S. ports were not being screened for nuclear materials. The National Research Council put a fine point on this security flaw when it called for a national network of portal monitors to flag radiation.
The screening technology had been well tested. DOE had for decades refined a radiation-detection technology known as PVT -- short for "polyvinyl toulene" -- that uses rugged plastics to flag radiation. Designed to stop thieves smuggling radioactive material away from nuclear sites, PVT portals could be adopted for smuggling in the opposite direction. In fact, they already had been. By 2001, DOE had installed 70 such monitors in Russia.
In 2002, the Customs Service worked with DOE's Pacific Northwest lab to begin installing PVT monitors. Since then, more than 1,200 portal monitors have been deployed, dual sentinels that bracket a steady flow of commercial traffic. Currently, the United States scans some 97 percent of the cargo entering the country with PVT monitors.
The monitors are not perfect, and one drawback comes down to physics. The heavy metals needed to make nuclear weapons -- in particular, enriched uranium and plutonium -- emit streams of energized light particles called gamma rays. The PVT detectors performed well in detecting these gamma rays, but they also sounded alarms when some normal cargo passed through. Kitty litter and ceramics, for example, are radioactive, so sorting threats from the mundane became the province of secondary screening, which sees agents pore over containers with hand-held scanners.
Scientists at DOE's Pacific lab thought it could be helpful in secondary screening to use spectroscopic portals, a detector technology that uses crystals, rather than plastic, to detect radiation. These detectors can read radiation fingerprints, potentially allowing automated differentiation of radioactive material. They were seen as a supplement to the PVT monitors.
In 2003, the Customs Service -- which had become the Bureau of Customs and Border Protection under the newly created Department of Homeland Security -- tasked the Pacific lab to conceive how a spectroscopic portal might look. At the time, few suspected that these portals would come to dominate the debate on nuclear security for more than half a decade.
'We could not afford to wait'
While the DOE lab worked, the Department of Homeland Security, which was flush with cash, began offering contracts for the development of prototype detectors through its Advanced Research Projects Agency (HSARPA). On Jan. 30, 2004, HSARPA announced a tender to develop prototypes, including spectroscopic portals. Any technology had to show cost savings over existing PVT monitors, it said.
Nearly six months after the HSARPA tender, the Pacific lab released its plan for crystal-based portals. The lab recommended purchasing a limited number for comparison against PVT portals. The new portals would be unproven, it warned, and the crystals used were "notorious for being prone to internal cracking ... as a result of mechanical or thermal shock."
Despite the lab's warnings, things changed at HSARPA after the 2004 presidential election.
It was in December 2004 that the Bush administration appointed Vayl Oxford, a former director for counterproliferation at the National Security Council, to lead HSARPA. That same month, the agency moved to start buying "advanced spectroscopic portals," or ASPs, quickly and at large volumes, asking contractors for their proposals.
While notionally for prototype development, HSARPA's request for proposals made clear -- in bold type -- that it intended to make a "large scale procurement of Spectroscopic Portals" by spring of 2005. At the time, testing of the first ASP prototypes had not begun.
A subsequent report by DOE's Pacific Northwest lab warned later: "It has not been demonstrated that all these [ASP] technologies will be suitable" for portal screening.
In congressional testimony, Oxford said later that the DNDO contracts were designed to merge research and production.
"We intentionally designed the contract so that once ASP is certified ... we could enter into production without major delays," Oxford said. "We knew that we could not afford to wait up to a full year to get production contracts in place."
Repeated attempts to reach Oxford for comment were unsuccessful.
DNDO was created on April 15, 2005. The Bush administration tasked the office with developing and acquiring a detection system for nuclear material and gave it sole federal authority for domestic purchases of portal detectors.
Named to lead the office was HSARPA's administrator, Oxford.
Many applauded the coordination that DNDO could bring to federal anti-nuclear policies. "The creation of the DNDO was a useful step in that direction," said Michael Levi, author of "On Nuclear Terrorism" and a fellow at the Council on Foreign Relations. But he added, "it's had a lot of growing pains."
Soon after DNDO's creation, articles appeared in the press decrying the PVT monitors, implying that they were part of a flawed process that resulted from panic created by Sept. 11. Few articles mentioned that the portals had previously been employed by DOE.
"[ASP] became a broader program, became hyped as a solution to the world's problems," said a scientist with knowledge of the program who requested anonymity because he is not allowed to speak with the media. "It became overblown."
The office's streamlining efforts ran counter to the acquisition system for other federal agencies. Called "technology readiness levels" and developed by NASA, the system attempts to account for likely technological failures. But DNDO, in effect, wanted to skip several tiers.
Congress soon began to worry about DNDO. On June 16, 2005, the Senate Appropriations Committee, chaired by Sen. Judd Gregg (R-N.H.), submitted a scathing review, calling DNDO an example "of action being taken before thoughtful planning despite the seriousness of problems being addressed. Hasty solutions are fostering an apparent false sense of security."
DNDO nonetheless moved ahead with ASPs and began developing a second high-tech project, called CAARS, which sought to automate the X-ray screening of cargo. It is good practice, Levi said, to pair radiation portals with X-rays, which can reveal metals like lead that may be blocking gamma rays. Such a combination is being used at some ports.
While much attention was paid to CAARS, Customs' existing network of X-ray detectors, originally used to detect drugs or illegal aliens, received no additional funding between 1999 and 2007, said the program's director, Patrick Simmons.
There are many holes in coverage, as well, Simmons said, with Customs needing an additional 50 detectors. The Obama administration pledged $100 million in the stimulus to help patch those holes.
Since the existing X-ray systems were the province of Customs, DNDO focused its attention on its advanced system, which it sold as a $1.3 billion program to detect dense metals. Three companies received grants to begin development and promised 20 automated detectors by 2008.
Unlike the ASP program, CAARS never got off the ground. Customs, which would run the detectors, had obvious issues with an X-ray system that addressed only nuclear threats. The agency had "little interest" in CAARS, Simmons said.
Last year, Sen. Joe Lieberman (I-Conn.) noted that CAARS had "essentially been abandoned following technical difficulties that would have made the system too complex to deploy in domestic ports." Lieberman questioned whether such problems had put the United States at risk.
SAFE Port Act
In the fall of 2005, DNDO tested 10 prototype ASPs at the Nevada Test Site, conducting some 7,000 runs, with "somewhat mixed results," GAO said. As the amount of radioactive material decreased in size, the detection abilities of ASP systems and PVT "converged," auditors found.
Despite these inconclusive tests, in July 2006, DNDO awarded ASP contracts to three companies: Raytheon Co., which had no prior experience making detectors; Thermo Electron Co.; and Canberra Industries. Full implementation of the ASPs would cost $1.2 billion, the DNDO estimated.
"I think there were grand expectations for DNDO" back then, said Chuck Galloway, DNDO's acting director, who joined the office in 2007. "I think we were being asked to [follow] some very challenging and crosscutting paths."
Pressure on the agency escalated after the Dubai Ports World controversy in early 2006. At issue was whether the sale of port management businesses in six U.S. seaports to a company owned by the government of Dubai, United Arab Emirates, would compromise port security.
In response, Congress passed the SAFE Port Act. Signed by President George W. Bush in October 2006, the law required DNDO and Customs to scan all containers entering the 22 busiest U.S. ports by Dec. 31, 2007, and specified that Homeland Security deploy "next generation radiation detection technology."
Meanwhile, the Senate lost confidence in DNDO, largely because of a GAO audit that found that the office "did not use the results of its own performance tests" to analyze the ASP system. Rather, the report said, DNDO relied on "assumptions" that ASP would work correctly 95 percent of the time, while providing no information on how this goal would be achieved.
Over the next two years, this cycle -- Congress withholding ASP funds while GAO probed DNDO's operations -- repeated. GAO found DNDO's second round of Nevada tests to be "methodologically flawed." The tests used the same nuclear material to synchronize and test the portals.
"It is highly unlikely that such favorable circumstances would present themselves under real-world conditions," GAO's Aloise dryly noted in testimony to Congress.
In the wake of the report, DNDO no longer emphasized the increased detection that could come with ASPs but rather focused on keeping containers moving. As Oxford told a House panel, "These nuisance alarms can take several minutes to resolve, causing some impact on the flow of commerce."
Today, DNDO's Galloway acknowledges that the "flow of commerce" is no reason to employ ASPs. There were no data that he is aware of, he said, to support DNDO's previous claims.
"Having visited those sites, I think very quickly you can come to the perspective that our detectors are not slowing down the flow of commerce," he said.
'Marginal' improvement at twice the price
Last year, DNDO finally established that the ASPs had made only a "marginal" improvement in detecting unshielded or lightly shielded nuclear materials. This at a cost of $800,000 each -- nearly double the price of PVT portals. According to GAO estimates, DNDO's program would end up costing $2 billion.
Nonetheless, it is undeniable the ASP is a more accurate technology, said Robert Dynes, a physicist and a former president of the University of California who led a National Academy of Sciences committee that examined the ASP program.
"These will work now, and they're better than what's out there," Dynes said. "But we can't really say how much better, because the science hasn't been done."
DNDO was racing to achieve certification for the ASP program and put pressure on the academy to submit its report by October 2008, Dynes said. "They were in a big hurry," he recalled.
But DNDO never met its deadlines and never received ASP certification. The office also terminated the contract of one contractor, Canberra, to lower the overall cost of implementing ASPs -- the $1.2 billion figure was always the maximum the program could cost, said Galloway, DNDO's acting director.
In its first budget proposal for fiscal 2010, the Obama administration cut DNDO's acquisition funds, partly because the office still has more than $126 million available for purchasing ASPs. Meanwhile, little money has been spent to upgrade the PVT monitors, particularly by refining algorithms used on PVT monitors to reduce the number of false alarms, a process called "energy windowing." Opinions differ on how much these can be improved, Galloway said.
"The issue is, can you squeeze more performance out of energy windowing?" he said, adding, "A small but very vocal group wants to squeeze the last bit of performance from energy windowing."
Since the current generation of radiation detectors is expected to be in use for an extended time, Congress has given DNDO until November to devise a plan that will upgrade the software in PVT portals and handheld scanners.
Meanwhile, Galloway is stressing that it is time to focus on other, unsecured routes that nuclear material may take into the country.
Such a holistic view makes far more sense, according to Tom Cochran, a nuclear physicist at the Natural Resources Defense Council who testified in congressional DNDO hearings.
"You have over 10 million illegal people in this country who don't come through the portals," Cochran said. "So why do you assume they'll come through the portals?"