When global alarm bells began ringing three years ago about the spread of a new coronavirus in China, authorities in Washington turned to intelligence agencies to learn more about the threat the virus posed to the United States.
But the most useful early warnings did not come from spies or wiretaps, according to a recent congressional review of classified reports from December 2019 and January 2020. Officials instead relied on public reports, diplomatic cables and analysis by medical experts. : all examples of so-called open source intelligence (OSINT).
Forecasting the next pandemic or the next government to fall will require better use of open source material, according to that report.
“There is little indication that the exquisite collection capabilities of the Intelligence Community were generating information that was valuable to lawmakers,” denounced the authors of the report, made by Democrats in the House Intelligence Committee.
That echoes what many current and former intelligence officials are increasingly warning: America’s $90 billion spy apparatus has been left behind by not embracing open-source intelligence-gathering, unlike its adversaries, including China.
This does not make traditional intelligence any less important. Spy agencies have unique powers to penetrate global communications and recruit agents. They scored a high-profile success when the administration of President Joe Biden released intelligence findings that turned out to be correct: that Russian President Vladimir Putin intended to invade Ukraine.
However, authorities and experts are concerned that the United States has not invested enough resources to analyze publicly available data or take advantage of advanced technologies that can yield crucial information.
Commercial satellite imagery, social media and other online data have given private companies and independent analysts new powers to reveal official secrets. China is known to have stolen or gained control of large amounts of personal data on US citizens. In Washington, there are growing concerns about Beijing’s influence on widely used apps like TikTok.
“Open sources are really an indicator of whether the intelligence community can protect the country,” says Kristin Wood, a former senior CIA official who is now CEO of the Grist Mill Exchange, a trading data platform. “Collectively, we as a nation are not preparing a defense for the munitions that our adversaries are already stockpiling.”
Intelligence agencies face several obstacles to using open source intelligence. Some are technological. Officers working on classified networks often cannot easily access the unclassified internet or open data sources, for example. There are also concerns about civil liberties and the protection of First Amendment rights to the US Constitution, which guarantees the rights of free speech and action.
But some experts also question whether agencies are constrained by the reflexive belief that top-secret information is more valuable.
Jim Himes, a Democratic representative from Connecticut and a longtime member of the Intelligence Committee, said he believed there needed to be “some culture change within places like the CIA, where people do what they do for the thrill of stealing secrets.” crucial instead of checking social media pages.”
During a 2017 test conducted by the National Geospatial-Intelligence Agency, a human team competed against a computer programmed with algorithms to identify Chinese surface-to-air missile sites using commercial imagery.
Both humans and the computer identified 90% of the sites, wrote Amy Zegart, a Stanford University professor, in the book “Spies, Lies, and Algorithms,” but it took the computer only 42 minutes and the human team 80 times more.
Reports created through the use of commercial satellites, online publications, and other open sources—such as daily analyzes of Russian and Ukrainian military tactics published by the Institute for the Study of War—are read widely by legislators and intelligence officials.
“There is a lot of open source capacity that the US intelligence community can have at its disposal,” says Frederick Kagan, a senior fellow at the American Enterprise Institute, an organization that oversees the creation of such reports. “What you have to do is figure out how to take advantage of that ecosystem instead of trying to buy it.”
Most of the 18 spy agencies in the United States have open source programs, from the CIA’s Open Source Enterprise to a 10-person program in the intelligence arm of the Department of Homeland Security. However, senior officials acknowledge that there is no consistency in how these programs analyze open source information or how they use and share it.
“We are not paying enough attention to each other, and so we are not sharing the lessons that different parts (of the intelligence community) are learning, and we are not scaling the solutions,” admitted Avril Haines, director of National Intelligence at United States during an industry event last year sponsored by the Potomac Officers Club group. “And we’re not taking advantage of some of the outside expertise, information and work that we could take advantage of.”
The CIA-based Open Source Initiative is the successor to the Foreign Broadcast Information Service, where for generations employees monitored radio broadcasts and translated them for analysts.
Much of that work has been transformed in the last decade. Where previously people had to travel long distances to collect tapes of radio transmissions in remote locations or areas where Americans were not welcome, sensors now transmit more signals automatically. And machine translation has largely taken the place of people having to listen to the tapes and transcribe them.
Still, officials acknowledge that they have to do more.
Haines has begun multiple reviews of open sources since becoming director of National Intelligence and is expected to finalize the recommendations this year. Some people involved in those reviews have suggested that the Open Source Initiative no longer be designated as a lead in OSINT efforts among spy agencies, according to people familiar with the reviews and who spoke on condition of anonymity to disclose the internal deliberations. of the government.
Three people close to the Open Source Initiative say the center had cut its budget for several years prior to last year. They argue that it is a sign that open source work has not always been consistently prioritized.
The CIA recently appointed new leadership for the Open Source Initiative and in 2021 created a dedicated “control center” for technology.
“We recognize that the importance of open sources is growing as the enormous volume of openly available data increases,” the agency said in a statement. “The CIA is working not only to keep up with this trend, but to stay ahead of it and ahead of our adversaries who are also using information from open sources.”
There is no consensus on whether the United States should create a new agency or center dedicated to open sources. Supporters say such a new organization could focus on adopting advanced technologies and creating more useful products, while opponents question whether it would be unnecessary and take resources away from other agencies.
Carmen Medina, a retired CIA deputy director of intelligence, is now studying how spy agencies can incorporate outside ideas and encourage their employees to be more creative and intuitive.
She suggests a pilot program in which a cell of open source analysts compete over several years against the regular results generated by people with top secret clearances.
Medina and others who have served in senior positions and have reported to White House officials believe that, in most cases, an open source group would be competitive and could even produce better analysis with information that is widely available.
“You can’t make sense of today’s world just by packaging small amounts of information,” he says. “I’ve come to believe that most of the time, the open source way of thinking about it is correct.”