Spy Tools Come to the Cloud
Cloud computing is increasingly being used for intelligence-gathering and "threat analysis" as government vendors like Amazon Web Services join forces with analytics outfits that are meshing cognitive computing and other forms of data analysis with cloud computing horsepower.
Among the initial platforms for much of this intelligence activity is the AWS "GovCloud," which went live last year. It includes special provisions to create a "government-community cloud" that meets additional security and operational standards such as restricting access to "U.S. persons."
Among Amazon's analytics partners is Digital Reasoning, a trusted cognitive computing startup based in Arlington, Va. It specializes in open source intelligence gathering. AWS and Digital Reasoning have been promoting real-time threat analysis and other open-source intelligence gathering tools on GovCloud, including a recent webinar. They are also targeting government contractors who are handling increasing amounts of sensitive open-source data.
While cloud vendors like AWS handle government security requirements, analytics service providers like Digital Reasoning are developing application and service controls. The analytics company's "Synthesys" cloud is currently available on all U.S. AWS cloud regions, the partners said.
The cloud-based cognitive computing tool offers analytics capabilities such as threat intelligence used to anticipate, among other things, network attacks and "knowledge visualization" products.
The cloud-based intelligence platform targets "big data and analytics problems, in particular [for] open-source computing challenges that need to be done at scale but that also need deep analytics to be able to provide the threat intelligence relevant to law enforcement and national security," Max Peterson, general manager of the AWS Global Public Sector unit, said during a recent webcast.
The Digital Reasoning cognitive computing tool is designed to generate "knowledge graphs of connected objects" gleaned from structured and unstructured data. These "nodes" (profiles of persons or things of interest) and "edges" (the relationships between them) are graphed, "and then being able to take this and put it into time and space," explained Bill DiPietro, vice president of product management at Digital Reasoning.
The partners noted that the elastic computing capability delivered by the AWS GovCloud and similar secure clouds is allowing customers to bring together much larger datasets. With the emergence of the Hadoop ecosystems and enterprise data hubs based on both private and public clouds, "you can ask bigger questions of your data," DiPietro argued.
The Holy Grail in all over this emerging open-source spy craft is discovering what data analysts refer to as the "unknown unknowns." Companies like Digital Reasoning are leveraging cloud platforms like the AWS GovCloud to unify siloed approaches and provide new search and discovery tools as a way to at least ask the right questions when "harvesting" and plowing through huge datasets.
The added capacity of "clusters on demand" allows for the crunching of larger datasets such as open-source intelligence.
"You have to be able to analyze the massive streams of data coming in and be able to identify…people who might be a on watch list" and generate alerts, DiPietro said.
From there, data analysts can begin applying predictive tools to make more inferences based on the connections made in large datasets stored in the cloud. Moreover, these analytics tools are able to leverage the horsepower of the AWS Elastic Computer Cloud.
DiPietro, a former CIA staff officer, said the biggest challenges were "to not only get capacity, but getting capacity on demand." The old problem of quickly outgrowing capacity has been solved via what DiPietro called a "cluster on demand."
Digital Reasoning uses this elastic capacity to run cognitive algorithms to "organize and unify data," DiPietro said, in a process the company refers to as "read, resolve and reason."
