From SourceWatch
Jump to navigation Jump to search

ADVISE (Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement) is a "research and development program within the Department of Homeland Security (DHS), part of its three-year-old 'Threat and Vulnerability, Testing and Assessment' portfolio. The TVTA received nearly $50 million in federal funding this year," Mark Clayton reported in the February 9, 2006, Christian Science Monitor.

ADVISE is "at the core" of a "massive computer system" "being developed by the US government ... that can collect huge amounts of data and, by linking far-flung information from blogs and e-mail to government records and intelligence reports, search for patterns of terrorist activity," Clayton wrote.

"The system - parts of which are operational, parts of which are still under development - is already credited with helping to foil some plots. It is the federal government's latest attempt to use broad data-collection and powerful analysis in the fight against terrorism. But," Clayton wrote, "by delving deeply into the digital minutiae of American life, the program is also raising concerns that the government is intruding too deeply into citizens' privacy."

Lee Tien, a "staff attorney with the Electronic Frontier Foundation, said that programs like ADVISE "are about connecting" the dots of the "traces" we leave behind everywhere, "analyzing and aggregating them - in a way that we haven't thought about, ... as we live our lives and make little choices, like buying groceries, buying on Amazon, Googling," Clayton said.

"A major part of ADVISE involves data-mining - or 'dataveillance,' as some call it. It means sifting through data to look for patterns. If a supermarket finds that customers who buy cider also tend to buy fresh-baked bread, it might group the two together. To prevent fraud, credit-card issuers use data-mining to look for patterns of suspicious activity.

"What sets ADVISE apart is its scope," Clayton reported. "It would collect a vast array of corporate and public online information - from financial records to CNN news stories - and cross-reference it against US intelligence and law-enforcement records. The system would then store it as 'entities' - linked data about people, places, things, organizations, and events, according to a report summarizing a 2004 DHS conference in Alexandria, Va. The storage requirements alone are huge - enough to retain information about 1 quadrillion entities, the report estimated. If each entity were a penny, they would collectively form a cube a half-mile high - roughly double the height of the Empire State Building.

"But ADVISE and related DHS technologies aim to do much more, according to Joseph Kielman, manager of the TVTA portfolio. The key is not merely to identify terrorists, or sift for key words, but to identify critical patterns in data that illumine their motives and intentions, he wrote in a presentation at a November conference in Richland, Wash.," Clayton wrote.

How ADVISE works

According to the "Data Sciences Technology for Homeland Security Information Management and Knowledge Discovery" Report of the DHS Workshop on Data Sciences conducted September 22-23, 2004, which was jointly released in January 2005 by Sandia National Laboratories and Lawrence Livermore National Laboratory:

  • ADVISE is "a system that is under 'spiral' development (meaning that it is being deployed simultaneously with development) and will provide a common platform that supports scalable knowledge management across multiple missions."
  • The system "includes tools for ingesting and canonicalizing massive quantities of information from many different sources. ... Some of the data comes from other databases ... Other data comes from free-form text document sources that must be processed to discover the entities and their relationships. Automatic tools for event extraction are used for some reports but are not yet very good."
  • "At ADVISE’s core, semantic graphs are used to organize the data entities and their relationships. ... A semantic graph organizes relational data by using nodes to represent entities and edges to connect related entities. Hidden relationships in the data are uncovered by examining the structure and properties of the semantic graph. Privacy and support policies are enforced by a security infrastructure. Several interfaces for browsing, querying, and viewing the results of queries are under development, including IN-SPIRE and Starlight, from the DHS National Visualization and Analytics Center (NVAC). The key to fusing disparate data from many sources in ADVISE is the exploitation of 'precomputed' relationship information by storing the data in a semantic graph. All nodes are related by the links between them on the graph."
  • For example, "a simple semantic graph" links "people (black nodes), workplaces (red nodes), and towns (blue nodes). The different link (or edge) types indicate different relationship types. For example, the fact that Person 13 and Person 15 have a green link between them indicates that they are friends with one another, while the orange link from Workplace 19 to Town 22 indicates that Workplace 19 is located in Town 22. In this example, the links are all bidirectional, but directed links can also be used."
  • "Confidences (or uncertainties) are attributes of both the nodes and edges. Studying such graphs can help in understanding the relationships between entities (e.g., what’s the shortest path between Persons 16 and 26?) and in making intelligent hypotheses (e.g., Persons 15 and 14 are linked by a common workplace and a common friend, so we may hypothesize that there is a good chance that they should also be connected by a 'Friends with' link)."

Resources and articles

Related SourceWatch articles

External articles







External resources

U.S. Government Documents

  • FY 2005 Plan: "Create a knowledge management architecture, known as ADVISE (Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement) to integrate the various information analysis and synthesis, visualization, and knowledge discovery component capabilities. ADVISE will also create and incorporate a comprehensive encyclopedia of CBRNE threat and effects data." and "Pilot ADVISE systems for the BTS Directorate will be installed. The ADVISE system comprises computer hardware, networking hardware, and a suite of analytical and visualization tools. A pilot system was installed at IAIP in FY 2004, that is, it was connected to their internal networks and made available for use by all IA analysts. The same process will be used for BTS. Replace the initial TVIS system at the Biodefense Knowledge Center with the enhanced ADVISE capability."
  • FY 2006 Plan: "A National Homeland Security Support System (NH3S) will be created using the ADVISE architecture and providing quantitative risk analysis and decision support capabilities."