M. Shahriar Hossain
ABSTRACT
Intelligence analysts grapple with many challenges, chief among them is the need for software support in storytelling, i.e., automatically 'connecting the dots' between disparate entities (e.g., people, organizations) in an effort to form hypotheses and suggest non-obvious relationships. We present a system to automatically construct stories in entity networks that can help form directed chains of relationships, with support for co-referencing, evidence marshaling, and imposing syntactic constraints on the story generation process. A novel optimization technique based on concept lattice mining enables us to rapidly construct stories on massive datasets. Using several public domain datasets, we illustrate how our approach overcomes many limitations of current systems and enables the analyst to efficiently narrow down to hypotheses of interest and reason about alternative explanations.
Supplemental Material
- R. J. Heuer, phPsychology of Intelligence Analysis.\hskip 1em plus 0.5em minus 0.4em\relax Center for the Study of Intelligence, CIA, 1999. {Online}. Available: https://www.cia.gov/library/Google Scholar
- R. Clark, phIntelligence Analysis: a Target-centric Approach.CQ Press, 2003.Google Scholar
- i2group, "The Analyst's Notebook," Last accessed: May 26, 2011, http://www.i2group.com/us.Google Scholar
- FMS Advanced Systems Group, FMS Inc., "Sentinel Visualizer," Last accessed: May 26, 2011, http://www.fmsasg.com/.Google Scholar
- E. Bier, E. Ishak, and E. Chi, "Entity Workspace: An Evidence File That Aids Memory, Inference, and Reading," in ISI'06, 2006, pp. 466--472. Google ScholarDigital Library
- PNNL, "Pacific Northwest National Laboratory, INSPIRE visual document analysis," Last accessed: May 26, 2011, http://in-spire.pnl.gov.Google Scholar
- H. Khurana, J. Basney, M. Bakht, M. Freemon, V. Welch, and R. Butler, "Palantir: a Framework for Collaborative Incident Response and Investigation," in IDtrust, 2009. Google ScholarDigital Library
- Information Interfaces Research Lab., Georgia Tech, "Jigsaw: Visual Analytics for Exploring and Understanding Document Collections," Last accessed: February 9, 2012, http://www.cc.gatech.edu/gvu/ii/jigsaw/.Google Scholar
- Y. Kang and J. Stasko, "Characterizing the Intelligence Analysis Process: Informing Visual Analytics Design through a Longitudinal Field Study," in VAST, 2011.Google Scholar
- D. Kumar, N. Ramakrishnan, R. Helm, and M. Potts, "Algorithms for Storytelling," TKDE, vol. 20, no. 6, pp. 736--751, 2008. Google ScholarDigital Library
- H. Kang, C. Plaisant, B. Lee, and B. B. Bederson, "NetLens: Iterative Exploration of Content-actor Network Data," Info. Vis., vol. 6, no. 1, pp. 18--31, 2007. Google ScholarDigital Library
- R. Alonso and H. Li, "Model-guided Information Discovery for Intelligence Analysis," in CIKM'05, 2005, pp. 269--270. Google ScholarDigital Library
- A. Koltuksuz and S. Tekir, "Intelligence Analysis Modeling," in ICHIT'06, 2006. Google ScholarDigital Library
- K. Chopra and C. Haimson, "Information fusion for intelligence analysis," in HICSS'05, 2005, p. 111a. Google ScholarDigital Library
- E. Lindahl, S. O'Hara, and Q. Zhu, "A Multi-agent System of Evidential Reasoning for Intelligence Analyses," in AAMAS'07, 2007, pp. 279:1--279:6. Google ScholarDigital Library
- J. Gersh, B. Lewis, J. Montemayor, C. Piatko, and R. Turner, "Supporting Insight-based Information Exploration in Intelligence Analysis," Commun. ACM, vol. 49, April 2006. Google ScholarDigital Library
- T. Coffman, S. Greenblatt, and S. Marcus, "Graph-based Technologies for Intelligence Analysis," Commun. ACM, vol. 47, pp. 45--47, March 2004. Google ScholarDigital Library
- P. Crossno, B. Wylie, A. Wilson, J. Greenfield, E. Stanton, T. Shead, L. Ice, K. Moreland, J. Baumes, and B. Geveci, "Intelligence Analysis Using Titan," in IEEE Symposium on Visual Analytics Science and Technology, 2007, pp. 241--242. Google ScholarDigital Library
- G. Chin, O. A. Kuchar, P. D. Whitney, M. Powers, and K. E. Johnson, "Graph-based comparisons of scenarios in intelligence analysis," in SMC'04, vol. 4, 2004.Google Scholar
- E. Bier, S. Card, and J. Bodnar, "Principles and Tools for Collaborative Entity-Based Intelligence Analysis," TVCG, vol. 16, no. 2, pp. 178--191, 2010. Google ScholarDigital Library
- M. Chau, J. J. Xu, and H. Chen, "Extracting Meaningful Entities from Police Narrative Reports," in Annual National Conference on Digital Government Research, 2002, pp. 1--5. Google ScholarDigital Library
- P. Pirolli and S. Card, "The Sensemaking Process and Leverage Points for Analyst Technology as Identified through Cognitive Task Analysis," in ICIA'05, 2005.Google Scholar
- L. Fang, A. D. Sarma, C. Yu, and P. Bohannon, "Rex: explaining relationships between entity pairs," Proc. VLDB Endow., vol. 5, no. 3, pp. 241--252, 2011. Google ScholarDigital Library
- K. Heath, N. Gelfand, M. Ovsjanikov, M. Aanjaneya, and L. Guibas, "Image Webs: Computing and Exploiting Connectivity in Image Collections," in CVPR, 2010.Google Scholar
- J.-P. Brassard and J. Gecsei, "Path Building in Cellular Partitioning Networks," in ISCA'80, 1980, pp. 44--50. Google ScholarDigital Library
- C. Faloutsos, K. S. McCurley, and A. Tomkins, "Fast Discovery of Connection Subgraphs," in KDD'04, 2004. Google ScholarDigital Library
- M. S. Hossain, J. Gresock, Y. Edmonds, R. Helm, M. Potts, and N. Ramakrishnan, "Connecting the Dots between PubMed Abstracts," PLoS ONE, vol. 7, no. 1, p. e29509, 2012.Google ScholarCross Ref
- D. Shahaf and C. Guestrin, "Connecting the Dots between News Articles," in KDD'10, 2010, pp. 623--632. Google ScholarDigital Library
- D. R. Swanson, "Complementary Structures in Disjoint Science Literatures," in SIGIR'91, 1991, pp. 280--289. Google ScholarDigital Library
- M. S. Hossain, C. Andrews, N. Ramakrishnan, and C. North, "Helping Intelligence Analysts Make Connections," in AAAI'11 Workshop on Scalable Integration of Analytics and Visualization, 2011.Google Scholar
- M. J. Zaki and N. Ramakrishnan, "Reasoning About Sets Using Redescription Mining," in KDD'05, 2005. Google ScholarDigital Library
- A. Beygelzimer, S. Kakade, and J. Langford, "Cover Trees for Nearest Neighbor," in ICML'06, 2006, pp. 97--104. Google ScholarDigital Library
- A. Leach and V. Gillet, phIntroduction to Chemoinformatics. Springer, 2007. Google ScholarDigital Library
- Alias-i., "LingPipe 4.1.0," Last accessed: Jan 31, 2012, http://alias-i.com/lingpipe, 2008.Google Scholar
- Apache Software Foundation, "OpenNLP," Last accessed: Jan 31, 2012, http://incubator.apache.org/opennlp.Google Scholar
- Stanford Natural Language Processing Group, "Stanford NER: http://nlp.stanford.edu/software/CRF-NER.shtml," Last accessed: Jan 31, 2012.Google Scholar
- B. Baldwin, "CogNIAC: High Precision Coreference with Limited Knowledge and Linguistic Resources," in ACL/EACL'97, Spain, July 1997, pp. 38--45. Google ScholarDigital Library
- AlchemyAPI., "AlchemyAPI: http://www.alchemyapi.com/api/categ/."Google Scholar
- F. J. Hughes, "Discovery, Proof, Choice: The Art and Science of the Process of Intelligence Analysis, Case Study 6, 'All Fall Down',Unpublished report," 2005.Google Scholar
Index Terms
- Storytelling in entity networks to support intelligence analysts
Recommendations
Storytelling with Signal Injection: Focusing Stories with Domain Knowledge
Machine Learning and Data Mining in Pattern RecognitionAbstractGiven a beginning and ending document, automated storytelling attempts to fill in intermediary documents to form a coherent story. This is a common problem for analysts; they often have two snippets of information and want to find the other pieces ...
Exploring Storytelling for Relationship Building in Offshore Outsourced Projects: An Action Research Investigation
HICSS '15: Proceedings of the 2015 48th Hawaii International Conference on System SciencesOver recent years, more and more companies have come to recognise the utility of storytelling in the workplace. Stories are thought to be an effective means of sharing information and can fulfil a range of knowledge management functions. Furthermore, ...
Conciliating coherence and high responsiveness in interactive storytelling
DIMEA '08: Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and ArtsInteractive storytelling is a new form of digital entertainment that brings together techniques and tools for the creation, visualization and control of interactive stories through electronic means. One of the main challenges of interactive storytelling ...
Comments