Fusion Methodologies in Crisis Management

Tractor is a system for understanding English messages within the context of hard and soft information fusion for situation assessment. Tractor processes a message through text processors using standard natural language processing techniques, and represents the result in a formal knowledge representation language. The result is a hybrid syntactic-semantic knowledge base that is mostly syntactic. Tractor then adds relevant ontological and geographic information. Finally, it applies hand-crafted syntax-semantics mapping rules to convert the syntactic information into semantic information, although the final result is still a hybrid syntactic-semantic knowledge base. This chapter presents the various stages of Tractor’s natural language understanding process, with particular emphasis on discussions of the representation used and of the syntax-semantics mapping rules.

A commander’s situation awareness is critical to his or her decision making in a crisis but the ability for a commander to form that situation awareness is often overestimated. Automated data fusion offers a means of supporting a commander’s situation awareness, with automated higher level fusion supporting the higher level functions of comprehension and projection. This chapter outlines a software-implemented psychological model that allows a machine to perform comprehension and projection, and interact with a commander.

Designing fusion systems for decision support in complex dynamic situations such as crises requires fusion of a large amount of multimedia and multispectral information coming from geographically distributed sources to produce estimates about objects and gain knowledge of the entire domain of interest. Information to be fused and made sense of includes but is not limited to data obtained from physical sensors, surveillance reports, human intelligence reports, operational information, and information obtained from social medial, opportunistic sensors and traditional open sources (internet, radio, TV, etc.). Successful processing of this information may also demand information sharing and dissemination, and action cooperation of multiple stakeholders. Decision making in such environment calls for designing a fusion-based human–machine system characterizing constant information exchange between all nodes of the processing. The quality of decision making strongly depends on the success of being aware of, and compensating for, insufficient information quality at each step of information exchange. Designing the methods of representing and incorporating information quality into such processing is a relatively new and a rather difficult problem. The chapter discusses major challenges and suggests some approaches to address this problem.

Retrieving items such as similar past events, or vessels with a specific characteristic of interest, is a critical task for crisis management support. The problem of information retrieval from incomplete databases is addressed in this paper. In particular, we assess the impact of the uncertainty representation about missing data for retrieving the corresponding items. After a brief survey on the problem of missing data with an emphasis on the information retrieval application, we propose a novel approach for retrieving records with missing data. The general idea of the proposed data-driven approach is to model the uncertainty pertaining to this missing data. We chose the general model of belief functions as it encompasses as special cases both classical set and probability models. Several uncertainty models are then compared based on (1) an expressiveness criterion (non-specificity or randomness) and (2) objective measures of performance typical to the Information Retrieval domain. The results are illustrated on a real dataset and a simulation controlled missing data mechanism.

Successful management of critical situations created by major natural and man-made activities requires monitoring, recognizing, fusing, and making sense of these activities in order to support decision makers in either preventing a crisis or acting effectively to mitigate its adverse impact. Context plays an important role in crisis management since it provides decision makers with important knowledge about current situations and situation dynamics in relation to their goals, functions, and information needs, to enable them to appropriately adapt their decisions and actions. Efficient context exploitation for crisis management requires a clear understanding of what context is, how to represent it and use it. The chapter provides a brief discussion of the key issues of the problem of context definition, representation, discovery, and utilization in crisis management.

Adaptive systems require technologies to enable high synchronicity between its users and their unfolding situation dynamics, in concert with system response actions. To be effective, a multi-dimensional view of context must be considered and incorporated. This work advances the development of such a system for RECON, an initiative to support intelligence analysts with a novel context-management and case-based recommendation capability. The central concepts involved in the management of explicit and implicit contexts are presented and are developed into a novel multi-agent approach. In particular a new context-sensitive cognitive model and a community of expert service-oriented agents are proposed to facilitate and improve system adaptations to user-specific, situational, and system states. These designs pave the way towards future developments and experiments in improving human–machine interaction with adaptive context-management systems.

In crisis situations, the challenge of understanding the current situation is tightly linked to the ability to process the variety and the amount of information provided by the multiple sources. In particular, social media can provide additional insight on real-time events, providing that the information that they relay is accurately retrieved, evaluated, and fused. In this chapter, we describe various mechanisms and functions necessary for information fusion and understanding, starting from social media exploration and retrieval, then describing the fusion process and the associated management of information uncertainty, concluding with a description of the methodology and experiments we use to tackle the intrinsic big volume of data and processing required for social media information analysis.

Natural and man-made crises pose severe challenges on emergency responders, as they need to gain timely Situation Awareness (SAW) in order to decide upon adequate rescue actions. Computational SAW systems aim at supporting humans in rapidly achieving SAW by means of Information Fusion (IF), thus reduce information overload by fusing data stemming from various sensors to situation-level information. Recently, the increasing popularity of social media on mobile devices has enabled humans to act as crowd sensors, who broadcast their observations on the unfolding crisis situation over social media channels. Consequently, SAW systems for crisis management would benefit from exploiting social media as additional data source. Therefore, the aim of this chapter is to investigate upon how crowd-sensing can be incorporated into SAW systems for crisis management, by elaborating on the following issues: How can the SAW system seek and retrieve additional information from social media that may complement the situational picture obtained with other types of sensors? How can the SAW system adapt this crowd-sensing alongside the monitored situation, to keep pace with the underlying real-world incidents? We attempt at illustrating potential solutions towards these questions, by examining how crowd-sensing can deliver input data for SAW systems, elaborating on the challenges such systems need to overcome in order to identify and extract relevant information from social media, and finally, discussing the architecture of a situation-adaptive SAW system capable of exploiting both conventionally sensed data and unstructured social media content.

Crises and disasters are covered continuously and without interruption by today’s media, especially social media. There is not a single significant occurrence within the flow of events which they do not document. Consequently, the information contained in media—especially social media like Facebook and Twitter—provides an often neglected potential which should not be overlooked. Through fusion of sources, diverse, mixed, and complementary types of information can be tapped into and combined. The difficulty of this process is to view, channel, prepare, and exploit this inhomogeneous and enormous amount of information. Automatic monitoring of traditional as well as social media sources allows to deriving risk factors and risk indicators for crises and disaster events quickly. Intelligence derived from this process allows for earlier and swifter reaction to potential situations of crisis and interrelationships. Current publicly described technical and electronic infrastructure for national and international crisis and disaster management is not able to perform comprehensive analyses of all media channels automatically. The continuous developments in the areas of multimedia and social media demand the creation of adequate methods of processing. Relevant manifestations of events are to be identified automatically from documents from traditional (TV, radio, web) as well as social media and document clusters of the examined multimedia documents are to be presented to situational awareness experts. The focus of the Quelloffene IntegrierteMultimedia Analyse (QuOIMA) project is on the research on and development of algorithms and methods to achieve this goal. Automatic analysis of content in the multimedia and social media domain forms a fundamental innovation. From a technical, social studies, and scientific point of view, the targeted insights and findings of this project, form a fundamental contribution to security research, reaching far beyond the quality of existing systems. The integration of findings regarding situational awareness will provide more realistic risk assessment increasing their possibilities to (re)act. End users extend their expertise and as a consequence the ability of the overall organizations to act.

Situation analysis for activities such as crisis management, military situation awareness, homeland security, or environmental monitoring is both enabled and challenged by access to enormous data sets. The advent of new sensing capabilities, advanced computing and tools available via cloud services, intelligent interconnections to mobile devices, and global interconnectivity with ever-increasing bandwidths provide unprecedented access to data and to computing. In addition, emerging digital natives freely share data and collaboration. Thus, on one hand situation analysts have great opportunities to access unprecedented amounts of information from sensors, human observers and online sources to assist in understanding an evolving situation. On the other hand, this access to huge data sources and computing can create a type of intelligence attention-deficit disorder, in which analysts are overwhelmed by the urgent, but lack the ability to focus on important data. This chapter provides a summary of this dilemma, describes a new analysis paradigm that links data-driven and hypothesis driven approaches, introduces a new prototype analyst workbench, and discusses an educational approach to empower the next generation of analyst.

Abductive inference (best-explanation reasoning) is a useful conceptual framework for analyzing and implementing the inferencing needed to integrate information from human and robotic sources. Inferencing proceeds from reports, to explanations for these reports, given in terms of hypothesized real-world entities and the processes by which the entities lead to the reports. Reports from humans and robotic sources are subject to different kinds of corruption, so they require different treatment as sources of evidence. The best explanation for a certain report might be that it presents a reliable statement that results from a chain of causality from the events reported, to their effects on human or robotic senses, and from there through transduction, processing, and reporting. Confidence in this explanation will be undercut by evidence supporting a rival explanation, such as one involving error or intended deception.

Each year, natural and anthropogenic crises disrupt the lives of millions of people. Local, national, and international crisis response systems struggle to cope with urgent needs during and immediately after a crisis. The challenges multiply as population grows, density of urban areas increases, and coastal areas become more vulnerable to rising sea levels. A typical crisis scenario requires coordinating many diverse players, including local, national and international military, other governmental, and non-governmental organizations. Often, no single entity is in charge of the response, making coordination even more difficult. There is an urgent need for better ways to allocate resources, maintain situation awareness, and reallocate resources as the situation changes. Information fusion is vital to effective resource allocation and situation awareness. Some of the greatest inefficiencies stem from the inability to exchange information between systems designed for different purposes and operating under different ownership. Information integration and fusion are too often entirely manual. Greater automation could support more timely, better coordinated responses in situations where time is of the essence. The greatest need is not for low-level fusion of sensor reports to classify and track individual objects, but for high-level fusion to characterize complex situations and support planning of effective responses. This paper describes challenges of high-level fusion for crisis management, proposes a technical framework for addressing high-level fusion, and discusses how effectively addressing HLF challenges can improve efficiency of crisis response. We illustrate our ideas with a case study involving a humanitarian relief operation in a flooding scenario.

Building and updating a situational picture of the scenario under consideration is the goal of the Situation Assessment (SA) Information Fusion (IF) process. The scenario generally involves multiple entities and actors where possibly only a few under direct control of the decision maker. SA aims at explaining the observed events (mainly) by establishing the entities and actors involved, inferring their goals, understanding the relations existing (whether permanently or temporarily) between them, the surrounding environment, and past and present events. It is therefore apparent how the SA process inherently hinges on understanding and reasoning about relations. SA is a necessary preparatory step to the following phase of Impact Assessment (IA) where the decision maker is interested in estimating the evolution of the situation and the possible outcomes, dangers and threats. SA and IA processes are particularly complex and critical for large-scale scenarios with nearly chaotic dynamics such as those affected by natural or man-made disasters. This chapter will discuss recent developments in information fusion methods for representing and reasoning about relational information and knowledge for event detection in the context of crisis management. In particular, network methods will be analysed as a means for representing and reasoning about relational knowledge with the purpose of detecting complex events or discovering the causes of observed evidence.

A concept for characterizing, predicting and recognizing threat situations is developed. The goal is to establish a systematic approach to automation of some of these functions. Approaches are discussed to address the fundamental problems of (a) sparse and ambiguous indicators of potential or actualized threat activity buried in massive background data; and (b) uncertainty in threat capabilities, intent and opportunities.

The notion of situation enables designers, maintainers, and users to abstract from lower-level entities and properties and to focus on the higher-level patterns that emerge in time. Situation management concerns a number of tasks including situation specification, situation detection (which may involve composite situation pattern recognition), and situation’s life cycle control. This chapter discusses how to approach situation management from a rule-based perspective. We present a rule-based situation management infrastructure to support the development of situation-aware applications and show its applicability to a scenario in the public health domain, concerning situations for detecting influenza epidemics.

Building on evidence from the field of risk perception and communication, two key roles of argumentation in crisis management are highlighted: (1) balancing trust construction and persuasive goals in crisis prevention and preparedness, and (2) ensuring time-efficient cross-examination of choice options in group decision making at a time of crisis. The implications for an information fusion approach to crisis management are discussed, suggesting a rich potential for future research.

Future crisis management systems need resilient and trustworthy infrastructures to quickly develop reliable applications and processes, apply fusion techniques, and ensure end-to-end security, trust, and privacy. Due to the multiplicity and diversity of involved actors, volumes of data, and heterogeneity of shared information; crisis management systems tend to be highly vulnerable and subject to unforeseen incidents. As a result, the dependability of crisis management systems can be at risk. This chapter presents a cloud-based resilient and trustworthy infrastructure (known as rDaaS) to quickly develop secure crisis management systems. The rDaaS integrates the Dynamic Data-Driven Application Systems (DDDAS) paradigm into a service-oriented architecture over cloud technology and provides a set of resilient DDDAS-As-A Service (rDaaS) components to build secure and trusted adaptable crisis processes. One service presented includes the fusion of information from human observers and surveillance systems to assess the credibility (trust) of a crisis alert. The fusion trust service within the rDaaS also ensures resilience and security by obfuscating the execution environment and applying Behavior Software Encryption and Moving Technique Defense over the users, machines, and communication network. A simulation environment for a nuclear plant crisis management case study is illustrated to build resilient and trusted crisis response processes.

The paper defines a consensus distribution with respect to experts’ opinions using a multiple quantile utility model. We show that the Steiner point (Schneider, Isr J Math 2:241–249, 1971) is the representative consensus probability. The new rule for aggregation of experts’ opinions, which can be simply evaluated by the Shapley value, is prudential and coherent.

Typically, in constructing a model for a random variable, one utilizes available samples to construct an empirical distribution function, which can then be used to estimate the probability that the random variable would exceed a prespecified threshold. However, in modeling extreme events, the threshold is often in excess of the largest sampled value observed thus far. In such cases, the use of empirical distributions would lead to the absurd conclusion that the random variable would never exceed the threshold. Therefore it becomes imperative to fit the observed samples with some appropriate distribution. For reasons explained in the paper, it is desirable to use the so-called stable distributions to fit the set of samples. In most cases, stable distributions are heavy-tailed, in that they do not have finite variance (and may not even have finite mean). However, they often do a very good job of fitting the data. This is illustrated in this paper via examples from various application areas such as finance and weather.

Traditional media have a long history in covering natural disasters and crises. In many instances, these media remain major providers of information about an event. In recent years, however, information about natural disasters has increasingly been disseminated on a significant scale via Social Media platforms. These media provide new, additional and complementary angles on events and, combined with traditional media, produce a more complete spectrum of coverage. We present an approach, combining information from across the different kinds of media—traditional as well as social—and also across multiple languages, providing opportunities for first responders and decision makers to gain improved situational awareness and allowing for improved disaster relief, support, mitigation and resilience measures. The approach is put into context by relating it to a long-term strategic model including horizon-scanning and risk-management activities and a 5-phase disaster model forming the basis for information gathering and dissemination activities. To illustrate the research efforts the QuOIMA (Quelloffene Integrierte Multimedia Analyse) project, based on the pillars of cross-media, multimedia, and multilingual processing and representing major aspects of the general framework is presented. QuOIMA focuses on the information gathering aspects from the point of view of a first responder and crisis manager or -communicator rather than the management of active (outgoing) communication. Initial findings on data collected during the 2013 Central European floods are reported and discussed.

Catastrophic civil events give rise to large and complex response operations involving many agencies and individuals. Coordination of this response operation has been a long standing problem that cannot be solved by simply creating better procedures; on the contrary, it is an emergent and transmuting phenomenon that arises in the interactions between multiple agents as they confront high risks, short time-scales and poor data. This chapter examines coordination by considering crisis management in terms of distributed multi-agent decision-making. Coordination is then identified separately with the planning, actions, communications and knowledge of this multi-agent system. This framework is used to examine decision-making coordination at the scene of a major railway accident at Clapham UK in 1988. Decisions affected by the particular difficulty of ensuring the electrical isolation of the accident scene are a focus for the case study. Factors influencing the quality of coordination are assayed from the case study and this analysis informs our understanding of crisis management preparedness and training.

Universities, colleges and research centers have always been incubation centers for ideas, intellectual property and research on new technologies in the United States. Prior to Patriot Act, the only mandates universities, colleges and research centers faced were with radioactive material (e.g. U-235) and environmental protection guidelines (e.g. waste). After the passage of the Bioterrorism Preparedness 256 and Response Act of 2002 and the Homeland Security Chemical Facility Anti-Terrorism Standards passed in 2007, higher education institutions as well as research centers now are having to control for biological agents and chemical elements that are now mandated to be reported to the federal government under specific criteria. That being stated, organizational decision-making in universities, colleges and research centers have organizational culture to take into account when new procedures and policies are to be implemented. The case study seen in this chapter analyzes the decision-making process in context of the organizational culture when the new mandates were passed by the federal government for homeland security and disaster response purposes.

Information integration processes utilized in a context-aware decision support system for emergency response are considered. The system supports decision making by providing fused outputs of different sources. The chapter demonstrates advantages of ontology-based context to integrate information and to generate useful decisions. A case study concerning a fire response scenario illustrates the system operation. This study focuses on planning fire response actions and evacuation of people in danger using the ride-sharing technology.