Data Fusion Applications

Frontmatter

Combining Two Imaging Modalities for Neuroradiological Diagnosis: 3D Representation of Cerebral Blood Vessels

Abstract

Today the integration of information from different imaging modalities in medicine such as Computer Tomography or Magnetic Resonance Imaging (MRI) is left to the physician and gets little support from computers. In the case of neuroradiological diagnosis, information about cerebral blood vessels is available from 3D volume data from Magnetic Resonance Angiography (MRA) and from 2D images generated by Digital Subtraction Angiography (DSA). The DSA images have a higher resolution than MRA data, and therefore neuroradiologists are highly interested in a 3D reconstruction of cerebral blood vessels from different DSA projections. On the other hand, MRA contains important functional information, the velocity of blood flow. This paper describes work in progress to make available to the physician the full 3D information from both imaging modalities including an approach to 3D reconstruction from DSA im ages which makes use of the MRA data. The 3D DSA reconstruction also opens the way to an integration of information from DSA with completely different types of information, for example information on anatomical structure or soft tissue from MRI. An integral part of this work is a pilot system for clinical validation.

As a typical case the neuroradiologist is interested in a 3D representation of the blood vessels surrounding an aneurysm which would significantly support a subsequent operation. To this end MRA data and DSA images are analyzed to extract information about the blood vessels such as position, orientation, width, and branchings. These items of information are input to the 3D reconstruction based on both imaging modalities. With these results physicians are able to inspect the relevant volume in three dimensions using various visualization tools including a combined display of MRA and the 3D DSA reconstruction.

The envisaged benefits of this system are reduced patient risk due to shorter DSA examinations involving less X-ray and contrast agent and improved representations of the pathology leading to a better diagnosis and treatment.

This work is part of COVIRA (Computer Vision in Radiology), project A2003 of the AIM (Advanced Informatics in Medicine) programme of the European Commission.¹ This project started in 1992 and will last until 1994.

Michael Bahner, Jürgen Dick, Bernd Kardatzki, Hanns Ruder, Matthias Schmidt, Arno Steitz, Carsten Bertram, Dietmar Hentschel, Thomas Hildebrand, Eckart Hundt, Robert Kutka, Sebastian Stier, Guido Gerig, Thomas Koller, Olaf Kübler, Gabor Szekely

Hybrid Inference Components for Monitoring of Artificial Respiration

Abstract

Medical expert systems are developed for supporting the complicated decision finding processes of the physicians. Generally, the decision finding in a clinical therapeutic process contains a complex data fusion problem. The problem is analysed and the conclusions are used for the development of medical expert systems.

The indentified pattern of problem solving behavior of the experts allows the structuring of the knowledge in sections. Different data structures need different representations and inference strategies. The higher organized expense for the data fusion is solved by implementation of a blackboard structure.

Finally, hybrid inference components for monitoring of artificial ventilation are proposed.

K. Gärtner, S. Fuchs, H. Jauch

Information Fusion in Monitoring Applications using the Category Model

Abstract

In many application areas similar monitoring tasks can be identified. The fundamental monitoring tasks are reported based on a technical domain analysis within different application areas. Common to most of the monitoring tasks are the recognition, surveillance, and control of objects, states and/or processes within a specific area.

Important for performing those basic tasks is the collection and processing of data coming from different sources. A typical characteristic of data acquired from different sources is the different type of data and the varying level of information. The process of combining data concerning a specific goal stated by a monitoring task is often called Data Fusion.

Here the more general term Information Fusionwill be introduced to describe the process of condensing data on varying level of information. The process of Information Fusion is based on an approach for a goal-oriented handling of information, the Category Model, which serves as the representational framework for the fundamental monitoring tasks.

Several results of this work have been performed within a cooperation with the ESPRIT-Project 5345 DIMUS (Data Integration in Multisensor Systems).

Wolfgang Steinke

A flexible Real-Time System for Vessel Traffic Control and Monitoring

Abstract

Surveillance systems are increasing their complexity in terms of real-time requirements, performance, sensor types and multiplicity, human computer interface and dependability. To this category belong Vessel Traffic Control Systems (VTS) that support control and monitoring of ship traffic in congested areas. The ESPRIT project 6373 TRACS, partially funded by CEC, is developing a new generation of VTS based on the exploitation of high performance computer and intelligent data processing & monitoring techniques. Keywords: surveillance system, VTS, distributed system, heterogeneous sensors, image processing, tracking, parallel architecture, data fusion, human computer interface, scene presentation, real-time system.

Livio Stefanelli

ESPRIT Project AZZURRO: Data Fusion for Marine Protection

Abstract

This paper will present the objectives of the AZZURRO project from the application point of view. This project is devoted to the development of advanced software techniques for the fusion of environmental data collected by two different sensors: an active sensor such as a Lidar Time Resolved Laser Fluorosensor (TRLF) and a passive one such as a multispectral scanner (Daedalus).

The main interesting aspects of this work will be:

• to prove if adequate data fusion techniques are able to implement the synergy existing between the measurements of the active and passive sensors;
• to develop data presentation methods acceptable by the final users;
• to demonstrate that the resulting thematic maps increase the knowledge of the marine environment providing also input for intervent actions.

AZZURRO is an Esprit III project (P7207). Agusta Sistemi(I), Onera(F), KUL(Katholieke Universiteit Leuven)(B), Piaggio(I) and Steria(F) are the partners involved in it. It will last three years starting from the 1 st of October 1992.

Susanna Ghelfo, Agusta Sistemi

ESPRIT Project DIMUS: Data Integration in Multisensor Systems

Abstract

The surveillance problem in the transport field seems to be quite critical due to the difficulty of controlling simultaneously different situations and environments.

Ansaldo Ricerche is participating to the realisation of a system capable of supporting the activity of a human operator in the surveillance of underground station environment. The first results of the project, called DIMUS, are shown in a demonstrator system that has been realised in a laboratory environment.

The demonstrator is able to perform:

• the environment monitoring with different sensors (TV cameras, microphones, photocells, tactile arrays, etc.);
• the processing and the fusion of the data coming from the sensors in order to detect the dangerous situations;
• the management of the interaction with the operator.

The project is carried out by a consortium of European Companies, Universities and Research Centres with Ansaldo Ricerche as main contractor and is partially founded by CEC ESPRIT programme (Project 5345 and 7809).

The paper shows the functionalities and the architecture of the DIMUS demonstrator, underlining the technologies used.

F. Benvenuto, M. Ferrettino, M. Pasquali, F. Perotti, P. Verrecchia

A Reflex-Based Approach to Fusion of Visual Data

Abstract

A system able to estimate in real-time the crowding level of an indoor environment is described. It receives visual information from cameras and it is based on a trainable Hyper Basis Functions network (HBF) that learns to map from a series of “examples” provided by a human operator to map visual inputs into crowding levels.

In this paper we present the preprocessing algorithms used to extract special features from gray levels images and a brief account of HBF technique. Experimental results obtained in real-life situations are finally reported and discussed¹.

A. Bozzoli, M. Rossi, R. Barbó, B. Caprile, G. Carlevaro

Sensor Fusion in a Peg-In-Hole Operation with a Fuzzy Control Approach

Abstract

An advanced robot system should be able to make independent decisions with the aid of diverse sensor information. Fuzzy logic is an appropriate tool to model and describe human experience and intuitive knowledge for decision-making. In this paper, we present a realization of a peg-in-hole operation with a fuzzy logic control approach. The sensor data of a force/torque sensor and a mini-camera vision system are integrated, and the fused information is used in order to decide the motion of the robot gripper for performing the insert operation. The information of these two sensors is used complementary to support the assembly process. Every data of a sensor is a weighted evidential information which influences the fuzzy inference. Additional sensors will be integrated in the next project phase. The above described research work is supported by the DFG German Research Council under contract SPP-RE489/22-2 and carried out at the Institute for Real-time Computer Systems and Robotics of the University of Karlsruhe.

Jianwei Zhang, Jörg Raczkowsky

Fuzzy Logic Techniques for Sensor Fusion in Real-Time Expert Systems

Abstract

Sensor data fusion is one of the main problems when developing on-line realtime expert systems. During the development of the MIP project several fuzzy logic techniques have beenbuilt up to help with this problem. MIP [1] is a realtime expert system for assistance to petrochemical processes, deployed and in production in a petrochemical plant of INH-REPSOL in Tarragona, Spain. The techniques presented in this paper ascertain confidence values to each sensor measurement. These techniques are currently in production in the plant.

J. A. Aguilar-Crespo, J. M. Domínguez, E. de Pablo, X. Alamán

Task-Directed Sensor-Planning

Abstract

Recently, there has been increased emphasis on employing reactive actions in robot task planning. The principle reasons for this change are to increase the robustness of robot actions by making them sensor controlled, and to accommodate dynamic, unpredictable environments. However, in many cases, supporting reactive mechanisms requires choosing sensor inputs for the reactive procedure. This paper addresses the issue of planning the sensing and fusion required to carry out a reactive robot program. A preliminary framework for planning and fusion is presented, and sensor planning is illustrated for the problem of replacing a mechanically attached plug in a space environment.

Gerhard Grunwald, Gregory D. Hager

Incremental Map-making in Indoor Environments

Abstract

This paper presents an algorithm that transforms laser range data from indoor environments, into a bird’s eye view of the robot surroundings. The algorithm is intended to be used for semi-autonomous robots, where the operator needs a global view of the environment. Emphasis is put on the important Line Extraction step, where two approaches are mentioned, one of which is a new Polyline Segmentation approach. Input data comes from odometry and a laser range scanner.

A. Ekman, D. Strömberg

Image Segmentation Improvement with a 3-D Microwave Radar

Abstract

Segmentation of intensity images is well known to be a complex problem. A single intensity image rarely provides all the information necessary for a correct segmentation. In this paper we describe how to improve segmentation results by fusing the intensity data with range data sensed by a microwave radar. The radar either gives a clue of where to partially repeat the segmentation process with different parameters or where to replace the intensity image segmentation data with the radar segmentation data. Practical results are obtained by applying the work to the vision system of an autonomous mobile robot where the goal is to identify and localize unknown objects.

A. Siebert, J. Ostertag, B. Radig, M. Rožmann, J. Detlefsen, J. Bernasch

A Vectorial Definition of Conceptual Distance for Prototype Acquisition and Refinement

Abstract

This paper addresses the problem of matching symbolic descriptions of structured objects. The adopted methodology is the basic component of the decision-making control of a learning system for the acquisition and the refinement of prototypes of visual objects. A vectorial matching evaluation is proposed, as opposed to traditional scalar similarity-measures. This allows the final result of the matching process to account for both the structural similarities of the compared objects, and the information about the reliability of available descriptions. The decision-making mechanism based on such vectorial representation is also described. With regards to the system’s overall flexibility, the advantage of connecting the matching output with the decision-making process via a common vectorial representation is highlighted.

Carlo Moneta, Gianni Vernazza, Rodolfo Zunino

Distributed Knowledge-based Systems for Integration of Image Processing Modules

Abstract

Knowledge Based Systems (KBSs) and their application to Image Processing problems is described. The focus is on Artificial Intelligence techniques for knowledge representation that have been used for Image processing purposes. Production Systems and Semantic Networks are addressed as the most widely used techniques. Then, distributed approaches to KBSs, which have been of growing interest in the last few years, are considered, together with uncertainty management techniques, which are a major issue when dealing with noisy data and incomplete a-priori knowledge. KBSs are also classified into two main categories depending on the type of knowledge representation: object centered and process centered KBSs. Existing Image Processing applications of KBSs, of both the centralized and the distributed types, are briefly reviewed.

C. Regazzoni

Spatial Fusion of Multisensor Visual Information for Crowding Evaluation

Abstract

Evaluation of crowding in complex environments is a problem currently addressed in the context of surveillance systems both to detect potentially dangerous situations (overcrowding) and for statistical purposes related to activity planning.

The project DIMUS (Data Integration in Multisensor Systems, ESPRIT P 5345) aims to attain such goals. In this paper, attention is focused on probabilistic Knowledge-Based techniques for statistical evaluation of crowding. To this end, a set of visual sensors are placed in a monitored scene to have different views of the scene itself.

The multilevel architecture of the system is modelled as a Bayesian Belief Network (BBN) of message-passing nodes. Each node corresponds to a virtual distributed processor that is used to obtain a probabilistic value of the locally detected crowding level.

Laboratory results, simulating real-life conditions, show that a good crowding evaluation can be achieved by the proposed approach.

M. Peri, C. Regazzoni, A. Tesei, G. Vernazza

Robust Multisensor Fusion in Underground Stations

Abstract

A major problem of data fusion in multisensor systems is the evaluation of multisensorial data correctness in order to ensure a correct and complete detection of the event occurrence (i.e. false alarms and missing alarm problems). One method is to construct high-level integrated sensorial data; this is bottom-up processing. Another method is to make predictions from models that impose constraints upon the event detection in a complex scene, like an underground station; this is top-downprocessing.

This paper investigates the bottom-upapproach in event detection based on a stepwise integration of the multisensorial data in DIMUS (Data Integration in Multisensor Systems, ESPRIT project 5345). Robust fusion of the sensor observations in the presence of sensor failures is ensured by redundant and diverse sensors. The current sensor reliability and a weighted voting decision strategy are used for the selection of the correct sensorial data. An efficient technique for evaluating the current reliability of the sensor observations during execution is also presented together with the construction of high level logical sensors for the detection of objects and persons in the prohibited areas of an underground station. Finally, the performance of the alarm reliability is investigated, and a Measure of Belief is defined based on statistical and dynamical alarm reliability indicators.

S. Pfleger, A. Milano

On Tracking Edges

Abstract

The tracking of contours extracted from image sequences is investigated. The algorithm is based on the fusion of intensity edges and motion information (extracted from optical flow) to infer the structure of objects in space. As far as the edge tracking process is concerned it is rather general and can be applied to any kind of “ego-” or “eco-” centric motion. Furthermore, in the case of ego-motion the constraint imposed by active motion of the camera can be exploited. Within this framework in order to facilitate the measure of the navigation parameters, a constrained egomotion strategy was adopted in which the position of the fixation point is stabilized during the navigation (in an anthropomorphic fashion). This constraint reduces the dimensionality of the parameter space without increasing the complexity of the equations.

The edge tracking causes an accumulation of the errors, relative to each instantaneous displacement, up to the global cumulative image displacement. These errors can be evaluated and reduced using a simple procedure, in which the computed image displacement is combined with a prediction based on the contour trajectory extrapolated from the preceding frames.

Experimental results on real image sequences are presented.

Massimo Tistarelli

Force and Position Sensing Resistors : An Emerging Technology

Abstract

Force Sensing Resistor™ devices (FSR™) superficially resemble a membrane switch, but unlike the conventional switch, change resistance inversely with applied force. For example, with a typical FSR sensor, a human finger applying a force from 0,1 N to 10 N will cause the sensor to change resistance continuously from 400 kΩ to 40kΩ. These sensors are ideal for touch control, and may be applied where a semi-quantitative sensor is called for that is relatively inexpensive, thin (>0,15 mm), durable (10.000.000 actuations), and environmentally resistant. These sensors can be made into arrays or single elements up to 60 cm × 80 cm, and cover forces in the tens of grams to tens of kilograms range.

Force and Position Sensing Resistor™ devices (FPSR™) can sense the position and normal force of a single actuator, such as a finger or a stylus, along either a straight line (a Linear Potentiometer) or on a planar surface (an XYZ Pad). Depending on the mechanical arrangement, positional resolution of 0,05 mm is possible.

Jannik Hagen, Michel Witte

Fusing Two Views using Object Motion

Abstract

A method is described for establishing plausible point correspondences between two widely separated views using statistical regularities in the motion of objects discovered through extended observation. Strong assumptions about the motion activity in a scene are required. These are approximately satisfied in many kinds of pedestrian scene.

David Hogg, Adam Baumberg

Mobile Robotics for the Surveillance of Fissile Materials Storage Areas: Sensors and Data Fusion

Abstract

A mobile robotics system is being developed to perform remote surveillance tasks inside a fissile material pilot storage area. The system is constituted of the operator’s console and a remotely guided vehicle carrying a manipulator arm and sensors (i.e., odometers, TV cameras, ultrasound sensors and a laser range finder). Nuclear Safeguards philosophy stresses the need for having sensors based on different physical principles, allowing for true sensor independence and complementarity. These aspects are important when fusing sensor data, in order to build corroborative evidence upon which decisions are to be taken. The paper describes the mobile robotics system and studies the implications of the on-board sensors in the system’s architecture both at the hardware and software levels. The use of sensor data for vehicle navigation is described, as well as the application of range images for environment authentication.

Joâo G. M. Gonçalves, Gilberto Campos, Vítor Santos, Vítor Sequeira, Filipe Silva

Data Fusion for Environmental Monitoring

Abstract

Environmental Monitoring is a very general term, comprising the whole spectrum of sensors and sensing—from local sensors to remote sensing from satellites. The term also implies a variety of sensor data processing aspects—from simple recording and report-generation to highly complex propagation models and prediction. This paper describes the aim of an ESPRIT project¹ that focuses on the monitoring of river and ground water on the basis of distributed sets of sensors.

Thies Wittig, Hao-Nhien Pham

Concluding Remarks: Advanced European Research on Data Fusion

Abstract

The workshop “Data Fusion Applications” presented recent research directions within the ESPRIT framework. The impact of the emerging data fusion technology on the recent European research is expressed by a number of 22 technical and overview papers reflecting the expertise of 69 experts in computer applications. Advanced research results on data fusion have been here discussed together with practical solutions in multi-sensor data fusion in a wide area of applications: medical diagnosis and patient monitoring, real-time expert systems, robotics, monitoring and control, marine protection, surveillance and safety in the public transportation, image processing and scene interpretation, and environment monitoring.

K. C. Varghese, S. Pfleger

Backmatter