Elsevier

Expert Systems with Applications

Volume 36, Issue 7, September 2009, Pages 10503-10511
Expert Systems with Applications

A cognitive surveillance system for detecting incorrect traffic behaviors

https://doi.org/10.1016/j.eswa.2009.01.034Get rights and content

Abstract

Traffic accidents, in which pedestrians are often involved, injure and kill many people yearly. One critical scene is the crosswalk, an important knocking down source due to both drivers’ and pedestrians’ incorrect behaviors. In this work, we propose a novel approach to detect and sanction these drivers’ abnormal behaviors. The multi-agent system paradigm embodies this approach, together with the use of FIPA standards, and a traffic ontology formalizes the domain knowledge. Besides developing the architecture, a simulator is also presented for generating random situations in the crosswalk scene. This system involves a first approximation in relation to the control of drivers’ abnormal behaviors in a crosswalk controlled by a traffic light, obtaining a high-scalable and flexible system which can be adapted to other domains.

Introduction

Traffic accidents injure and kill many people yearly. Most of these accidents involve pedestrians, which represent one of the largest sources of traffic-related casualties and fatalities. Table 1 shows the figures related to urban zone accidents in Spain during 2005 (information obtained from the Information System of the Spanish Traffic General Management or DGT). It is important to remark that most of the accidents are due to frontal and lateral collisions. However, the third negative figure refers to knockdowns, which involve the 41% of the mortal casualties and represent the highest number of fatalities in the urban area. One critical scene is the crosswalk, which can be controlled by a traffic light. In Spain, the DGT recorded 1504 traffic offences made on crosswalks in the year 2005, and 2647 in relation to a traffic light in the same year. As the reader can suppose, these figures only capture the traffic offences recorded, but not the whole incorrect behaviors carried out in all the Spanish cities.

In 2006, and to help to reduce these figures, a penalty points system was passed in Spain, which has similar characteristics to the systems implanted in other European countries. Drivers own a certain number of points that can be reduced in case of sanctions (with the corresponding economic fine). The aim of this system is to make drivers aware of driving responsibly. However, this system is not enough due to the resources needed to monitor potential traffic offences and, therefore, other approaches are needed to control them. Linked to this problem, there are several situations that are practically impossible to detect unless a traffic policeman controls them personally. Even so, there are many elements in the scene to be controlled and loss of concentration may appear (in the case of using a monitor). One of these scenes is the crosswalk, in which detecting both drivers’ and pedestrians’ incorrect behavior would require that a traffic policeman controls the crosswalk continuously.

Several approaches have been proposed for traffic management and traffic surveillance from a general point of view, as the works developed by Mohammadian, 2006, Tomas and Garcia, 2005, Uddin and Shioyama, 2005. These systems are video-based and, therefore, most of them need a person to detect incorrect behaviors through a monitor. Thus, this type of systems can be understood as surveillance-based systems for passively monitoring the traffic.

In this context, and with the aim of solving the previously discussed problems, we propose a solution for semi-automatically identifying and sanctioning these behaviors. To that end, we suggest a multi-agent architecture composed of intelligent agents specialized in the different tasks that make up this system. Thus, important benefits are obtained due to the autonomy, the social ability, and the reactivity of the proposed agents. In addition, the use of ontologies to reliably represent the knowledge about drivers’ behavior and to reason on it is introduced. This way, monitoring and control of crosswalk scenes can be carried out without directly spending human resources. Moreover, this system could be integrated into the penalty points system to semi-automatically penalize, for instance, to those drivers which ignore the obligation imposed by a red traffic light.

Section snippets

Related work

For a security expert system being able to behave intelligently, several steps should be covered: (i) capture and identification of the scene elements, (ii) detection of simple and composed events which take place in an environment, (iii) behavior analysis of each element and, finally, (iv) decision making process for acting accordingly. In Fig. 1, we propose a set of stages for designing a surveillance-based system. The conceptual architecture is composed of layers with different functionality

Scene description

We have chosen a common urban traffic situation that consists of a two-way street with a crosswalk controlled by a traffic light, as shown in Fig. 2. The function of such traffic light is to control the crosswalk and clearing the traffic. There are two reasons which justify the choice of this scene: the importance of reducing traffic accidents and the approach of using a delimited scene, whose complexity can be gradually increased.

The traffic light has two operation modes: active and inactive.

Experimental results

Besides testing the system by making use of a specific simulator developed for this work, we have also analyzed the video captured from a real urban zone. This environment corresponds with the simulated urban area, that is, a crosswalk regulated by a traffic light. Fig. 7 shows four frames related to such urban traffic zone. Obviously, the video has not been analyzed in real time because we are not allowed to use a static camera on the street for surveillance purposes. The objects relevant to

Conclusions

In this work, a novel approach to detect anomalous behaviors in a crosswalk has been proposed. The main advance of this work consists of dealing with the surveillance problem by defining the normality of the surveillance environment with high-level knowledge. The establishment of this system in a real world environment may involve a reduction of the number of accidents related to pedestrians in a crosswalk, thanks to the correction of drivers’ abnormal behavior by semi-automatically sanctioning

Acknowledgements

This work has been funded by the Regional Government of Castilla-la Mancha (PAC06-0141 and PBC06-0064) and the Spanish Ministry of Education and Science(TIN2007-62568).

References (17)

  • D. Makris et al.

    Path detection in video surveillance

    Image and Vision Computing

    (2002)
  • P. Remagnino et al.

    Distributed intelligence for multi-camera visual surveillance

    Pattern Recognition

    (2004)
  • Foresti, G. L., Micheloni, C., & Snidaro, L. (2004). Event classification for automatic visual-based surveillance of...
  • Foundation for Intelligent Physical Agents FIPA. (2000). FIPA Request Interaction Protocol Specification,...
  • Foundation for Intelligent Physical Agents FIPA. (2004). FIPA Agent Management Specification,...
  • Ghanem, N., DeMenthon, D., Doermann, D., & Davis, L. (2004). Representation and recognition of events in surveillance...
  • I. Haritaoglu et al.

    W4: Real-time surveillance of people and their activities

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (2000)
  • M. Henning

    A new approach to object-oriented middleware

    IEEE Internet Computing

    (2004)
There are more references available in the full text version of this article.

Cited by (32)

  • Multiple objects tracking by a highly decisive three-frame differencing-combined-background subtraction method with GMPFM-GMPHD filters and VGG16-LSTM classifier

    2020, Journal of Visual Communication and Image Representation
    Citation Excerpt :

    In addition, for providing safety instructions for driver assistance or even for the evolution of driver-less intelligent transportation systems, an expert tracking system is necessary. Indeed, acquiring accurate and up-to-date traffic information and statistics through traffic surveillance is increasingly high in demand for many reasons for law enforcement [6,7], for immediate controlling of traffic signals [8], and for collecting statistical data [9]. The tracking of vehicles can be categorized into three major approaches, viz. remote sensing-based [10,11], sensor network-based [12], and video-based [13,14] tracking methods.

  • Cognitive visual anomaly detection with constrained latent representations for industrial inspection robot

    2020, Applied Soft Computing Journal
    Citation Excerpt :

    In the research area, there have been wide studies on developing intelligent systems with cognition in various situations. Vallejo et al. [38] developed a cognitive surveillance system that can detect incorrect traffic behaviors from drivers. They embodied a multi-agent system paradigm and formalize the domain knowledge using ontology.

  • An expert fuzzy system to detect dangerous circumstances due to children in the traffic areas from the video content analysis

    2012, Expert Systems with Applications
    Citation Excerpt :

    One of the most prominent application areas for the development of surveillance systems is the traffic monitoring. In this context, there are applications of three different purposes: works that ‘obtain information on different traffic parameters’ (Collins et al., 2000), works that carry out a ‘traffic control for toll purposes or sanctions’ (Mohammadian, 2006; Vallejo, Albusac, Jimenez, Gonzalez, & Moreno, 2009) or researches whose objective is ‘monitoring to detect accidents automatically’ (Bo, Qimei, & Fan, 2006; Castro, Delgado, Medina, & Ruiz-Lozano, 2011; Lee, Hellinga, & Saccomanno, 2007). In the latter sense, a little studied aspect is the existence of children in traffic areas.

  • Pedestrian detection for intelligent transportation systems combining AdaBoost algorithm and support vector machine

    2012, Expert Systems with Applications
    Citation Excerpt :

    Therefore, road traffic safety has received much concern by governments and social organizations in China, such as the developments associated with intelligent transportation system (ITS) and safety driving assistant system (SDAS) technologies. The object of these technologies is to enhance comfort and safety of driver and road user (Gandhi & Trivedi, 2007; Vallejo, Albusac, Jimenez, Gonzalez, & Moreno, 2009). Among the factors that may contribute to traffic accidents, human error is one of the most important factors, such as driver’s inattention and wrong decisions.

View all citing articles on Scopus
View full text