Human fall detection on embedded platform using depth maps and wireless accelerometer

Highlights

• An embedded system for fully automatic fall detection.

• A method to achieve reliable fall detection with low false alarm ratio.

• An algorithm with low computational demands for person extraction in depth images.

• A freely available database for evaluation of fall detection, consisting of both accelerometric and depth data.

Abstract

Since falls are a major public health problem in an aging society, there is considerable demand for low-cost fall detection systems. One of the main reasons for non-acceptance of the currently available solutions by seniors is that the fall detectors using only inertial sensors generate too much false alarms. This means that some daily activities are erroneously signaled as fall, which in turn leads to frustration of the users. In this paper we present how to design and implement a low-cost system for reliable fall detection with very low false alarm ratio. The detection of the fall is done on the basis of accelerometric data and depth maps. A tri-axial accelerometer is used to indicate the potential fall as well as to indicate whether the person is in motion. If the measured acceleration is higher than an assumed threshold value, the algorithm extracts the person, calculates the features and then executes the SVM-based classifier to authenticate the fall alarm. It is a 365/7/24 embedded system permitting unobtrusive fall detection as well as preserving privacy of the user.

Introduction

Assistive technology or adaptive technology is an umbrella term that encompasses assistive and adaptive devices for people with special needs [1], [2]. Special needs and daily living assistance are often associated with seniors, disabled, overweight and obese, etc. Assistive technology for aging-at-home has become a hot research topic since it has big social and commercial value. One important aim of assistive technology is to allow elderly people to stay as long as possible in their home without changing their living style.

Wearable sensor-based systems for health monitoring are an emerging trend and in the near future they are expected to make possible proactive personal health monitoring along with better medical treatment. Inertial measurement units (IMUs) are low-cost and low power consumption devices with many potential applications. Current miniature inertial sensors can be integrated into clothes or shoes [3]. Inertial tracking technologies are becoming widely accepted for the assessment of human movement in health monitoring applications [4]. Wearable sensors offer several advantages over other sensors in terms of cost, weight, size, power consumption, ease of use and, most importantly, portability. Therefore, in the last decade, many different methods based on inertial sensors were developed to detect human falls. Falls are a major cause of injury for older people and a significant obstacle in independent living of the seniors. They are one of the top causes of injury-related hospital admissions in people aged 65 years and over. The statistical results demonstrate that at least one-third of people aged 65 years and over fall one or more times a year [5]. An injured elderly may be laying on the ground for several hours or even days after a fall incident has occurred. Therefore, significant attention has been devoted to developing an efficient wearable system for human fall detection [6], [7], [8], [9].

The most common method for wearable-device based fall detection consists in the use of a tri-axial accelerometer and a threshold-based algorithm for triggering an alarm. Such algorithms raise the alarm when the acceleration is larger than a threshold value [10]. A variety of accelerometer-based methods and tools have been proposed for fall detection [11]. Typically, such algorithms require relatively high sampling rate. However, most of them discriminate poorly between activities of daily living (ADLs) and falls, and none of which is universally accepted by elderly. One of the main reasons for non-acceptance of the currently available solutions by seniors is that the fall detectors using only accelerometers generate too much false alarms. This means that some daily activities are erroneously signaled as fall, which in turn leads to frustration of the users.

The main reason of high false ratio of accelerometer-based systems is the lack of adaptability together with insufficient capabilities of context understanding. In order to reduce the number of false alarms, many attempts were undertaken to combine both accelerometer and gyroscope [6], [12]. However, several ADLs like quick sitting have similar kinematic motion patterns with real falls and in consequence such methods might trigger many false alarms. As a result, it is not easy to distinguish real falls from fall-like activities using only accelerometers and gyroscopes. Another drawback of the approaches based on wearable sensors, from the user's perspective, is the need to wear and carry various uncomfortable devices during normal daily life activities. In particular, the elderly may forget to wear such devices. Moreover, in [13] it is pointed out that the common fall detectors, which are usually attached to a belt around the hip, are inadequate to be worn during the sleep and this results in the lack of ability of such detectors to monitor the critical phase of getting up from the bed.

In general, the solutions mentioned above are somehow intrusive for people as they require wearing continuously at least one device or smart sensor. On the other hand, these systems, comprising various kinds of small sensors, transmission modules and processing capabilities, promise to change the personal care, by supplying low-cost wearable unobtrusive solutions for continuous all-day and any-place health and activity status monitoring. An example of such solutions with a great potential are smart watches and smartphone-based technologies. For instance, in iFall application [14], data from the accelerometer is evaluated using several threshold-based algorithms and position data to determine the person's fall. If a fall is inferred, a notification is raised requiring the user's response. If the user does not respond, the system sends alerts message via SMS.

Despite several shortcomings of the currently available wearable devices, the discussed technology has a great potential, particularly, in the context of growing capabilities of signal processors and embedded systems. Moreover, owing to progress in this technology, data collection is no longer constrained to laboratory environments. In fact, it is the only technology that was successfully used in large scale collection of people motion data.

Video-cameras have largely been used for detecting falls on the basis of single CCD camera [15], [16], multiple cameras [17], specialized omni-directional ones [18] and stereo-pair cameras [19]. Video based solutions offer several advantages over others including the capability of detection of various activities. The further benefit is low intrusiveness and the possibility of remote verification of fall events. However, the currently available solutions require time for installation, camera calibration and they are not cheap. As a rule, CCD-camera based systems require a PC computer or a notebook for image processing. While these techniques might work well in controlled environments, in order to be practically applied they must be adapted to non-controlled environments in which neither the lighting nor the subject tracking is fully controlled. Typically, the existing video-based devices for fall detection cannot work in nightlight or low light conditions. Additionally, the lack of depth information can lead to lots of false alarms. What is more, their poor adherence to real-life applications is particularly related to privacy preserving. Nevertheless, these solutions are becoming more accessible, thanks to the emergence of low-cost cameras, the wireless transmission devices, and the possibility of embedding the algorithms. The major problem is acceptance of this technology by the seniors as it requires the placement of video cameras in private living quarters, and especially in the bedroom and the bathroom.

The existing video-based devices for fall detecting cannot work in nightlight or low light conditions. In addition, in most of such solutions the privacy is not preserved adequately. On the other hand, video cameras offer several advantages in fall detection over wearable devices-based technology, among others the ability to detect and recognize various daily activities. Additional advantage is low intrusiveness and the possibility of remote verification of fall events. However, the lack of depth information may lead to many false alarms. The existing technology permits reaching quite high performance of fall detection. However, as mentioned above it does not meet the requirements of the users with special needs.

Recently, Kinect sensor has been proposed to achieve fall detection [20], [21], [22]. The Kinect is a revolutionary motion-sensing technology that allows tracking a person in real-time without having to carry sensors. It is the world's first low-cost device that combines an RGB camera and a depth sensor. Thus, if only depth images are used it preserves the person's privacy. Unlike 2D cameras, it allows tracking the body movements in 3D. Since the depth inference is done using an active light source, the depth maps are independent of external light conditions. Owing to using the infrared light, the Kinect sensor is capable of extracting the depth maps in dark rooms. In the context of reliable fall detection systems, which should work 24 h a day and 7 days a week it is very important capability, as we already demonstrated in [21].

In order to achieve reliable and unobtrusive fall detection, our system employs both the Kinect sensor and a wearable motion-sensing device. When both devices are used our system can reliably distinguish between falls and activities of daily living. In such a configuration of the system the number of false alarms is diminished. The smaller number of false alarms is achieved owing to visual validation of the fall alert generated on the basis of motion data only. The authentication of the alert is done on the basis of depth data and analysis of the features extracted on depth maps. Owing to the determined in advance parameters describing the floor the system analyses not only the shape of the extracted person but also the distance between the person's center of gravity and the floor. In situations in which the use of the wearable sensor might not be comfortable, for instance during changing clothes, bathing, washing oneself, etc., the system can detect falls using depth data only. In the areas of the room being outside of the Kinect field of view the system can operate using data from motion-sensing device consisting of an accelerometer and a gyroscope only. Thanks to automatic extraction of the floor no calibration of the system is needed and Kinect can be placed according to the user preferences at the height of about 0.8–1.2 m. Owing to using of depth maps only our system preserves privacy of people undergoing monitoring as well as it can work at nighttime. The price of the system along with working costs are low thanks to the use of low-cost Kinect sensor and low-cost PandaBoard ES, which is a low-power, single-board computer development platform. The algorithms were developed with respect to both computational demands as well as real-time processing requirements.

The rest of the paper is organized as follows. Section 2 gives an overview of the main ingredients of the system, together with the main motivations for choosing the embedded platform. Section 3 is devoted to short overview of the algorithm. A threshold-based detection of the person fall is described in Section 4. In Section 5 we give details about extraction of the features representing the person in depth images. The classifier responsible for detecting human falls is presented in Section 6. The experimental results are discussed in Section 7. Section 8 provides some concluding remarks.