Design and implementation of a distributed fall detection system based on wireless sensor networks

Much of the existing study on IoT has focused on addressing the power and computational resource constraints by the design of specific routing, MAC, and cross-layer protocols [14, 15]. However, for specific applications, further efforts should be directed toward finding ways to improve the sensing efficiency and minimize the data transmitted between sensor nodes. The focus of our study is to design a novel sensing paradigm to achieve the fall detection based on the resource-constrained wireless network.

Using wearable sensors is the most common method of detecting falls and other abnormal behaviors. Several researchers have explored the use of wearable acceleration sensors that placed in human clothing [16, 17]. Based on the multidimensional signals, a simple threshold or posture model was built to detect abnormal activities [18]. Bourke and Lyons [19] used gyroscopes mounted on the torso to measure the pitch and rolling angular velocity. They introduced a threshold-based algorithm to analyze the collected changes of angular velocity to make the fall alarm. Although accelerators and gyroscopes are able to provide discriminative time-varying signals for fall detection, they are intrusive and their usage is restrictive. The reason is that they need the cooperation of the elderly, which largely depends on the person's ability and willingness. The elder may forget to wear them, and wearable sensors will cause discomfort to the wearer.

For camera-based methods, the body shape change analysis algorithms are based on a general principle that the shape of a lying person is significantly different from that of a standing person. For example, Anderson et al. [20] used an HMM-based algorithm to detect the fall. The HMMs use the width-to-height ratio of the bounding box extracted from the silhouette. However, a single camera limits the viewing angle of the scene and, more importantly, the residents. The collaboration of multiple cameras can overcome this limitation. Cucchiara et al. [21] used a 3D shape of the human body to detect the fall. The 3D body shape is obtained by multiple cameras calibrated in prior. The advantages of employing camera-based methods include [6]: (1) Compared with wearable sensors, they are less intrusive because they are installed on the building, not worn by users. (2) The recorded video can be used for remote post verification and analysis. Generally speaking, body shape change detection can be real-time, whereas the 3D body shape needs more computation and more cameras. However, the existing algorithms in this category are mainly based on the shape feature extracted from the human contour, e.g., the width-to-height ratio of the bounding box; the accurate background and shadow substraction is the basic premise. This premise sometimes will be violated in real environment and then the performance of the camera-based methods will degrade.

Moreover, the high dimensional input visual data stream brings great computational and communication pressure and limitation for constructing pervasive visual sensor networks. In addition, the privacy is an inevitable concern of employing the camera-based methods. Recently, several studies exploit the advantages of PIR sensors in the automated surveillance. Shankar et al. [22] explored the response characteristics of PIR sensors and used them for human motion tracking. Hao et al. [23] confirmed a multiple-lateral-view based wireless PIR sensor system to perform human tracking. The human location can be surmised from the angle of arrival (AoA) of the distributed sensor modules. Hao et al. [24] subsequently showed the PIR sensor has the potentiality to have a reliable biometric solution for the verification/identification of a small group of human subjects. Burchett et al. [25] gave a lightweight biometric detection with PIR sensors. However, the studies mentioned above do not address how to discriminate the abnormal behavior from normal behavior as done in our studies.

The first study that employs the PIR sensors for fall detection is conducted by Sixsmith and Johnson [26]. They used an integrated pyroelectric sensor array (16 × 16) to collect human motion information without sensing the background. They installed the device on the wall for capturing the thermal image of the human body and estimating the vertical velocity. A neural network was trained to detect the falls in realistic scenarios. However, only the vertical velocity information is not robust enough to detect the fall. Their system is intrinsic to capture the body shape change as the camera-based method does. Besides, their experiments did not analyze the system's ability to distinguish between the fall and other similar normal vertical activities.

Recently, Liu et al. [27] used the direction-sensitive PIR sensors to construct a distributed fall detection system. Their distributed sensing paradigm is aimed at capturing the synergistic motion patterns of head, upper-limb and lower-limb. Their experiments results are encouraging. However, to capture the motion patterns of different parts of the human body, their system deployment has to be side-view, which means it is very easily occluded by other objects, e.g., furniture. Furthermore, their system is view-dependent, which means that for each sensor node, it could only detect the fall happened perpendicularly to the FOV of the PIR sensors. These limitations will be overcome in our design of a novel efficient sensing model.

To obtain the discriminative spatio-temporal feature of the fall, it is necessary to analyze the difference between the fall and other normal activities. This is the key to detect the fall in an efficient way. Based on the analysis, we design the corresponding sensing model.

The sophisticated optical flow method is employed to refine the analysis of the spatio-temporal feature as a human performs different activities. The estimation of pixel motion in two consecutive frames yields the optical flow computation [28]. Some sample images of normal activities and the fall are shown in Figure 1. The motion images are taken from a video of which the sample rate is 25 frames/s. We select three frames and its corresponding optical flow vector images in each category of activities for visualization. We divide the monitored region into four sub-regions, as shown in Figure 1a; furthermore, we aggregate the horizontal vector magnitude within these regions separately, denoted as horizontal motion energy (HME), as shown in Figure 2. The HME reflects the horizontal component of the human motion that crosses the sub-region perpendicularly, and will be used as the cue to analyze the spatio-temporal feature of different activities. As shown in Figure 2a "walking" and 2d "jogging", the peaks of HME of each sub-region appear one by one at roughly fixed interval, which reflect the motion characteristic of "walking" and "jogging" that pass these four sub-regions sequentially at about the same horizontal speed. As shown in Figure 2b,c, the peaks of HME of "sitting down" and "standing up" disappear or appear gradually, as they are controlled human activities. By contrast, the "fall" will cause the HME output of the adjacent sub-regions to overlap in a relatively short period of time, which corresponds with the velocity features of the fall [29], as shown in Figure 2e. These observations consist with the dynamics of the fall. In [29], Wu found two velocity features of the fall: (1) the magnitude of both vertical and horizontal velocities of the trunk will increase dramatically during the falling phase, reaching up to 2 to 3 times that of any other controlled movement; (2) the increase of the vertical and horizontal velocities usually occurs simultaneously, about 300-400 ms before the end of the fall process, which are strongly dissimilar with the controlled human activities.

Based on the above analysis, the time-varying HME of each sub-region is a discriminative spatio-temporal feature which can be used to distinguish the fall from other normal activities. To leverage this feature for fall detection in an efficient fashion, the feature-specific system should: (1) segment the monitored region into sub-regions and (2) the sensors collect the energy variation of each sub-region. This is the inspiration of our sensing model design, which will be elaborated in the succeeding sections.

To capture the most discriminative spatio-temporal feature of the fall, namely the HME of each sub-region, the sensing model has to be designed deliberately. Our model springs from the reference structure tomography (RST) paradigm, which uses multidimensional modulations to encode mappings between radiating objects and measurements [12].

The schematic diagram of our sensing model is shown in Figure 3. The object space refers to the space where the thermal object moves. The measurement space refers to the space where the PIR sensors are placed. The reference structure specifies the mapping from the object space to the measurement space [12], and is used to modulate the FOV of each PIR. In the case of opaque reference structure, the visibility function v _j (r) is binary valued depending on whether the point r in the object space is visible to the j th PIR sensor:

The function of the PIR sensors is to transform the incident radiation into measurements. The measurement of the j th PIR sensor is given by

where "*" denotes convolution, h(t) is the impulse response of the PIR sensor, Ω ∈ R³ is the object space covered by the FOV of the j th PIR sensor, v _j (r) is the visibility function, and s(r,t) is the thermal density function in the object space.

Assume that there are M sensors in the measurement space, and their FOVs are multiplexed. Thus, every point r in the object space can be associated with a binary signature vector [v _j (r)] ∈ {0, 1} ^M , which specifies its visibility to these M sensors. In the object space, contiguous points with the same signature form a cell that is referred to as a sampling cell. As a result, the 3D object space Ω can be divided into L discrete non-overlapping sampling cells, denoted as Ω _i

where i, j = 1, ..., L. Then (1) can be rewritten in discrete form

where v _ji is the j th element of the signature vector of Ω _i , and s i ( t ) = h ( t ) * ∫ Ω i s ( r , t ) d r is the sensor measurement of sampling cell Ω _i .

Then (3) can be written in a matrix form as

where m = [m _j (t)] ∈ ℝ^M×1is the measurement vector, V = [v _ji ] ∈ ℝ^M×Lis the measure matrix determined by the visibility modulation scheme, and s = [s _i (t)] ∈ ℝ^L×1is the sensor measurement of the sampling cells.

As the analysis mentioned in Section 3.1, to capture the discriminative spatio-temporal feature of the fall, it requires that the system can sense the time-varying HME of each sub-region. In our sensing model design, each sampling cell corresponds to a sub-region of the monitored region, and the PIR sensors are employed to capture the time-varying HME of these sampling cells. Therefore, our sensing model satisfies the design requirements. Our sensing model is an intrinsic non-isomorphic model, which means the number of PIR sensors M is less than the number of sampling cells L, and the measurement of each PIR sensor is a linear combination of the sampling cells [30]. However, its sensing efficiency is high, that is, it can robustly detect the fall by processing low-dimensional sensor data directly. The reason lies in that the our sensing model captures the most discriminative spatio-temporal feature of the fall by efficient spatial segmentation. In Section 5, we will elaborate the system implementation, and list the specification of the reference structure.

To represent the energy variation of the time-varying PIR sensor signals, it is critical to select an appropriate feature. Because the short time energy (STE) has been proved effective in depicting the energy variation of the sine-like waveform [31], we employ it as the feature of the PIR signals. The STE of the n th frame of the j th PIR is defined as

where j ∈ {1, ..., M} is the index of the PIR sensor, Z _n is the total number of the sampling points in the n th frame, avSTE _j (n) is the average energy of the sampling points, and m _j (k) is the signal amplitude of the k th sampling point.

Based on the extracted signal feature, STE of each frame, we can continue to design the corresponding classifier. The design of the classifier is problem-specific. Fall detection could be regarded as a binary classification problem, that is, fall or other normal activities. However, because it is difficult to design a single classifier to accomplish the task, the coarse-to-fine strategy is a better choice [32]. Thus, we design a binary hierarchical classifier for the fall detection.

The hierarchical classifier in our study is based on the hidden Markov models (HMMs). HMMs have been demonstrated as a powerful tool for modeling time-varying sequence data, such as speech [33] and video stream [34]. The parameters of a HMM can be denoted compactly by λ = (A, B, Π), where A = {a _ij } represents the hidden state transition probabilities matrix, B = {b _i (p(n))} denotes the probability density distribution of the observation vector, and Π = {π _i } is the initial state probability vector [33]. The parameters are learned from training data using Baum-Welch method. This is done for each class separately.

The binary hierarchical classifier we designed is the two-layer HMMs model, as shown in Figure 4. The normal activities include normal horizontal activities and normal vertical activities. The first-layer HMMs is responsible to classify the unknown activities into normal horizontal activities and the rest. The horizontal activities include walking and jogging, and the rest activities include fall, sitting down, and standing up. In other words, we need to train two HMMs to separate these two groups of activities, G₁ = {walking, jogging} and G₂ = {fall, sitting down, standing up}. Based on the Bayesian rule, given the sequence P = [p _j (n)], the likelihood output p(λ _i |P) is proportional to p(P|λ _i ). That is, label the input sequence P to the HMM with the height likelihood,

where λ₁ and λ₂ correspond to G₁ and G₂, respectively.

By the same method, the second-layer HMMs is to distinguish the fall from other normal vertical activities (sitting down and standing up).

In this section, we present some of the important hardware aspects of the preindustrial prototype developed by the authors, as shown in Figure 6. The sensor node will be mounted on the ceiling 3m above the floor, as shown in Figure 7. Hemispherical shape of the Fresnel lens arrays, model 8005, are obtained from Haiwang Sensors Corporation [35], and the PIR detectors, D205B, are obtained from Shenba Corporation [36]. The MCU embedded in the sensor node and the sink are Chipcon CC2430 modules. The CC2430 module combines the RF transceiver with an industry-standard enhanced 8051 MCU, 128 KB flash memory, 8KB RAM [37]. After configure the CC2430, the data generated by the sensor node will be sent to the sink based on the 2.4G Hz IEEE 802.15.4 (ZigBee) protocol, and then the sink will transport the data to the PC host by RS232 serial port for data processing.

We propose a proof-of-concept implementation scheme of the reference structure for our SensFall system. The optical flow analysis is view-dependent; however, to detect the fall that may happen in all directions within the monitored region, our system should be view-independent. Thus, the design of the reference structure is based on two principles: (1) the volume of each sampling cell is to be as equal as possible; (2) the sampling cells are symmetric along the axis of the monitored region.

For each PIR sensor, there is a Fresnel lens array located one focal length away from the detector. The Fresnel lens array is composed of a number of small Fresnel lens, which will collect the thermal energy within their FOV. Before visibility modulation, the FOV of each PIR sensor is a full cone. The opaque masks play the role of reference structure. The first type of mask, Type I, is a fan shape, as shown in Figure 8a. After applying such mask, the FOV of the PIR sensor is no longer a full cone, but partial cone shape, called fan cone. The fan cone's sweep angle is 120°, only 1/3 of the full cone. The second type of mask, Type II, is a ring shape, as shown in Figure 8b. The FOV of the PIR sensor after masked is still a full cone, but its cone angle β is less than that of the original cone. These two types of masks provide two degree of freedom (DOF) spatial partitions, bearing segmentation by Type I mask, and radial segmentation by Type II mask. By using these two kinds of masks and adjusting the parameters appropriately, the design principles (1) and (2) will be achieved.

In our system implementation, seven PIR sensors with masks are multiplexing to segment the object space several sampling cells. Four PIR sensors are masked by Type I mask, and the rest three PIR sensors are masked by type II mask. The specification of the FOV of each PIR, the sector angle ϕ and cone angle β, are listed in Table 1. In such configuration, the object space is segmented into 17 sampling cells, as shown in Figure 8c. The sampling cells are symmetric along the cone axis, and they could detect the falls happening in all directions. It means that the monitoring region of the sensor node is view independent.

Referring to Equation (4), M = 7, L = 17, and the measurement matrix V is given by

Our system can be regarded as a concrete implementation of the geometric reference structure [13]. The PIR sensors with Fresnel lens array are essentially sensitive to thermal change, especially at the boundaries of the sampling cells, which are preferable in terms of sensing efficiency in the sense of reducing the number of sensors involved without degrading the sensing performance. When the fall occurs, the human body will cross the boundaries of the sampling cells, and output of the PIR sensors array will reflect the spatio-temporal characteristic of the action. Typical output of the PIR sensors for different human activities is shown in Figure 9. Because the dynamic process of the fall is quite different from other normal activities, the output of the PIR sensors provides a powerful cue for fall detection.

The software framework of SensFall is shown in Figure 10. In our implementation, the sample rate of each PIR output is 25 Hz. The STE is calculated based on 2 s window with 1 s overlapping. A threshold is set to determine the starting point and ending point of each activity. Two layer HMMs will be trained by the training samples, and save as the model parameters λ = (A, B, Π) for testing. The main difference between our SensFall system and its camera-based counterpart is that our system does not contain the component of background segmentation [38]. The reason lies in the characteristic of the PIR sensors that they could only sense the motion of the thermal target, not including the chromatic background. As a result, the most obvious merit of our SensFall is its low dimensional input data stream for data processing.