Elsevier

Expert Systems with Applications

Volume 42, Issue 19, 1 November 2015, Pages 6552-6577
Expert Systems with Applications

Nonintrusive system for assistance and guidance in smart homes based on electrical devices identification

https://doi.org/10.1016/j.eswa.2015.04.024Get rights and content

Highlights

  • We present an assistive system for guiding cognitively-impaired people in performing daily activities.

  • The system is based on electrical signal analysis with a single sensor.

  • The system is able to detect cognitive errors and send prompts accordingly.

  • The system has been implemented, deployed and tested in real home environment.

  • The simulation of real-case scenarios has shown promising results.

Abstract

Recently, sensors and actuators have quickly spread throughout our everyday life. These devices are robust, cheap, accessible, connected to the Internet, etc. With the growing needs in terms of human and medical resources to help cognitively-impaired people to remain at home, researchers are investing in new ways to exploit this technology with artificial intelligence, in order to build expert systems to assist the residents in their daily activities. Several systems have been proposed in the last few years, mostly based on binary sensors, cameras and other sensors such as Radio-frequency identification (RFID) tags. Cameras are very intrusive, binary sensors (such as movement detectors) give only basic information, and other types of sensors (such as RFID) need complex deployment. In this context, this paper presents a new assistive expert system based on electric device identification to address the problem of guidance and supervision in the performance of activities for people with cognitive disorders living in a smart home. This system is solely based on a single power analyzer placed in the electric panel. We propose an algorithmic approach used to recognize erratic behaviors related to cognitive deficits and provides cues to guide the person in the completion of an ongoing task. This is achieved through load signatures study of appliances represented by three features (active power (P), reactive power (Q) and line-to-neutral), which allows to determine the errors committed by the resident. We implemented this system within a genuine smart-home prototype equipped with household appliances used by the patient during his morning routines. Different multimedia prompting devices (iPad, screen, speakers, etc.) were used. We tested the system with real-case scenarios modeled from former clinical trials, allowing demonstration of accuracy and effectiveness of our system in assisting a cognitively-impaired resident in the completion of daily activities.

Introduction

In the last few years, technological devices, such as sensors and actuators, have become wildly spread in our everyday life. This reality, combined with recent advances in the field of Artificial Intelligence (AI) (Russell & Norvig, 2010) has led to the emergence of a new research paradigm, which is called Ambient Intelligence (AmI). Ambient intelligence (Ramos, Augusto, & Shapiro, 2008) refers to a multidisciplinary approach which consist of enhancing a common environment (room, building, car, etc.) with technology (e.g. infrared sensors, pressure mats, controllable lights, speakers and screens, radio-identification devices, etc.), in order to build a system that makes decisions based on real-time information and historical data to benefit the users within this environment,. In this way, technology merges with the environment, becoming nonintrusive, but remains ready to react to the occupant’s needs and to provide assistance. The main application of this AmI concept involves the development of smart homes (Augusto & Nugent, 2006), which are in fact houses equipped with ambient intelligent agents providing advanced, assistive services to a resident, in the performance of Activities of Daily Living (ADL). In fact, each of these ambient agents can be seen as a distinct expert system (Jackson, 1998), which emulates the decision-making ability of a human expert in order to provide adapted guidance at the right moment.

Given the recent emergence of this research field, several assistive tools, taking the forms of deployed expert systems in smart environments, have been proposed (Augusto et al., 2013; Peter & al., 2014; Giroux et al., 2015, Kyle et al., 2014, Fahim et al., 2014). These various ADL assistive systems are usually based on sensors, actuators, and multimedia equipment (Chu et al., 2012, Haigh et al., 2006, Lindsay et al., 2007, Mihailidis et al., 2007, Patterson et al., 2002, Patterson et al., 2005, Pollack et al., 2003, Rudary et al., 2004). Again in a few cases, the remote interaction for the monitoring and the assistance is available with the guidance system through communication technologies (Afridi, 2012, Haigh et al., 2006, Lindsay et al., 2007; Giroux & al., 2014; Bouchard & al., 2015). In fact, the purpose of these systems is to ensure a certain form of autonomy for these elderly people by assisting them in the performance of their daily tasks according to the types of errors monitored (temporal constraints, spatial constraints, specific ways of execution, etc.). Currently, most applications and systems designed to assist patients in ADL are intrusive and complex to deploy, to adjust, to control; many of these applications and system ignore the specificity of the cognitive deficit or just do not take errors into account (Fortin-Simard D., Bouchard K., & Bouchard B., 2015).

In summary, the literature on assistive systems can be divided into three groups: camera or vision-based systems, system relying on binary sensors, and systems based on signal analysis (RFID, Electrical, etc.). First, vision-based systems (Boger et al., 2006, Mihailidis et al., 2007; Czarnuch & al. 2013, Peters &t al., 2014) are intrusive, have a lack of robustness (sensible to brightness, colors change, etc.) and the information can be difficult to extract (image processing). In contrast, systems based on binary sensors (Haigh et al., 2006, Pollack et al., 2003; Kyle & al., 2013; Giroux et al., 2015) such as movement detectors, electromagnetic contacts or other basic sensors are not intrusive and the data are easily extractable, but they are inadequate to ensure fine-grained monitoring due to the lack of information they provide. The last type of systems (Arab et al., 2014, Fortin-Simard et al., 2015, Patterson et al., 2005) relies on signal analysis from different kinds of analogical sensors such as RFID, ultrasounds, etc. However, RFID technology is difficult to work with due to the inherent imprecision of the received signal strength. Also, most kinds of analogical sensors need a complex infrastructure with antennas, tags, etc. For these reasons, it makes them difficult to deploy, they need maintenance and are complex to install in an existing home. Finally, we can note that very few of these existing systems take into account the cognitive profile of the resident in the assistive process.

In this paper, we propose a new nonintrusive expert system for assistance and guidance in smart homes based on electrical device identification. The core of the system relies on an algorithmic approach used to recognize erratic behaviors related to cognitive deficits; it provides cues to guide the person in the completion of ongoing tasks. In this regard, one of the new features of our system is an appropriate cognitive assistance corresponding to the types of cognitive errors (step omission, step inversion, perseverance, temporal constraint, and cognitive overload) performed by the patient and based on a well-established cognitive test named the Naturalistic Action Test (NAT) (Schwartz, Segal, Veramonti, Ferraro, & Buxbaum, 2002).This is achieved through the study of load signatures of appliances represented by three features (active power (P), reactive power (Q) and line-to-neutral), which allows to determine the errors committed by the resident. This system is based on a single power analyzer only, which is placed in the electrical panel. We implemented this system within a real smart-home prototype equipped with household appliances used by the patient during his morning routine. Different multimedia prompting devices (iPad, screen, speakers, etc.) were used. We tested the system with real-case scenarios modeled from former clinical trials, which demonstrated the potential of our system in regards of the accuracy and the effectiveness when assisting a cognitively-impaired resident in the completion of his daily activities.

The paper is structured as follows: Section 2 defines some terms and concepts related to monitoring and assistance provided to cognitively-impaired people. Section 3 explains the categories of errors that may occur during sequences of activities. Moreover, the various errors that can be detected during the execution of the scenarios will also be described in this part. Section 4 defines and details related works about assistive applications and systems. Section 5 presents our new contribution based on an activity recognition system (Belley, Gaboury, Bouchard, & Bouzouane, 2014) in a three-dimensional space as well as two real scenarios subject to be observed in practice by ordinary and cognitively-impairment people during the morning routine. Consequently, Section 6 reveals the constraints of our new system when assisting cognitively-impaired patients.

Section snippets

Assistive technology

An assistive technology is a technology used to help people with physical or mental disabilities to perform some activities that might otherwise be difficult or impossible to achieve (Beech & Roberts, 2008). Consequently, these technologies help them to become more independent in their daily activities (Beech and Roberts, 2008, Iliev and Dotsinsky, 2011, Lindsay et al., 2007) and ensure their security (Miskelly, 2001). Moreover, assistive technology must be appropriate to avoid conflict with

Categories of errors

When a person with cognitive impairment begins a sequence of activities, cognitive assistance is helpful, because it recognizes which sequence of activities is in progress and again, it detects if necessary the problems related to this sequence. Indeed, although it is complex to determine, we can assume which kind of errors the patient has achieved according to the time elapsed between two actions, the order of actions, the expected duration for each action, and the actions in the sequence,

Related works

In the last several years, researchers (Augusto et al., 2013; Peter & al., 2014; Giroux et al., 2015, Kyle et al., 2014) have developed assistive systems exploiting plan-recognition models and to concretely experiment them in smart homes equipped with such cognitive assistance. Assisted cognition usually combines ideas from sensor networks (Augusto et al., 2013) and ubiquitous computing (Giroux et al., 2015), as artificial intelligence (Bouchard & al., 2014) and human–computer interaction (HCI)

New system for assistance

We have developed a nonintrusive and economic model for assistance, represented by the Fig. 1, based on a method with steady-state operations and in which variables considered correspond to the time when the scenario begins, the period of use of each appliance that we can evaluate through the active power (P), the reactive power (Q) and the line-to-neutral that supplies the appliance. The following are formulas (Belley et al., 2014) of the active power (P), expressed in watts, and reactive

Implementation and methodology

To experiment the assistance system proposed, an NIALM system (Belley et al., 2014) has been installed and deployed into our laboratory smart-home prototype infrastructure. Indeed, this system consists of a smart modular power analyzer (model: WM30 96), from the Carlo Gavazzi’s company, which is located in the main electric panel of the university laboratory. Here, this device monitors the power consumption within the smart home from a single electric source. In our case, our system mainly

Experiments and comparison of results

In order to experiment and assess the effectiveness of our assistive system based on the electrical load signatures of appliances, we conducted ten consecutive tests for different selected sequences of tasks. In fact, because of the number of possibilities related to sequences with or without error in the behavior, to save time and avoid repeating the same sequences, only two dissimilar sequences of tasks for each scenario (1 and 2), have been rigorously verified and tested. It corresponds to a

Comparison of our system

The comparison of our guidance system with those previously described in the Section 4 is rather complex. After considering all methodologies of each system, we notice that they are dissimilar; because they target various customers, they work with disparate information and have different end goals with regard to the category of activities that the system will assist. Consequently, it becomes really challenging to define comparison standards. However, our guidance system, compared to other

Limitation of our system

Despite the promising results obtained and the advantages that our prototype could bring to society, our system presents some limitations which must be addressed. For instance, during monitoring of a task-performance, information specific to its completion can hardly be determined; only speculations and deductions can be made. In other words, it means that it is only possible to determine whether the cognitively-impaired person turns on the toaster, but not if he puts bread into toaster or if

Conclusion

In this paper, we proposed a new nonintrusive expert system for assistance and guidance in smart homes, based on electrical signal analysis. We presented a new algorithmic approach capable of recognizing erratic behaviors related to cognitive deficits and of providing hints/prompts and reminders to guide a cognitively-impaired person in the completion of daily activities. The recognition aspect of the system relies on a very robust and intuitive approach to analyze the load signatures of

References (40)

  • C. Belley et al.

    An efficient and inexpensive method for activity recognition within a smart home based on load signatures of appliances

    Pervasive and Mobile Computing

    (2014)
  • M.E. Pollack et al.

    Autominder: An intelligent cognitive orthotic system for people with memory impairment

    Robotics and Autonomous Systems

    (2003)
  • Afridi, A. H. (2012). Mobile social computing as assistive technology for improving elderly care. In: Paper presented...
  • Alzheimer’s Association. Mild cognitive impairment. Retrieved August (2013), from...
  • J. Augusto et al.

    Night optimised care technology for users needing assisted lifestyles

    Behaviour & Information Technology, Taylor & Francis

    (2013)
  • Arab, F., Bauchet, J., Pigot, H., Giroux A., & Giroux S. (2014), Design and assessment of enabling environments for...
  • Beech, R., & Roberts, D. (2008). Assistive technology and older people. SCIE website–briefing paper...
  • J. Boger et al.

    A planning system based on Markov decision processes to guide people with dementia through activities of daily living

    Information Technology in Biomedicine, IEEE Transactions on

    (2006)
  • Y. Chu et al.

    Interactive activity recognition and prompting to assist people with cognitive disabilities

    Journal of Ambient Intelligence and Smart Environments

    (2012)
  • S. Czarnuch et al.

    A real-world deployment of the COACH prompting system, Journal of Ambient Intelligence and Smart Environments (JAISE) – Design and Deployment of Intelligent

    Environments

    (2013)
  • M. Fahim et al.

    ATHENA: A personalized platform to promote an active lifestyle and wellbeing based on physical

    Mental and Social Health Primitives. Sensors

    (2014)
  • Fortin-Simard, D., Bilodeau J. S., Bouchard, K., Gaboury, S., Bouchard, B., & Bouzouane A. (2015), Exploiting passive...
  • T. Giovannetti et al.

    Coffee with jelly or unbuttered toast: commissions and omissions are dissociable aspects of everyday action impairment in Alzheimer’s disease

    Neuropsychology

    (2008)
  • Giroux, S., Bier, N., Pigot, H., Bouchard, B., Bouzouane, A., Levasseur, M., Couture, M., Bottari, C., Swaine, B.,...
  • K.Z. Haigh et al.

    The independent lifestyle assistant: Lessons learned

    Assistive Technology

    (2006)
  • G.W. Hart

    Nonintrusive appliance load monitoring

    Proceedings of the IEEE

    (1992)
  • I. Iliev et al.

    Assisted living systems for elderly and disabled people: A short review

    International Journal of Bioautomation

    (2011)
  • Jackson, Peter (1998), Introduction To Expert Systems (3 ed.), Addison Wesley, p. 2, ISBN...
  • D.F. Kyle et al.

    Automated Detection of Activity Transitions for Prompting

    IEEE Transactions on Human-Machine Systems.

    (2014)
  • Lapointe, J., Bouchard, B., Bouchard, J., Potvin, A., & Bouzouane, A. (2012). Smart homes for people with Alzheimer’s...
  • Cited by (28)

    • A time-series clustering methodology for knowledge extraction in energy consumption data

      2020, Expert Systems with Applications
      Citation Excerpt :

      They are responsible for maintaining the inner comfort through managing heating, ventilation and lighting conditions. However, a major concern is that those systems are prone to noise, light pollution and health problems (Belley, Gaboury, Bouchard, & Bouzouane, 2015). An additional complication is that energy demand is expected to increase in the near future due to the climate change caused by inefficient usage of energy, population growth and increase of time spent in buildings (Cao, Dai, & Liu, 2016).

    • Non-intrusive load disaggregation based on composite deep long short-term memory network

      2020, Expert Systems with Applications
      Citation Excerpt :

      The realization of NILD has high application value, which can bring benefits to the power company and users (Paterakis, Erdinc, Bakirtzis, & Catalao, 2015). Compared with intrusive load monitoring, the remarkable advantages of NILD mainly manifested in its economic investment is small and its practicability is strong (Belley, Gaboury, Bouchard, & Bouzouane, 2015). Therefore, since the NILD was put forward, it has attracted wide attention.

    • Identification of the electrical load by C-means from non-intrusive monitoring of electrical signals in non-residential buildings

      2019, International Journal of Electrical Power and Energy Systems
      Citation Excerpt :

      The study cases in their work validate the method reducing the likeness of similar appliances. In another example focused on home applications, Corinne et al. [10] developed a non-intrusive expert system for assistance in smart homes based on electrical device identification. This work uses algorithmic approach to analyze load signatures represented by active power, reactive power and line-to-neutral, this way the proposed system can recognize erratic behaviors.

    • User action and facial expression recognition for error detection system in an ambient assisted environment

      2018, Expert Systems with Applications
      Citation Excerpt :

      Fig. 2 represents the mains components of a common ambient assisting system integrated to a smart environment. In Belley et al. (2015), the authors introduced a nonintrusive ambient assisting system integrated to the LIARA lab. It main purpose is to provided assistance and guidance to the elderly and people with cognitive disabilities during the performance of ADL.

    • Recognizing multi-resident activities in non-intrusive sensor-based smart homes by formal concept analysis

      2018, Neurocomputing
      Citation Excerpt :

      Their high computational costs are also a major drawback. For the non-intrusive one, besides various home appliances providing identifiable electrical load signature [27], WSNs establish an object network consisting of electronic components such as sensors, actuators or RFID tags to monitor environmental conditions. These components are usually embedded or attached to household furniture, doors, windows and other daily necessities.

    View all citing articles on Scopus
    View full text