Nonintrusive system for assistance and guidance in smart homes based on electrical devices identification
Introduction
In the last few years, technological devices, such as sensors and actuators, have become wildly spread in our everyday life. This reality, combined with recent advances in the field of Artificial Intelligence (AI) (Russell & Norvig, 2010) has led to the emergence of a new research paradigm, which is called Ambient Intelligence (AmI). Ambient intelligence (Ramos, Augusto, & Shapiro, 2008) refers to a multidisciplinary approach which consist of enhancing a common environment (room, building, car, etc.) with technology (e.g. infrared sensors, pressure mats, controllable lights, speakers and screens, radio-identification devices, etc.), in order to build a system that makes decisions based on real-time information and historical data to benefit the users within this environment,. In this way, technology merges with the environment, becoming nonintrusive, but remains ready to react to the occupant’s needs and to provide assistance. The main application of this AmI concept involves the development of smart homes (Augusto & Nugent, 2006), which are in fact houses equipped with ambient intelligent agents providing advanced, assistive services to a resident, in the performance of Activities of Daily Living (ADL). In fact, each of these ambient agents can be seen as a distinct expert system (Jackson, 1998), which emulates the decision-making ability of a human expert in order to provide adapted guidance at the right moment.
Given the recent emergence of this research field, several assistive tools, taking the forms of deployed expert systems in smart environments, have been proposed (Augusto et al., 2013; Peter & al., 2014; Giroux et al., 2015, Kyle et al., 2014, Fahim et al., 2014). These various ADL assistive systems are usually based on sensors, actuators, and multimedia equipment (Chu et al., 2012, Haigh et al., 2006, Lindsay et al., 2007, Mihailidis et al., 2007, Patterson et al., 2002, Patterson et al., 2005, Pollack et al., 2003, Rudary et al., 2004). Again in a few cases, the remote interaction for the monitoring and the assistance is available with the guidance system through communication technologies (Afridi, 2012, Haigh et al., 2006, Lindsay et al., 2007; Giroux & al., 2014; Bouchard & al., 2015). In fact, the purpose of these systems is to ensure a certain form of autonomy for these elderly people by assisting them in the performance of their daily tasks according to the types of errors monitored (temporal constraints, spatial constraints, specific ways of execution, etc.). Currently, most applications and systems designed to assist patients in ADL are intrusive and complex to deploy, to adjust, to control; many of these applications and system ignore the specificity of the cognitive deficit or just do not take errors into account (Fortin-Simard D., Bouchard K., & Bouchard B., 2015).
In summary, the literature on assistive systems can be divided into three groups: camera or vision-based systems, system relying on binary sensors, and systems based on signal analysis (RFID, Electrical, etc.). First, vision-based systems (Boger et al., 2006, Mihailidis et al., 2007; Czarnuch & al. 2013, Peters &t al., 2014) are intrusive, have a lack of robustness (sensible to brightness, colors change, etc.) and the information can be difficult to extract (image processing). In contrast, systems based on binary sensors (Haigh et al., 2006, Pollack et al., 2003; Kyle & al., 2013; Giroux et al., 2015) such as movement detectors, electromagnetic contacts or other basic sensors are not intrusive and the data are easily extractable, but they are inadequate to ensure fine-grained monitoring due to the lack of information they provide. The last type of systems (Arab et al., 2014, Fortin-Simard et al., 2015, Patterson et al., 2005) relies on signal analysis from different kinds of analogical sensors such as RFID, ultrasounds, etc. However, RFID technology is difficult to work with due to the inherent imprecision of the received signal strength. Also, most kinds of analogical sensors need a complex infrastructure with antennas, tags, etc. For these reasons, it makes them difficult to deploy, they need maintenance and are complex to install in an existing home. Finally, we can note that very few of these existing systems take into account the cognitive profile of the resident in the assistive process.
In this paper, we propose a new nonintrusive expert system for assistance and guidance in smart homes based on electrical device identification. The core of the system relies on an algorithmic approach used to recognize erratic behaviors related to cognitive deficits; it provides cues to guide the person in the completion of ongoing tasks. In this regard, one of the new features of our system is an appropriate cognitive assistance corresponding to the types of cognitive errors (step omission, step inversion, perseverance, temporal constraint, and cognitive overload) performed by the patient and based on a well-established cognitive test named the Naturalistic Action Test (NAT) (Schwartz, Segal, Veramonti, Ferraro, & Buxbaum, 2002).This is achieved through the study of load signatures of appliances represented by three features (active power (P), reactive power (Q) and line-to-neutral), which allows to determine the errors committed by the resident. This system is based on a single power analyzer only, which is placed in the electrical panel. We implemented this system within a real smart-home prototype equipped with household appliances used by the patient during his morning routine. Different multimedia prompting devices (iPad, screen, speakers, etc.) were used. We tested the system with real-case scenarios modeled from former clinical trials, which demonstrated the potential of our system in regards of the accuracy and the effectiveness when assisting a cognitively-impaired resident in the completion of his daily activities.
The paper is structured as follows: Section 2 defines some terms and concepts related to monitoring and assistance provided to cognitively-impaired people. Section 3 explains the categories of errors that may occur during sequences of activities. Moreover, the various errors that can be detected during the execution of the scenarios will also be described in this part. Section 4 defines and details related works about assistive applications and systems. Section 5 presents our new contribution based on an activity recognition system (Belley, Gaboury, Bouchard, & Bouzouane, 2014) in a three-dimensional space as well as two real scenarios subject to be observed in practice by ordinary and cognitively-impairment people during the morning routine. Consequently, Section 6 reveals the constraints of our new system when assisting cognitively-impaired patients.
Section snippets
Assistive technology
An assistive technology is a technology used to help people with physical or mental disabilities to perform some activities that might otherwise be difficult or impossible to achieve (Beech & Roberts, 2008). Consequently, these technologies help them to become more independent in their daily activities (Beech and Roberts, 2008, Iliev and Dotsinsky, 2011, Lindsay et al., 2007) and ensure their security (Miskelly, 2001). Moreover, assistive technology must be appropriate to avoid conflict with
Categories of errors
When a person with cognitive impairment begins a sequence of activities, cognitive assistance is helpful, because it recognizes which sequence of activities is in progress and again, it detects if necessary the problems related to this sequence. Indeed, although it is complex to determine, we can assume which kind of errors the patient has achieved according to the time elapsed between two actions, the order of actions, the expected duration for each action, and the actions in the sequence,
Related works
In the last several years, researchers (Augusto et al., 2013; Peter & al., 2014; Giroux et al., 2015, Kyle et al., 2014) have developed assistive systems exploiting plan-recognition models and to concretely experiment them in smart homes equipped with such cognitive assistance. Assisted cognition usually combines ideas from sensor networks (Augusto et al., 2013) and ubiquitous computing (Giroux et al., 2015), as artificial intelligence (Bouchard & al., 2014) and human–computer interaction (HCI)
New system for assistance
We have developed a nonintrusive and economic model for assistance, represented by the Fig. 1, based on a method with steady-state operations and in which variables considered correspond to the time when the scenario begins, the period of use of each appliance that we can evaluate through the active power (P), the reactive power (Q) and the line-to-neutral that supplies the appliance. The following are formulas (Belley et al., 2014) of the active power (P), expressed in watts, and reactive
Implementation and methodology
To experiment the assistance system proposed, an NIALM system (Belley et al., 2014) has been installed and deployed into our laboratory smart-home prototype infrastructure. Indeed, this system consists of a smart modular power analyzer (model: WM30 96), from the Carlo Gavazzi’s company, which is located in the main electric panel of the university laboratory. Here, this device monitors the power consumption within the smart home from a single electric source. In our case, our system mainly
Experiments and comparison of results
In order to experiment and assess the effectiveness of our assistive system based on the electrical load signatures of appliances, we conducted ten consecutive tests for different selected sequences of tasks. In fact, because of the number of possibilities related to sequences with or without error in the behavior, to save time and avoid repeating the same sequences, only two dissimilar sequences of tasks for each scenario (1 and 2), have been rigorously verified and tested. It corresponds to a
Comparison of our system
The comparison of our guidance system with those previously described in the Section 4 is rather complex. After considering all methodologies of each system, we notice that they are dissimilar; because they target various customers, they work with disparate information and have different end goals with regard to the category of activities that the system will assist. Consequently, it becomes really challenging to define comparison standards. However, our guidance system, compared to other
Limitation of our system
Despite the promising results obtained and the advantages that our prototype could bring to society, our system presents some limitations which must be addressed. For instance, during monitoring of a task-performance, information specific to its completion can hardly be determined; only speculations and deductions can be made. In other words, it means that it is only possible to determine whether the cognitively-impaired person turns on the toaster, but not if he puts bread into toaster or if
Conclusion
In this paper, we proposed a new nonintrusive expert system for assistance and guidance in smart homes, based on electrical signal analysis. We presented a new algorithmic approach capable of recognizing erratic behaviors related to cognitive deficits and of providing hints/prompts and reminders to guide a cognitively-impaired person in the completion of daily activities. The recognition aspect of the system relies on a very robust and intuitive approach to analyze the load signatures of
References (40)
- et al.
An efficient and inexpensive method for activity recognition within a smart home based on load signatures of appliances
Pervasive and Mobile Computing
(2014) - et al.
Autominder: An intelligent cognitive orthotic system for people with memory impairment
Robotics and Autonomous Systems
(2003) - Afridi, A. H. (2012). Mobile social computing as assistive technology for improving elderly care. In: Paper presented...
- Alzheimer’s Association. Mild cognitive impairment. Retrieved August (2013), from...
- et al.
Night optimised care technology for users needing assisted lifestyles
Behaviour & Information Technology, Taylor & Francis
(2013) - Arab, F., Bauchet, J., Pigot, H., Giroux A., & Giroux S. (2014), Design and assessment of enabling environments for...
- Beech, R., & Roberts, D. (2008). Assistive technology and older people. SCIE website–briefing paper...
- et al.
A planning system based on Markov decision processes to guide people with dementia through activities of daily living
Information Technology in Biomedicine, IEEE Transactions on
(2006) - et al.
Interactive activity recognition and prompting to assist people with cognitive disabilities
Journal of Ambient Intelligence and Smart Environments
(2012) - et al.
A real-world deployment of the COACH prompting system, Journal of Ambient Intelligence and Smart Environments (JAISE) – Design and Deployment of Intelligent
Environments
(2013)
ATHENA: A personalized platform to promote an active lifestyle and wellbeing based on physical
Mental and Social Health Primitives. Sensors
Coffee with jelly or unbuttered toast: commissions and omissions are dissociable aspects of everyday action impairment in Alzheimer’s disease
Neuropsychology
The independent lifestyle assistant: Lessons learned
Assistive Technology
Nonintrusive appliance load monitoring
Proceedings of the IEEE
Assisted living systems for elderly and disabled people: A short review
International Journal of Bioautomation
Automated Detection of Activity Transitions for Prompting
IEEE Transactions on Human-Machine Systems.
Cited by (28)
Assistance from the Ambient Intelligence: Cyber–physical system applications in smart buildings for cognitively declined occupants
2023, Engineering Applications of Artificial IntelligenceA time-series clustering methodology for knowledge extraction in energy consumption data
2020, Expert Systems with ApplicationsCitation Excerpt :They are responsible for maintaining the inner comfort through managing heating, ventilation and lighting conditions. However, a major concern is that those systems are prone to noise, light pollution and health problems (Belley, Gaboury, Bouchard, & Bouzouane, 2015). An additional complication is that energy demand is expected to increase in the near future due to the climate change caused by inefficient usage of energy, population growth and increase of time spent in buildings (Cao, Dai, & Liu, 2016).
Non-intrusive load disaggregation based on composite deep long short-term memory network
2020, Expert Systems with ApplicationsCitation Excerpt :The realization of NILD has high application value, which can bring benefits to the power company and users (Paterakis, Erdinc, Bakirtzis, & Catalao, 2015). Compared with intrusive load monitoring, the remarkable advantages of NILD mainly manifested in its economic investment is small and its practicability is strong (Belley, Gaboury, Bouchard, & Bouzouane, 2015). Therefore, since the NILD was put forward, it has attracted wide attention.
Identification of the electrical load by C-means from non-intrusive monitoring of electrical signals in non-residential buildings
2019, International Journal of Electrical Power and Energy SystemsCitation Excerpt :The study cases in their work validate the method reducing the likeness of similar appliances. In another example focused on home applications, Corinne et al. [10] developed a non-intrusive expert system for assistance in smart homes based on electrical device identification. This work uses algorithmic approach to analyze load signatures represented by active power, reactive power and line-to-neutral, this way the proposed system can recognize erratic behaviors.
User action and facial expression recognition for error detection system in an ambient assisted environment
2018, Expert Systems with ApplicationsCitation Excerpt :Fig. 2 represents the mains components of a common ambient assisting system integrated to a smart environment. In Belley et al. (2015), the authors introduced a nonintrusive ambient assisting system integrated to the LIARA lab. It main purpose is to provided assistance and guidance to the elderly and people with cognitive disabilities during the performance of ADL.
Recognizing multi-resident activities in non-intrusive sensor-based smart homes by formal concept analysis
2018, NeurocomputingCitation Excerpt :Their high computational costs are also a major drawback. For the non-intrusive one, besides various home appliances providing identifiable electrical load signature [27], WSNs establish an object network consisting of electronic components such as sensors, actuators or RFID tags to monitor environmental conditions. These components are usually embedded or attached to household furniture, doors, windows and other daily necessities.