ForeSight - Platform Approach for Enabling AI-based Services for Smart Living

The term “Smart Living” comprises several areas that are separated today: energy management, health and home automation [1]. Furthermore, smart home is a core element in a connected world. There is need for intelligent applications, which fulfil cross-domain use cases. Smart buildings, which include smart homes and commercial buildings, will take an important role to enable smart grid [2] and smart city related approaches. Such approaches can only be realised with intelligent, situation-adaptive control opportunities and building related services [3]. This leads to more comfort, better assistance and increased safety and security as well as improved resource efficiency and reduced overall costs. The reason for such advanced opportunities is the intense usage of AI. This paper describes a concept for a platform approach to enable such an intense AI usage in smart living.

The economic relevance of connected homes or buildings is proven by several key figures [4, 5]. In Germany there are currently approximately 19 million residential buildings and around 42 million households. The residential buildings have a share of approximately 98% of the total building stock. 57% of all Germans and 72% of single households lived for rent in 2015. [6] The 23 million rental housing units are managed by about 68,000 companies. Due to the situation described above, it is demanded at government level that the topic of digitisation and connectivity for the housing industry will be addressed. For other European countries, the situation seems not exactly the same, but similar.

There are currently various stand-alone smart home systems on the market. Some of these are based on wired or wireless technologies, e.g. KNX [7], Homematic [8, S. 231], Z-Wave [8, S. 216], free@home or digitalSTROM [9]. Moreover, most of the systems offer cloud-based remote access possibilities as an additional option for controlling the building from abroad. Of course, the biggest value proposition for the tenant is if he is able to add various devices from different vendors and consume third party services in his system. There are various middleware systems, which follow this idea like the openHAB system [10] or ioBroker.

Middleware systems usually provide an abstraction layer for devices. Thus, these systems are able to ensure data transfer between such systems and therefore achieve interoperability. In smart homes and for assistance systems in general, privacy and security issues [11] play an important role in Europe. This means that each data record should be extended with the information, who is allowed to do what actions at what time with it.

According to the paragraph before, it becomes obvious that available semantic information is useful to include domain knowledge in suitable data structures. For this purpose the concept of ontologies exists. Ontologies are semantic orders of domain related terms and their relationships [12]. Due to ontologies it is possible to start semantic search operations, combine two knowledge bases or test data for inconsistencies [13]. In the domain of smart living the SAREF ontology [14], BRICK [15] and Web of Things [16] are ontologies or approaches which should be investigated. The world of ontologies and knowledge representations lost some importance in recent years because solutions for numerous challenges were delivered by another part of AI, so-called machine learning (ML). In ML, algorithms and statistical methods are used to find solutions for optimization problems, e.g. more and more pre-trained neural networks offer robust high-quality answers to related questions [21]. In the context of smart living, speech recognition [22], object identification or energy management are relevant examples, where ML is used successfully.

Requirements are rapidly increasing in the smart building context, i.e. for achieving the energy transition flexible control and monitoring mechanisms need to be offered. Another challenge for German society is to assist elderly people and enable them to live in their preferred environment as long as possible. To achieve this, AI-based smart living technologies can help to extend this period of time. Thus, it is necessary to increase the amount of connected devices and enable energy consumption measurement and localisation technologies to create individual adaptable use cases. Currently, systems lack this functionality and do not offer such individually profile-based approaches. Future smart home systems need to have the specific user in mind, detect users’ behaviour and make the correct conclusions. Smart speaker can help here to locate and identify persons. Moreover voice user interfaces (VUI) can bring smart home technology to people who are not used to smartphones. The first numbers of sold smart speakers show that VUIs are a game changer. Anyway, there are lots of people who are still reluctant, because they do not know what happens to their data. Summing up, promising smart home platforms should offer a VUI and address privacy issues as well.

Existing middleware systems are slowly beginning to ensure interoperability [18, 19]. In addition to automating tasks, smart home systems begin to offer authentication options based on OAuth. This is pretty common for social media websites, i.e. signing in with your Google or Facebook account to another website [20]. On the one hand, this is quite comfortable for the user but on the other hand it will be even more advanced to sign in with your preferred service and avoid sharing data to lots of different servers abroad. Current platforms and eco systems lack organizational interoperability. This interoperability level should be persuaded, because only then it will be possible to create a flexible eco system with different partners, who are able to include their specific services and corresponding business models. The ideal place to store semantic information is in an ontology, e.g. the SAREF ontology. Ontologies can be extended [17], hence this should be considered by almost any platform project in the smart living domain.

Furthermore, concepts are needed which enable the transition from hard-coded if-else-statements towards context aware, dynamic and situation adaptive, self-learning approaches, which consider future trends as well. These new approaches need to add to current best practice approaches, e.g. plug-and-play mechanisms like the OSGI-based openHAB system. Changing the perspective from system to device, it is possible to speak from “Thinking Objects”.

The following ForeSight platform approach was initiated from members of the German Smart Living Business Initiative, a network of smart home and smart building experts. The goal of the ForeSight platform is to create a significant contribution to the further development of smart buildings and homes by providing cross-domain AI-based solutions in combination with established building automation technologies.

In the past, there have been various platform approaches, most of these were not able to conquer the market. There are technical, legal and economic challenges to deal with. In a first step core partners were allowed to found ForeSight. In this six months lasting project a useful platform concept needs to be created and relevant partners have to be identified, so that the value added chain is completed. In addition to that, established research institutes and organisations are added to the consortium to develop the concept further and further.

Due to literature research and experience, possible partners were detected. Afterwards, the team decided to start the working phase by founding a matrix based organisation, composed of several working groups (WG) (see Fig. 1). The figure shows WGs with domain specific focus and groups with cross-sectoral topics. The WG coordinators need to collect relevant data of their WG and interconnect with other WGs.

The created ForeSight platform concept should be evaluated and verified in laboratories and real world environments. Therefore, possible user stories and use cases need to be identified and implemented. There are several user story catalogues but usually there is no direct combination with AI, which is one of the core elements of this platform approach. Therefore, a new template for user stories was created and given to the members of the different WGs. This template considers relations to AI services, which are necessary for the specific use case.

With reference to the challenges section, ForeSight wants to reach a high level of interoperability, i.e. that service providers with different business models can replace their services without much effort. The resident focuses on one use case and the platform takes care of necessary processes in the background. To identify users is one of ForeSight’s basic features and the key to enable user specific use case adaptions. Horizontal interoperability ensures that the manufacturers and cross-domain interchangeability of devices works. Vertical interoperability considers that interoperability between two services is available, i.e. energy management use cases are interchangeable, although tenants are clients from different energy providers. Following the thinking objects approach it is necessary to offer the most appropriate AI method in relation to a client’s use case. It makes sense to offer three subsystems: first, the AI method platform module; second, an IoT platform module to make sure that all commands can be transported to several vendor-independent devices and third, service-related apps which can be built and ran on the IoT platform module and are allowed to connect to the AI module. Several AI base services are necessary and will be made available in ForeSight:

The AI platform connects with ForeSight service apps and the IoT platform component. This service-related app offers the AI module data and a preferred use case. Now the AI method finder needs to check the quality of the data, pick the most appropriate AI tool and start the function. Afterwards, the answer of the AI based platform module will be sent back. To generate high quality answers for accessing apps, several tasks need to be addressed (see Fig. 2).

To get an impression of the time schedule, an excerpt of the ForeSight roadmap is shown below.

In addition to the time schedule we defined some milestones and evaluation steps:

When designing a platform, one of the first questions which arises is who wants to operate such a platform. The ForeSight consortium consists of several companies, which are experienced in running platforms. A flexible service-based approach for running the platform seems to outperform a monolithic approach, as first interviews showed.

Numerous use cases from the ML area require high quality data in order to train the ML models, e.g. neural networks. Such data is often not available and must be generated, which usually takes an enormous amount of time. It will be hard to consider such soft facts in the AI method finder function to answer the service-based apps.

In the past, several critical factors for smart home platforms were identified, i.e. IT security, privacy and economic beneficial business models for all stakeholders. The appropriate treatment of these topics needs to be considered from the very beginning to avoid major concept changes later, which results in high costs.