Distributed, Ambient and Pervasive Interactions

This paper presents the design and programming of origami-based, dynamic surfaces. Two functional prototypes were developed: a fixed-site prototype and a modular prototype. The paper discusses the hardware and software design of each prototype. The hardware design part discusses the physical structure of the system. The design of the system software includes developing algorithms to make decisions and providing information to the actuation mechanism to respond by changing the physical state of the system. The design of the modular-based prototype employs a “minimum inventory maximum diversity” strategy to develop programmable units that can be tessellated in a variety of ways to construct culturally inspired, dynamic geometric patterns.

VR is making inroads into industrial applications, especially in product design and presentation, training and simulation use cases. A focus of these applications is on the visualization, exploiting 3D immersive graphics as a central feature of VR. While navigation is often required in these applications to select a suitable location for observing the visualization or interacting with the simulation content it is not in the focus of the designers, often leading to the selection of techniques that are convenient during development because they are available in the implementation toolkit. However, these may be less than ideal for the industrial application context. A large variety of VR techniques for navigation have been proposed in the literature and implemented, both for motion control and guidance, but there is currently little established knowledge on their respective benefits and shortcoming, especially in non-gaming applications. To enable designers of industrial VR environments to make informed choices of navigation techniques we present a testbed that enables quick prototyping and comparative evaluation in the specific application context and with the intended target audience, making a user centered selection of navigation techniques viable.

Contemporary computer systems fully integrate people’s daily lives in the most diversified contexts. The adequate construction of those systems demands well-defined patterns for including a whole community of users encompassing people with disabilities. Current interaction scenarios with pervasive computing technologies present design challenges for the applicability of standards and guidelines such as those of the W3C Web Content Accessibility Guidelines (WCAG), suited for Web systems. Such scenarios require further studies to deal with the endeavor of supporting designers to project pervasive interactive systems that consider all people, regardless of their condition and ability, allowing different types of interactions to arise. In this article, we conduct an exploratory study to investigate the potential and limitations of current accessibility guidelines for pervasive systems. We explore the last decade of literature studies presenting solutions and challenges for accessibility found for pervasive and ubiquitous computing contexts. The results of this research indicate that it is desirable to define accessibility models for pervasive systems if we consider an accessibility layer that will indicate principles and solutions that should be generic enough to support different categories of impairments and contexts.

Smart environments, comprised of networked embedded devices, improve the lives of their users by providing them with a variety of assistive services that traditional built environments are incapable of supporting. However, as the number of connected devices in smart environments continue to increase, so does the level of complexity involved in interacting with these environments. Traditional human-computer interaction techniques are not always well-suited for smart environments and this poses some unique usability challenges. To facilitate interactions within such technology-rich smart environments, new models and interaction interfaces need to be developed. In this paper we propose a multi-modal approach to smart environment interaction and explore two novel interaction interfaces: gesture-based interface and mixed-reality-based interface. We also conducted a user study to compare the learnability, efficiency and memorability of these new interfaces to two more commonly used interfaces: voice-based interface and a smartphone GUI-based interface. Our user study experiment involved four light control tasks that subjects were asked to complete using the four interaction interfaces. Study subjects found different interaction techniques to be more suitable for different tasks based on the type, complexity and context of the task. Our analysis of the study results and subject feedback suggest that a multi-modal approach is preferable to a uni-modal approach for interacting with smart environments. We suggest that novel interaction techniques be further explored in order to develop efficient multi-modal approaches along with the widely used techniques.

Ambience is about the meaning of silence that cannot be expressed in words, and ambient interaction is about the perception of it. The present paper discusses three critical and fundamental aspects in the way ambient environments can be meaningfully designed in modern information society; 1) how design can merge that which is difficult to externalize together with that which is easy to externalize, and does so in a fulfilling way; 2) what is the most effective method for arriving at a mythological conclusion that could resolve the conflict between the many opposing forces at play?; and 3) what is “emptiness” based on eastern philosophy perspective, and how it can be applied to designing ambient environment? We shed light on how our thinking of design and information-based society should adapt moving forward by using universal thinking and human consciousness in a new, “primitive” coexistence with modern information technology.

This paper explores design strategies to use smells as a medium of interaction between the body and the space. In a living environment, encounters and experience of smells creates and manifests connections to the space. Along with the other sensory stimuli, smells communicates information about the space. Olfactory interactions can be expressed through its experiential relationship with the body. Design examples as discussed in this paper, express the aesthetics of interaction with smells that are dynamic and temporal. Human perception connects through the patterns and weave the fluid movements across the soft boundaries of smells. Over a range from discrete to ambient presence of smells, this paper discusses tangible and intangible interaction with smells in the design examples. Through active or passive actuation of smells, designing for smell diversity in a living environment would create interactions at many different levels and will add to experiencing spaces in a multisensorial way.

IoT development is complex. To reduce this complexity, IoT platforms provide a set of resources and functionalities to enable application development and support its execution. In this work, we present a human-centered approach for requirements elicitation and mapping them to application resources in IoT platforms, using empathy, definition and ideation methods. A previous study by the authors has identified 11 categories of resources provided by 47 IoT platforms to developers in their application layers. From this set, 6 categories were selected for this work: schedulers and triggers, message and notification triggers, big data and analytics, artificial intelligence and machine learning, dashboards, and services. We invited 18 members of 8 projects for a workshop and divided them in 4 teams, according their project areas, which are: Industry 4.0 (6 participants), Environmental Disasters (4 participants), Environmental Management (3 participants) and Pollution (5 participants). We divided the workshop in 3 phases: warm-up, with user journey mapping, requirements identification using “how might we” questions as a trigger and requirements clustering the questions by the 6 selected categories of resources or an extra category named “others” for those which could not be related to any previous category. Our contribution for the IoT application development is an approach for turning easier requirements elicitation using DT techniques, covering the stages of empathise, definition and ideation, with well-available materials and considering the resources present at application layer of IoT platforms.

Ambient User Experience design in architecture implies consideration of various intersecting and sometimes overlapping design fields such as interaction and architectural design with the aim to achieve a continuous and cohesive user experience across devices, time, and space. In this paper, Ambient User Experience design is explored in relation to Design-to-Robotic-Production and -Operation processes developed at TU Delft, which link computational design with robotic production and operation. Several case studies involving the integration of sensor-actuators into the built environment are discussed with respect to mapping activities through constraints and enablers and designing with the time as a variable.

Smart textiles have established a foothold in different academic fields, such as in chemistry, engineering, and in human-computer interaction (HCI). Within HCI, smart textiles are present in research in many ways, for example, as context, as means, or as focus. However, interdisciplinary projects tend to leave the implications of and to textile design without notice. How can a project utilise a textile designer’s skills to feed back to textile design from an interdisciplinary project? In this paper, we present a case study, where a textile designer’s role extends beyond the prototype production, and we analyse the project in light of textile design. Our findings show that textile design can augment data collection and analysis. We conclude with a discussion towards inclusion of textile design in HCI.

How can AI (Artificial Intelligence) and IoT (Internet of Things) technologies be combined to support positive interactions in ambient spaces? This was our overarching research question. In addressing this question, we worked in close collaboration with an IT-company and explored one IoT-based design and one AI-based design, and we examined ways of integrating the two designs in an interactive system for the particular use context of a parking lot for cars. In this use context we explored the design of an interactive system that would enable car drivers to easily find an available parking lot, while staying focused on driving rather than being focused on traditional man-machine interaction. With this model in mind we designed two prototype systems based on the idea of “implicit interaction”, and we considered how the use of AI (computer vision) and IoT (connected sensors) would enable a reliable solution that would allow the system to “see” available parking lots, and recommend these to the car driver – implicitly and automatically. In this paper we present our work on these two prototypes and our ideas on how to combine AI and IoT in this design. We also report from a preliminary user in order to answer a fundamental question here, i.e. –“How can we develop interaction models for such ambient spaces that adds to the quality of life for its users and inhabitants?”. We conclude this paper with our design recommendations for how to design for positive interactions by relying on implicit interaction models and the combination of AI and IoT technologies.

This paper explores conceptual design methods in the vehicle HMI design process. By adopting theory framework construction and case studies, we propose a mood board as the tool to integrate “software” and “hardware” in-vehicle HMI interaction design. The effectiveness and feasibility of this tool are supported by the industrial design undergraduate course, which is the transportation concept design. We were using the proposed mood board tool as an innovative approach assisting electric vehicle HMI design. Students of industrial design and interaction design got involved in the design process. We received positive feedback as to the effectiveness of the method with a satisfactory outcome overall.

Projection-based assitive systems that guide users through assembly work are on their way to industrial application. Previous research work investigated how people can be supported with such systems. However, there has been little work on the question on how to generate and author sequential instructions for assitive systems. In this paper, we present a new concept and a prototypical implementation of an assitive system that can be taught by demonstrating an assembly process. By using a combination of RGB and depth cameras, we can generate an assembly instruction of Lego Duplo bricks based on the demonstration of a user. This generated manual can later on be used for assisting other users in the assembly process. By our prototype system, we show the technological feasibility of assistive systems that can learn from users.

Recent years have seen an explosion in the academic and commercial applications of digital assistants. These technologies have become increasingly prolific, and their use has resulted in fairly rigid and standardized techniques for achieving a desired result. Whether this includes physical actions, or direct voice queries to the system via pre-defined wake words and queries, there is a stark boundary between the human and the system. We aim to explore a shift in the paradigm of these current implementations, to that of an Ambient Intelligent (AmI) environment in which users can interface with the system in a more natural, seamless, and multi-modal manner. Applications of this type of technology range from assisted living, to smart conference rooms and meeting spaces. In this paper we introduce an architectural framework for building an ambient intelligent platform using a combination of video and audio sensors to capture and process the data in a given area of interest.

The capabilities of object detection are well known, but many projects don’t use them, despite potential benefit. Even though the use of object detection algorithms is facilitated through frameworks and publications, a big issue is the creation of the necessary training data. To tackle this issue, this work shows the design and evaluation of a prototype, which allows users to create synthetic datasets for object detection in images. The prototype is evaluated using YOLOv3 as the underlying detector and shows that the generated datasets are equally good in quality as manually created data. This encourages a wide adoption of object detection algorithms in different areas, since image creation and labeling is often the most time consuming step.

The development, analysis, and refinement of modern Embedded Systems (ES) and Internet of Things (IoT) applications require the use of complex models. These, in turn, require a variety of modeling techniques that can be used for different aspects of a given ES or IoT project. ES and IoT projects nowadays are well beyond pencil-and-paper development and analysis, so sophisticated tools with well-designed human-computer interfaces are required. Such tools do exist for various modeling techniques individually – for example, tools for Petri Net modeling, tools for UML modeling, etc. Some tools are able to translate a model in one methodology to a second model or to a programming language. However, no tool exists that integrates the major modeling methods used in ES and IoT into a single package that allows designers and engineers to explore through a single interface all the aspects of the project. This makes it difficult for the designers and engineers to see how operations, or even changes, modeled in one methodology impact models in the other methodologies. Designers and engineers can’t see the whole picture within a single tool. We argue that such a tool is needed for ES and IoT and propose a set of requirements that such an integrated tool must satisfy in order to be truly useful in the design and analysis of ES and IoT projects.

“Smart” buildings that can sense and detect people’s presence have been in use for the past few decades, mostly using technologies that trigger reactive responses such as turning on/off heating/ventilating, lighting, security, etc. We argue that to be considered truly smart, buildings must become “aware” about the locations and activities of their inhabitants so they can proactively engage with the occupants and inform their decision making with respect to which actions to execute, by whom and where.To help assess the potential impact of “aware” buildings on their occupants, we are developing a multi-agent simulation-powered building management system that can sense human and building assets, extrapolate patterns of utilization, simulate what-if scenarios and suggest changes to user activities and resource allocation to maximize specific Key Performance Indicators (KPIs). The system is able to evaluate the implications of potential conflict resolution strategies and account for individual and collaborative activities of different types of users in semantically rich environments.Sensing in our case is based on Visible Light Communication (VLC) technology, embedded in a building’s LED lighting system. It can detect the actors, where they are located and what they do. To understand what happens in each space at any given time the information derived from the VLC system is combined with models of users’ activity schedules, profiles, and space affordances.We demonstrate our approach by hypothetically applying it to a Cardiac Catheterization Laboratory (CCL). The CCL is high-intensity hospital unit, second only to the Emergency Department in terms of the urgency of the cases it must handle. An aware building will help both patients and staff to allocate their (always scarce) resources more efficiently, saving time and alleviating stress.

The latest smartphones have advanced sensors that allow us to recognize human and environmental contexts. They operate primarily on Android and iOS, and can be used as sensing platforms for research in various fields owing to their ubiquity in society. Mobile sensing frameworks help to manage these sensors easily. However, Android and iOS are constructed following different policies, requiring developers and researchers to consider framework differences during research planning, application development, and data collection phases to ensure sustainable data collection. In particular, iOS imposes strict regulations on background data collection and application distribution. In this study, we design, implement, and evaluate a mobile sensing framework for iOS, namely AWARE-iOS, which is an iOS version of the AWARE Framework. Our performance evaluations and case studies measured over a duration of 288 h on four types of devices, show the risks of continuous data collection in the background and explore optimal practical sensor settings for improved data collection. Based on these results, we develop guidelines for sustainable data collection on iOS.

The concept of artificial intelligence (AI) has become increasingly prevalent in the industry, but there is still insufficient understanding about what AI can exactly do for manufacturing companies. This paper focuses on the domain of product research and development (R&D), and aims to depict how AI can assist in industrial R&D activities. Through a comprehensive review of literature, this paper identified three major drawbacks in traditional product R&D approach, namely, low success rate, long research cycle, and difficulty in management. Subsequently, based on the characteristics of AI technology, this paper proposes and discusses a number of scenarios to demonstrate how AI can be applied to support R&D activities. Comparing with traditional product R&D, the advantages of AI-based R&D are proposed and summarized: 1) More objective identification of user requirements to drive enterprise innovation; 2) more precise exploration of market trends; 3) higher efficiency in product design; 4) less risks in R&D process; and 5) improved knowledge sharing ability. This paper will be of interests and value to practitioners and researchers concerned with AI usage in manufacturing contexts.

Projection-based assistive systems have shown to be a promising technology to support workers during manual assembly processes in industrial manufacturing by projecting instructions into the working area. While existing studies have investigated various aspects of these systems, little research has been conducted regarding the way in which the user accesses the provided instructions. In this paper we analyze the eye movements of users during the repeated execution of an assembly task at a projection-based assistive system in order to gain insights into the utilization of the presented instructions. For this purpose, we analyzed eye tracking recordings from a user study with 15 participants to investigate the sequences in which the respective instructions are observed by the users. The results show a significantly lower number of nonlinear gaze sequences as well as a significantly higher number of steps without observing the instructions during the repeated use of the assistive system. In addition, there was a significantly lower task completion time during repeated use of the assistive system.

Artificial Intelligence (AI) is an integral part of our lives with AI systems to revolutionise our daily practices. At the same time, the rapid pace of AI innovations entails inherent risks that can range from cyber-crime to social discrimination. Here, we administered a large scale survey ( $$n=1298$$ ) assessing peoples’ concerns and expectations regarding AI’s influence on society in the future decade. The AI concerns employed in this study, originate from the “One hundred year study on Artificial Intelligence” project. Taking Norway as a case study, we discuss the participants’ prioritisation of concerns for their socio-demographic characteristics. Our findings show a divide in the society; with younger generations to expect a positive impact of AI on our lives in the future decade. More sceptical groups are afraid of structural changes in the economy and job losses, while supporters see opportunities that will improve our life quality. These findings can inform both academics and policymakers that should work closely to ensure fairness, explainability and maintain a trusting relationship between AI and society.

User-centered physical-digital systems let designers create interactive interfaces filled with special moments and experiences, giving brand owners the chance to have profound communication with their consumers. In fact, product’s packaging has recently begun to investigate as one of such interfaces to form a strong link between manufacturers and their end-users. Microprocessors, sensors, actuators and wireless data-exchange supporting chips can be embedded, into packaging design creating an extended user interface – a touchpoint for a visual, tactile and digital encounter with consumers. Near Field Communication (NFC) is one of the rapidly increasing technologies that researchers begin to investigate as a potential tool for enhanced consumer-brand communication. However, although NFC is available in the market since late 2010, the technology is still not widely applied to the packaging industry. As a result, the main purpose of this research is to investigate the current state-of-the-art and potentials of NFC system. The results of this study provide a systematic review of NFC characteristics, including technological capabilities, consumer- and brand-oriented benefits, and technology- and user-centered potential barriers for NFC to become widely accepted. The findings of this study are expected to contribute to a better understanding of the effectiveness of NFC-enabled packaging, allowing brands to dynamically adapt to emerging consumer needs by improving their products and services.

Cyber-physical systems refer to environments that are sensitive and responsive to people, where users’ activities in a physical environment are enhanced by digitized services, thus calling for an Ambient UX design approach. Designing for experiences in such complex systems implies facing UX investigation in a holistic manner. The present work encompasses an initial overview of the methods employed to investigate systematically different facets of UX by actively involving users and based on experiments involving biometrics monitoring and other solutions to collect data. Investigation approaches are conceived as belonging to four layers of analysis, namely (i) physiological, (ii) behavioural, (iii) self-reported, and (iv) expert evaluation. The major contribution of the current research lies in the methodological integration firstly adopting a theoretical stance for UX investigation through multimodal quadrangulation, and secondly in a discussion on the applications of the approach as performed in a multidisciplinary research laboratory.

From smart homes to highly energy-optimized office building and smart city, the adoption of living in smart spaces requires that the inhabitants feel comfortable with the level of data being collected about them in order to provide smartness. However, you usually provide this consent on—or best before—your very first interaction. Thus, firstly your consent might vary over the time of usage. Secondly, it is not always obvious if data is currently collected or not. This paper addresses two missing elements in the interaction with a smart environment: First, the general concept of dynamicity of consent to data collection. Second, provision of a physical interaction to gather and change consent and a physical feedback on the current data collection status. By the feedback being physical we mean being visual, haptic or accoustic, in order to allow natural perception by the users in the physical space. For both components we provide examples which show how one could make both the current status as well as the consent physical and discuss the user perception. We argue that having a physical interaction to start potentially privacy-invasive data collections is a useful enrichment for legal consent, and physically visible status is helpful to make a decision.

Artificial Intelligence is increasingly integrating into everyday life and is becoming an increasingly pervasive reality. Domestic AI-enhanced devices are aggressively conquering new markets, nevertheless such products seem to respond to the taste for novelty rather than having a significant utility for the user, remaining confined to the dimension of the gadget or toy. Interestingly, although AI has been indicated as a new material for designers, the design discipline has not yet fully tackled this issue. Moving from these premises, the MeEt-AI research program aims at developing a new UX assessment method specifically addressed to AI-enhanced domestic devices and environments. Accordingly, we frame the project within the vast and variegated field of UX assessment methods, focusing on three main aspects of UX assessment – methodology, UX dimensions and analyzed objects – by looking at what current methods propose from the standpoint of AI-enhanced domestic products and environments. What emerges are general considerations that are at the basis of the positioning of the MeEt-AI research program.

Jeremy Bentham is known for designing an institutional building, a prison named a panopticon and some alternatives to this concept. One of the alternatives are the inverted or constitutional panopticon in which the purpose is to let the governed, the citizens’ see and monitor the governors. Hence, the concept of inverted panopticon can be used to describe an analyze privacy protecting devices. In this paper we report on a national study on citizens’ opinion and attitudes to devices that can protect the user from being seen and listened to, with 1289 participants. At this stage of the work, we have not done statistical analysis of factors that might reveal differences between citizens, but as an exploratory study, we provide an overview of how the two privacy protecting devices were received by Norwegian citizens, based on survey responses. Our aim is to build a foundation for future studies that will investigate the inverted panopticon concept in a society in which personal data has become a currency.

In a world full of sensors that gather personal data and digital solutions that use these data to provide feedback and personalized experiences, biofeedback is increasingly involved in the definition of new paradigms for tailoring interactions. Companies are collecting and using personal data to propose personalized services. Content providers are pushing users to produce data in order to create personalized storytelling experiences. In this context, the tech market is offering new low-cost solutions able to gather biodata. The paper reports the results of evidence-based explorations aimed at formalizing knowledge regarding the use of passive and unconscious interaction to control the fruition of storytelling artifacts. We investigate a new interaction paradigm that promise to seamlessly enable unconscious and enactive interactions for movie experiences. We propose the use of emotion recognition and eye-tracking as exploratory technologies that promise to be a potential contribution to richer access to the spectators’ emotional involvement. We reflect on disruptive power of non-invasive technologies, given by the possibility to be used for home-cinema experiences. Investigating on emotional states of users in their decision we leverage on the emotive-cognitive data as a matter of creation and enabling of tailored movie experiences. Our research intends to explore the possibility of extracting knowledge from recognition of facial expressions that will contribute to foster its use in real-time passive interaction using emotion recognition as a trigger of enactivity that is not limited to interactive storytelling but opens new scenarios in the design of proactive systems for screens, spaces and environments. Furthermore, we provide suggestions as guidelines for the design of enactive experiences that leverage on emotion recognition and eye-tracking.

The development of open data by local governments and data platforms for each field is progressing. These are broad ranged data on each area, such as traffic, disaster prevention, restaurants and services, and are expected to be useful information sources for citizens and tourists. On the other hand, these data are usually deployed in a network reachable place, but when they have to be handled individually according to its own format, and in some cases, conversion both in format and in semantics are required, which is a barrier to use.By the way, in existing information services for tourists, especially smartphones application services for tourists, the content provided are selective and limited in some specific fields and target areas covered. That is, there is a problem on coverage in content. In addition, there are many cases where has a problem with the cost of maintaining and updating content, and the content may often be obsolete.Given the existence of data platforms developed for each field, the paper presents the data linkage challenges that enable them to be integrated and used. And also the paper shows a one-stop smart city application that has been developed using that function. This application provides information for tourists during normal times, and it can handle the situation in town, such as congestion, in real time in order to grasp the flow and local stagnation that occur in bursts at the time of events or accidents.

Taking Jiuzhaigou Scenic Area as an example, on the basis of combing the literature and combining the characteristics of the intelligent scenic area, a smart landscape composed of 14 indicators.District service quality evaluation system, through the questionnaire survey method to obtain tourist evaluation data on these indicators. Based on the IPA analysis method, SPSS 21.0 was used to analyze the importance-satisfaction difference analysis of the sample data. Evaluation indicators are classified into advantage area, maintenance area, improvement area and opportunity area, and put forward suggestions to improve wireless network coverage, strengthen the construction of a personalized service intelligence system, and strengthen the connection between virtual networks and physical landscapes.

Information technologies have dramatically changed our real world. There are two major concepts have played a role to change the real world. Through the computation concept, our various tasks can be automated. Also, the informatization concept makes us possible to quantify our daily activities. These concepts offer infinite possibilities for our future. However, another concept named the digital enhancement through virtualization concept has not been well discussed previously for changing the real world, but the concept has strong power to refine our real world. In this paper, we use the concept to virtualize collective human eyesight capabilities. We have developed a prototype implementation of the distributed platform to virtualize collective human eyesight and hearing capabilities and extracted some insights from the current platform. The paper concludes by exploring several future challenges to enhancing our current approach.

The important role of citizens towards smart city success has been increasingly recognized by police makers, practitioners and academics. In light of this, smart apps are probably the most appealing smart city element to citizens, who are using these on a daily basis. Nevertheless, little was known about citizens attitudes and perceptions regarding the usage of smart city apps. In this paper, we reported the results derived from a questionnaire survey with 577 citizens in Guangzhou, China. The study investigated their use experience of smart city apps, in order to identify potential shortcomings of these apps and provide reference for their future optimization. The results show that Chinese citizens have high intention to use smart city apps, but they also have concerns about app service responsiveness, information accuracy, system reliability, perceived cost, and perceived risk.

This work addresses the grand challenge for human-computer interaction of learning and creativity, in the context of smart cities. Through a review of the research literature for learning in relation to attuning and then for creativity in relation to adaptability, a theoretical perspective and conceptual framework is developed for learning and creativity in smart cities. Using an exploratory case study approach combined with an explanatory correlational design, relationships for learning and creativity in urban environments are explored using survey data and in-depth interviews. Additionally, emotion/affect is explored in relation to learning and creativity in terms of the experience of comfort in urban environments. This work is significant in that it provides a hybrid approach to understanding the grand challenge of learning and creativity through the use of both quantitative and qualitative data and analysis. This paper contributes to the research literature for smart cities, learning cities, and creativity in smart environments; develops and operationalizes a conceptual framework for learning and creativity in smart cities; and points to the importance and potential for learning and creativity relationships in smart, urban environments.

In smart cities, citizens’ lives and data will be increasingly intertwined with the systems used by local, state, and federal governments. Given such a context, this paper focuses on the role that automated decision(-making) systems (ADS) play—and could play—within smart cities and unpacks the challenges of stakeholder participation in determining this role. To address future scenarios and propose a provisional framework for participating in and with ADS-laden cities and government, we begin with the case of New York City’s Automated Decision Systems Task Force as a concrete example of the challenges of regulating, governing, and participating with ADS. As a single example, we explore the particularities surrounding the Task Force, including the mobility of its policy, the difficulty of defining the issue, and finally the proposed framework for overseeing ADS in New York City. We introduce two practices of participating in city-making: participatory governance and participatory design. As we explain, these practices emphasize the role of stakeholders—e.g. citizens, bureaucrats, private industry actors, and community groups—in crafting the policy and technology that they use or are impacted by. We then propose a provisional framework for ADS transparency and accountability based on these participative practices. We recommend enabling citizens to directly help regulate and design automated decision-making systems with the caveats that doing so can be messy, contextual, and frictional. In sum, this paper advocates that all communities should be empowered to understand and impact the systems determine their destiny.

We present a new model for encouraging people to get involved with monitoring and taking part in the life of cities. Cities could be smarter if IoT and people could serve as engaged and pro-active data resources (i.e., crowd sensing). This study tackles two challenges: methods by which the privacy of people who act as sensors/actuators can be guaranteed and methods to create a unified programming model for crowd sensors alongside other IoT functions. To achieve these goals, we introduce a new concept called Lokemon (Location Monster). Each sensing space is characterized as a personified target. Lokemon asks users to imagine themselves to be monsters associated with target spots when achieving sensing tasks. Lokemon is also expressed as a PubSub node so that the data from Lokemon can be easily accessed in the same way as data from IoT is assessed. The article explains the concept of Lokemon and its programming model. We report our evaluation of the effectiveness of Lokemon in a campus experiment that was performed for four weeks.

Smart city services and applications have gradually come into the life of citizens. Citizens’ use of smart city services is necessary for the sustainable development of such services. However, the existing literature has a large difference in research conclusions on the relationship between smart city service user behaviors and their influencing factors. In this paper, the meta-analysis method was used to review 54 independent samples in 52 studies, the related effects of 33 influencing factors were analyzed. The meta-analysis results show that 31 influencing factors have significant effects on the adoption, use, and continuance behaviors of smart service users. In addition, sample groups have a moderating effect on the relationship between user behaviors and variables such as social influence. This paper has clarified the effects of influencing factors and one moderator, which has a certain reference value for improving the use intention and participation of users of smart services and applications.

There have been few products or research efforts on designing for people with intellectual disabilities. The caretaking companies have no enough caretakers to keep an eye on the clients with intellectual disabilities, who are suffering from circadian rhythm disorder. We report on the design of a system to detect the sleeping behavior of people with intellectual disabilities in the daytime. The system utilizes an inexpensive 2D camera with computer vision techniques to track whether a user is falling asleep. The system wakes up the clients gradually in several steps when a long-time sleeping is detected. Our study explores taking eye aspect ratio, head orientation as inputs, and a support vector machine to managing complex situations in real life.

The rapid aging of developed societies is creating an increasing number of cases of dementia and other aging-related diseases. Physical exercise has been shown to be beneficial for retaining cognitive, as well as physical function. In this paper we describe a system that we have developed for motivating older people to exercise. We begin by demonstrating how activity declines with age using an analysis of survey data (from 2003–2015) reported by the U.S. Bureau of Labor and Statistics on American Time Use Usage. We use multiple discriminant analysis to characterize which activities tend to become more, or less frequent as people age. We then review previous exergame approaches for motivating physical activity. After this we discuss a pedaling system that is designed to motivate and facilitate physical activity in the elderly using a combination of competition, social interaction, and engaging video content. We conclude with a discussion of how to adapt exergaming innovations to the context of use.

Sustainable interface design has been receiving a great deal of attention in efforts towards a sustainable society. The Tojo-Tei Historic House exhibition at Matsudo International Science and Art Festival 2018 focused on a new type of interface with the aim of raising mythological awareness in the midst of a modern city. A tele echo tube (TET) is a speaking tube interface that interacts acoustically with a deep mountain echo via a slightly vibrating lampshade-like interface. TET allows users to interact with a mountain echo in real time, through an augmented echo-sounding experience with vibration, over a satellite data network. Since 2009, we have continuously improved the architecture of this system to increase the interaction performance and improve the experience. In order to evaluate the interaction performance, an experimental system was constructed to study general rather than laboratory use. In this paper, we discuss the basis requirements for the system for the Tojo-Tei Historic House exhibition at the Matsudo International Science Art Festival 2018.

This paper discusses how smart learning in urban environments can mediate citizen digital skills and competences learning initiatives supported by standards such as the European Commission citizen Digital Competences Framework [8], thus helping a broad range of urban populations to gain essential knowledge and skills for navigating the digitised services of the societal urban systems around them. Smart learning, based on cultural, civic or community interests and placed within a context of ad-hoc urban learning experiences set in authentic learning hyper-local environments might support and develop citizen digital literacies and competences through a wide variety of informal learning activities. This kind of technologically mediated learning acts as an implicit conduit to channel the development of a number of skills and literacies involved in the use of digital apps and devices, and the manipulation of knowledge content both digitally created as well as consumed. Additionally, development of ‘soft’ skills such as community participation, confidence building and language literacy are increased in digitally connected spheres, enabling citizens to act with more self assured agency within these territories. The paper refers to the author’s own doctoral research findings developed from a phenomenographic investigation into smart learning journeys to suggest a ‘pedagogy of experience complexity’ for smart learning as support for these kinds of learning activities.

In this paper, we describe about a kind of data flow design that between ubiquitous learning log system called SCROLL and learning analytics and visualizing system called Learning Analytics Dashboard (LAD). SCROLL is a ubiquitous learning system what is logging students’ learning behaviors data in database, and SCROLL can provide students suitable learning method and location to learn efficiently. Lots of paper show that it is appreciate to share the learning data in SCROLL to the other learning analytics system like LTI, Bookroll, Moodle and so on. Learning Analytics Dashboard (LAD) is also a learning data analytics and visualizing system. So share students’ learning data from SCROLL to LAD to show and help students to know their students’ learning situation is the proposal of this paper.

Changsha Kiln, world-renowned for its rich under-glazed porcelain, is a famous export porcelain kiln in the Tang Dynasty, began to thrive during the middle and late periods of the Tang Dynasty (618–907 AD), but then declined during the Five Dynasty periods (907–960 AD). Since the beginning of the new century, with the country’s increasing emphasis on the protection and innovation of cultural heritage, Changsha Kiln has gradually moved towards revival. However, the lack of ceramic products design talents, difficult to blend traditional and modern styles, and lack of support for creative design are obstacles to revival. To this end, this paper proposes an open creative design platform for cultural relics re-creation in Changsha kiln. Three basic components built based on deep learning technology in this platform: cultural relics knowledge base, cultural relic image feature database, and search engine based on semantics and images. With this platform, provide cultural relics element retrieval and creative design services for the general public, cultural creative designer and SMEs, which can promote the integrated development of culture and technology, and promote the cultural industry to become a pillar industry of the national economy.

The existing researches and developments of dashboard visualizing results from learning analytics mainly serve the instructors instead of learners in a direct manner. Effective visualizations extracted from learning log data can help the students to reflect and compare studying activities and access their metacognition to improve their self-regulated learning. For such purposes, we designed a reading path graph for visualizing the studying activities on slide pages used as teaching materials in classes intuitively, as one of the key functions of the learning dashboard. By providing the comparisons between the user’s own situation and the class overview, the visualization is expected to motivate the further actions of using other tools of the learning dashboard and reflecting studies. This paper introduces our exploration of the data process flows of extracting necessary data from a large number of operational logs for the visualization, and the techniques and strategies applied for rendering the graphics effectively. We implemented the data processing module with Python3 and the web-based visualization module of the reading path graph with JavaScript based on D3.js considering the extensibilities. The issues engaged in the development of prototypes are discussed, which will lead to the improvement of future prototypes and better designs of user experiments for formative evaluations as the next step of this research.

Recently, various educational data such as clickstream data and eye movement data have been collected from students using e-learning systems. Learning analytics-based approaches also have been proposed such as student performance prediction and a monitoring system of student learning behaviors for supporting teachers. In this paper, we introduce our recent work as instances of the use of clickstream data and eye movement data. In our work, the clickstream data is used for representing student learning behaviors, and the eye movement data is used for estimating page areas where the student found difficulty. Besides, we discuss advantages and disadvantages depending on the types of educational data. To discuss them, we investigate a combination of highlights added on pages by students and eye movement data in page difficulty estimation. In the investigation, we evaluate the similarity between positions of highlights and page areas where the student found difficulty generated from eye movements. It is shown that areas in the difficult pages correspond to the highlights in this evaluation. Finally, we discuss how to combine the highlights and eye movement data.

A visit to a place of natural beauty is known to have restorative potential. Immersing oneself in nature - relaxing, contemplating, meditating, walking and so on – can help improve one’s mental and physical wellbeing. Suitably designed VR can encourage beneficial meditative states as well as healthy physical activities. We see fully immersive forms of VR as a form of “synthetic consciousness” that is a modern addition to the three clearly established classic states of consciousness: wakefulness, dreamless, and rapid-eye-movement (REM) sleep. Certain therapeutic and self-care mental health therapies such as mindfulness-based stress reduction (MBSR) meditation can assist individuals to achieve relative peace of mind. We describe the development of aesthetically-appealing VR programs that were designed to induce mental states of equanimity, hopefulness and child-like wonder, referring back to historical aspects of art and design. We also report work where VR was used to embed actions required for exercise within a meaningful experience with the exercise itself, and the associated effort, becomes secondary in the mind of the participant to the flow, and narrative logic, of the interaction being performed. To increase the “stickiness” or attractiveness of our VR approach to exercise we also introduce the idea of rewards for exercise carried out correctly. User groups of “healthy normal” adults, mental health patients with clinically significant anxiety, and frail elderly at risk of institutionalization have provided helpful and generally positive feedback.

This paper introduces three computer-assisted applications designed for learning foreign vocabulary in an informal setting. The first one, images recommendation application, generates appropriate image recommendations for representing a word. It tackles the challenge for a foreign language learner to determine appropriate images from a standard web search engine such as Google, Yahoo, Flicker, etc. The second application, learning context representation application, generates learning contexts automatically from lifelogging images. It addresses problems associated with describing a learning context in the forms of hand-written descriptions, keeping notes, or taking memos. The third application we discuss here, namely location-based associated word recommendation application, generates recommendations of associated words in a particular learning location by analyzing word learning histories. It seeks to answer a critical question: what I should learn next? This is a critical challenge for the users of ubiquitous learning tools. In order to recommend potential vocabularies which a learner could be learning in a particular location, this study recommends associated words and topic-specific vocabularies. These applications are for AIVAS (Appropriate Image-based Vocabulary Learning System), a platform for computer-assisted vocabulary learning. We report here several evaluations, including human assessment and data-driven assessments, that have been carried out to reveal the importance of these systems.

In this research, through a project named Circuit Game, a conceptual framework has been developed by which new processes of integrating craft and technology for educating electrical concepts are explored. The framework incorporates Do It Yourself (DIY) methods through making and assembly procedures. The intention is to engage architecture students in the process of crafting a new technological artifact while understanding difficult concepts related to electronics. Buechley & Hill define HCI as the development of new interfaces for interaction between people and technology (Buechley and Hill 2010). The authors seek to discover how a composite textile interface can invite people into thinking about electronics, materials, and design processes and eventually engage them in the process of designing circuits through crafting. Base materials, DIY making, and DIY assembly processes are three interwoven components of the circuit game. The role of these elements in the learning process are investigated through the development of a circuit game.

This paper conducts literature research from the fields of art research, digital protection, and productive protection. We build an intangible cultural heritage knowledge platform, which aims at the protection of Taoyuan woodcarving craft. Furthermore, it conducts interactive design research and practice of the system. The purpose of this study is to establish a knowledge platform, to speed up the industrial process of Taoyuan woodcarving craft by sharing the database, and to improve the innovative design and development system of Taoyuan woodcarving. By extracting basic knowledge, modeling knowledge, functional knowledge, process knowledge, and aesthetic knowledge, a case specification and data model are established to create a rapid design channel. The goal is to shorten the practice cycle of design innovation to help the economic and design value conversion of Taoyuan woodcarving craft intangible cultural heritage.

In this paper, we describe the support system for job-hunting students to learn job-hunting related terms by using an eBook and a chat system. Job-hunting process is very unique and complicated in Japan. Job-hunting students face difficulties in many phases. Many job-hunting related terms are not used in daily conversation and very new to them. Therefore, it is necessary to support them to learn them. The objectives of this study are (1) to clarify whether there is a positive correlation between active involvement of eBook learning and test performance and (2) to clarify whether the use of our chat system was effective in learning job-hunting related terms. The result of the evaluation showed that as for (1), there was no statistically significant correlation between the two factors. However when focusing on the international students, the correlation coefficient increased. It was aligned with the fact that this system was originally designed to support international students. As for (2), when the comparison was made between with and without InCircle delivery, there was no statistically significant difference between them in terms of Pre- and Post-test improvement. However the students gave the high score when they were asked its helpfulness. The score implies that the students were satisfied with its helpfulness.

With the accelerated pace of life in modern society, various sleep problems have become increasingly prominent and spread to younger groups. Meanwhile, people are beginning to accept the use of IoT products to improve their sleep quality. This paper designs a non-invasive sleep assistance system based on IoT that integrates a non-invasive device of collecting sleep data, a sleep-assisted light linkage and a mobile application. Before the design, we conduct questionnaires and in-depth interviews to understand user characteristics and related demands, construct personas, and conduct competitive product analysis to determine the specific product form. The sleep data collection device is a circular mat based on the Arduino platform and the EMFi sensor, and the light linkage is a sleep-assisted device coordinated with the circular mat. The mobile application can display some information vividly, such as the user’s physiological data, sleep quality, sleep suggestions, and so on. We complete the functional prototype of the data collection device and the light linkage, and the high-fidelity prototype of the mobile application. In addition, we collect feedbacks and discuss the improvement direction of the sleep assistance system in the future.

Geography is a broad subject, and maps are essential tools for learning geography. It is necessary to educate children about geography and maps at an early age. Digital educational tools play an increasingly important role in children’s education. However, few studies focus on digital map design for children conforming to their cognitive abilities. Inspired by that, we propose PuzMap, a novel puzzle map, which facilitated with various sensory channels, visual, auditory, tactual, and olfactory. It can create specific experiences when it is manipulated. PuzMap delivers knowledge of geographical distribution to children. It also introduces the concept of ocean current and its movement principle through the multi-sensory approach, which is rarely mentioned before. When finishing the puzzle map, children can scan the pattern on each distribution module to enjoy the AR patterns about this area. In this paper, we detail the system design of PuzMap and present the design principles of children map. We also conducted studies to explore our system’s usability. The results showed that PuzMap could help children learn geography knowledge, and it is engaged for children to play.

During childhood, play is important for promoting the mental and physical development of children. For this reason, those involved with childcare (guardians and childcare workers) need to create an environment suitable for child development and provide children with support and guidance. However, because there are unique elements to the development of each individual child, childcare workers caring for large numbers of children, and guardians in remote areas, may struggle to oversee the daily development of the children in their care. In this paper, we propose toys with built-in sensors are used to acquire motion data during play activities, and a system intended to aid in estimating the child development stage by creating a data visualization for childcare workers. To establish the proposed system, a ball-type device was created using a built-in acceleration sensor as a prototype of a toy with a built-in sensor. Using this prototype, we conducted an experiment to verify whether it is possible to discern five types of ball activities when focusing on changes in ball-throwing movements related to child development. As the results based on two types of learning algorithms (SVM and RF) indicate, in each case the activity could be identified with approximately 70% accuracy.