Case-Based Reasoning Research and Development

Frontmatter

Searching Museum Routes Using CBR

Abstract

In this paper, we describe a CBR solution to the route planning problem for groups of people. We have compared keyword coverage results for our CBR approach and heuristic search algorithms. User preferences are important for individual visits but when dealing with group visits there are other aspects to consider. In our case study a group of people plans a visit to MIGS (Museo de Informática Garcia Santesmases http://​www.​fdi.​ucm.​es/​migs/​), a museum about computer science history located at the Computer Science Faculty of Complutense University in Madrid. CBR results are promising and we discuss the benefits of the experience in the case base when planning a group visit. CBR has become specially appropriate given that it assists the knowledge discovery task when learning about subtle differences affecting the suitability of group plans over individual plans computed by traditional search algorithms.

Jesús Aguirre-Pemán, Belén Díaz-Agudo, Guillermo Jimenez-Diaz

Comparative Evaluation of Rule-Based and Case-Based Retrieval Coordination for Search of Architectural Building Designs

Abstract

To support the early conceptualization phase in architecture with computer-aided solutions, in particular, with retrieval systems that can find similarly structured building designs in comprehensive collections of such designs, a number of approaches were presented to date. In the Metis project two retrieval coordination approaches (coordinators) were developed to govern the search of similar (sub-)structures of architectural designs. The main task of both coordinators is to select the retrieval method that is appropriate for the given user query. First approach is a standalone service that uses rules only to coordinate the retrieval and can use subgraph matching and database search methods, whereas the second one is rule- and case-based and is part of a distributed system for case-based retrieval of architectural designs. We compared both coordinators in a user study to find out which strengths and weaknesses both coordinators currently possess, and for which retrieval scenarios of the architectural conceptualization phase they could be appropriate. The results showed that the complexity of the particular scenario and the purpose of search are the main points that differentiate both coordinators. The rule-based coordinator performed better when a search for exact (sub-)structures was required, whereas the rule- and case-based coordinator is appropriate for queries aimed to be used for exploration and general search for inspiration. Visualization of the results of both coordinators is in need of improvement.

Viktor Ayzenshtadt, Christoph Langenhan, Johannes Roith, Saqib Bukhari, Klaus-Dieter Althoff, Frank Petzold, Andreas Dengel

Case Representation and Similarity Assessment in the selfBACK Decision Support System

Abstract

In this paper we will introduce the selfBACK decision support system that facilitates, improves and reinforces self-management of non-specific low back pain. The selfBACK system is a predictive case-based reasoning system for personalizing recommendations in order to provide relief for patients with non-specific low back pain and increase their physical functionality over time. We present how case-based reasoning is used for capturing experiences from temporal patient data, and evaluate how to carry out a similarity-based retrieval in order to find the best advice for patients. Specifically, we will show how heterogeneous data received at various frequencies can be captured in cases and used for personalized advice.

Kerstin Bach, Tomasz Szczepanski, Agnar Aamodt, Odd Erik Gundersen, Paul Jarle Mork

Accessibility-Driven Cooking System

Abstract

Research area of designing recipes is an attractive problem for the CBR community. In this paper we deal with the problem of presenting the recipe information in an understandable format for a certain user. As different users have different presentation needs, we discuss the suitability of taking the user profile into account to personalize the presentation of a suggested recipe in a cooking system. Our system relies on text simplification processes that were born from the need of people who have difficulties reading and understanding textual contents. Our system collects a case base with the best choice of presentation for a certain collective. Given a recipe plus details on the user profile (age, genre, educational level, languages, disability and special needs) the system retrieves from a case base the best presentation and modify the recipe presentation according to the specific user needs. The system includes learning capacity as if the final presentation is difficult for the specific user the system can easily provide her with alternate presentations. Results on the preliminary experiments are very promising and show the applicability of a CBR approach to personalize and simplify textual recipe presentations for different collectives.

Susana Bautista, Belén Díaz-Agudo

Inferring Users’ Critiquing Feedback on Recommendations from Eye Movements

Abstract

In recommender systems, critiquing has been popularly applied as an effective approach to obtaining users’ feedback on recommended products. In order to reduce users’ efforts of creating critiquing criteria on their own, some systems have aimed at suggesting critiques for users to choose. How to accurately match system-suggested critiques to users’ intended feedback hence becomes a challenging issue. In this paper, we particularly take into account users’ eye movements on recommendations to infer their critiquing feedback. Based on a collection of real users’ eye-gaze data, we have demonstrated the approach’s feasibility of implicitly deriving users’ critiquing criteria. It hence indicates a promising direction of using eye-tracking technique to improve existing critique suggestion methods.

Li Chen, Feng Wang, Wen Wu

Eager to be Lazy: Towards a Complexity-guided Textual Case-Based Reasoning System

Abstract

Finding an ideal representation for a case-base is important for a CBR system. This choice of an ideal representation is guided by the complexity of the cases. Based on the needs of each individual case, richer features are used for representation if required. While the framework is fairly general, this paper demonstrates its effectiveness on text classification due to the ease of evaluation. Each test case is treated differently by the classifier, in that if a shallow representation is deemed adequate for assigning a class label, the algorithm does away with a richer representation which is computationally expensive to generate. We also provided a time-budgeted evaluation of our framework which suggests that it holds promise in minimizing redundant or misleading comparisons and minimize time without compromising on effectiveness.

K. V. S. Dileep, Sutanu Chakraborti

Personalized Opinion-Based Recommendation

Abstract

E-commerce recommender systems seek out matches between customers and items in order to help customers discover more relevant and satisfying products and to increase the conversion rate of browsers to buyers. To do this, a recommender system must learn about the likes and dislikes of customers/users as well as the advantages and disadvantages (pros and cons) of products. Recently, the explosion of user-generated content, especially customer reviews, and other forms of opinionated expression, has provided a new source of user and product insights. The interests of a user can be mined from the reviews that they write and the pros and cons of products can be mined from the reviews written about them. In this paper, we build on recent work in this area to generate user and product profiles from user-generated reviews. We further describe how this information can be used in various recommendation tasks to suggest high-quality and relevant items to users based on either an explicit query or their profile. We evaluate these ideas using a large dataset of TripAdvisor reviews. The results show the benefits of combining sentiment and similarity in both query-based and user-based recommendation scenarios, and also disclose the effect of the number of reviews written by a user on recommendation performance.

Ruihai Dong, Barry Smyth

Concept Discovery and Argument Bundles in the Experience Web

Abstract

In this paper we focus on a particular interesting web user-generated content: people’s experiences. We extend our previous work on aspect extraction and sentiment analysis and propose a novel approach to create a vocabulary of basic level concepts with the appropriate granularity to characterize a set of products. This concept vocabulary is created by analyzing the usage of the aspects over a set of reviews, and allows us to find those features with a clear positive and negative polarity to create the bundles of arguments. The argument bundles allow us to define a concept-wise satisfaction degree of a user query over a set of bundles using the notion of fuzzy implication, allowing the reuse experiences of other people to the needs a specific user.

Xavier Ferrer, Enric Plaza

Incorporating Transparency During Trust-Guided Behavior Adaptation

Abstract

An important consideration in human-robot teams is ensuring that the robot is trusted by its teammates. Without adequate trust, the robot may be underutilized or disused, potentially exposing human teammates to dangerous situations. We have previously investigated an agent that can assess its own trustworthiness and adapt its behavior accordingly. In this paper we extend our work by adding a transparency layer that allows the agent to explain why it adapted its behavior. The agent uses explanations based on explicit feedback received from an operator. This allows it to provide simple, concise, and understandable explanations. We evaluate our system on scenarios from a simulated robotics domain by demonstrating that the agent can provide explanations that closely align with an operator’s feedback.

Michael W. Floyd, David W. Aha

Inferring Student Coding Goals Using Abstract Syntax Trees

Abstract

The rapidly growing demand for programming skills has driven improvements in the technologies delivering programming education to students. Intelligent tutoring systems will potentially contribute to solving this problem, but development of effective systems has been slow to take hold in this area. We present a novel alternative, Abstract Syntax Tree Retrieval, which uses case-based reasoning to infer student goals from previous solutions to coding problems. Without requiring programmed expert knowledge, our system demonstrates that accurate retrieval is possible for basic problems. We expect that additional research will uncover more applications for this technology, including more effective intelligent tutoring systems.

Paul Freeman, Ian Watson, Paul Denny

Combining CBR and Deep Learning to Generate Surprising Recipe Designs

Abstract

This paper presents a dual-cycle CBR model in the domain of recipe generation. The model combines the strengths of deep learning and similarity-based retrieval to generate recipes that are novel and valuable (i.e. they are creative). The first cycle generates abstract descriptions which we call “design concepts” by synthesizing expectations from the entire case base, while the second cycle uses those concepts to retrieve and adapt objects. We define these conceptual object representations as an abstraction over complete cases on which expectations can be formed, allowing objects to be evaluated for surprisingness (the peak level of unexpectedness in the object, given the case base) and plausibility (the overall similarity of the object to those in the case base). The paper presents a prototype implementation of the model, and demonstrates its ability to generate objects that are simultaneously plausible and surprising, in addition to fitting a user query. This prototype is then compared to a traditional single-cycle CBR system.

Kazjon Grace, Mary Lou Maher, David C. Wilson, Nadia A. Najjar

Qualitative Case-Based Reasoning for Humanoid Robot Soccer: A New Retrieval and Reuse Algorithm

Abstract

This paper proposes a new Case-Based Reasoning (CBR) approach, named Q-CBR, that uses a Qualitative Spatial Reasoning theory to model, retrieve and reuse cases by means of spatial relations. A qualitative distance and orientation calculus (\(\mathcal {EOPRA}\)) is used to model cases using qualitative relations between the objects in a case. A new retrieval algorithm is proposed that uses the Conceptual Neighborhood Diagram to compute the similarity measure between a new problem and the cases in the case base. A reuse algorithm is also introduced that selects the most similar case and shares it with other agents, based on their qualitative position. The proposed approach was evaluated on simulation and on real humanoid robots. Preliminary results suggest that the proposed approach is faster than using a quantitative model and other similarity measure such as the Euclidean distance. As a result of running Q-CBR, the robots obtained a higher average number of goals than those obtained when running a metric CBR approach.

Thiago P. D. Homem, Danilo H. Perico, Paulo E. Santos, Reinaldo A. C. Bianchi, Ramon L. de Mantaras

Ensemble of Adaptations for Classification: Learning Adaptation Rules for Categorical Features

Abstract

Acquiring knowledge for case adaptation is a classic challenge for case-based reasoning (CBR). To provide CBR systems with adaptation knowledge, machine learning methods have been developed for automatically generating adaptation rules. An influential approach uses the case difference heuristic (CDH) to generate rules by comparing pairs of cases in the case base. The CDH method has been studied for case-based prediction of numeric values (regression) from inputs with primarily numeric features, and has proven effective in that context. However, previous work has not attempted to apply the CDH method to classification tasks, to generate rules for adapting categorical solutions. This paper introduces an approach to applying the CDH to cases with categorical features and target values, based on the generalized case value difference heuristic (GCVDH). It also proposes a classification method using ensembles of GCVDH-generated rules, ensemble of adaptations for classification (EAC), an extension to our previous work on ensembles of adaptations for regression (EAR). It reports on an evaluation comparing the accuracy of EAC to three baseline methods on four standard domains, as well as comparing EAC to an ablation relying on single adaptation rules, and assesses the effect of training/test size on accuracy. Results are encouraging for the effectiveness of the GCVDH approach and for the value of applying ensembles of learned adaptation rules for classification.

Vahid Jalali, David Leake, Najmeh Forouzandehmehr

Similarity Metrics from Social Network Analysis for Content Recommender Systems

Abstract

Online judges are online systems that test programs in programming contests and practice sessions. They tend to become big problem live archives, with hundreds, or even thousands, of problems. This wide problem statement availability becomes a challenge for new users who want to choose the next problem to solve depending on their knowledge. This is due to the fact that online judges usually lack of meta information about the problems and the users do not express their own preferences either. Nevertheless, online judges collect a rich information about which problems have been attempted, and solved, by which users. In this paper we consider all this information as a social network, and use social network analysis techniques for creating similarity metrics between problems that can be then used for recommendation.

Guillermo Jimenez-Diaz, Pedro Pablo Gómez Martín, Marco Antonio Gómez Martín, Antonio A. Sánchez-Ruiz

Analogical Transfer in RDFS, Application to Cocktail Name Adaptation

Abstract

This paper deals with analogical transfer in the framework of the representation language RDFS. The application of analogical transfer to case-based reasoning consists in reusing the problem-solution dependency to the context of the target problem; thus it is a general approach to adaptation. RDFS is a representation language that is a standard of the semantic Web; it is based on RDF, a graphical representation of data, completed by an entailment relation. A dependency is therefore represented as a graph representing complex links between a problem and a solution, and analogical transfer uses, in particular, RDFS entailment. This research work is applied (and inspired from) the issue of cocktail name adaptation: given a cocktail and a way this cocktail is adapted by changing its ingredient list, how can the cocktail name be modified?

Nadia Kiani, Jean Lieber, Emmanuel Nauer, Jordan Schneider

Adaptation-Guided Feature Deletion: Testing Recoverability to Guide Case Compression

Abstract

Extensive case-based reasoning research has studied methods for generating compact, competent case bases. This work has focused primarily on compressing the case base by deleting entire cases, based solely on their competence contributions. Recent work proposed an alternative which compressed individual cases by selectively deleting their internal contents. Early studies of this approach, termed flexible feature deletion (FFD), demonstrated that for suitable domains, such as domains with cases of varying sizes for which case usefulness can be retained despite internal deletions, even very simple FFD approaches may outperform standard per-case methods. However, more sophisticated methods are needed. Because FFD’s internal changes to cases can be seen as a form of case adaptation, this paper investigates whether the adaptation knowledge of a system can be harnessed to improve FFD. This paper proposes tying FFD choices directly to adaptation knowledge and presents results on a competence-preserving FFD method which prioritizes feature deletions by the recoverability of deleted features through case adaptation. Evaluation of recoverability-based FFD in a path-finding domain supports that it provides superior competence retention compared to standard flexible feature deletion at the same level of compression.

David Leake, Brian Schack

Applicability of Case-Based Reasoning for Selection of Cyanide-Free Gold Leaching Methods

Abstract

Designing hydrometallurgical experimental work, not to mention entire processes, is a complex task involving various ore properties and their combined effects on the available treatment methods. Gold leaching is one hydrometallurgical process, cyanide being the predominantly utilized leaching agent since late 1800s. Case-based reasoning (CBR) has previously been applied for selecting established process chains for a given gold ore, but with this paper, we are taking this previous research of gold processing towards cyanide-free leaching methods that are currently in development stage and not yet industrially applied. The utilization of CBR for cyanide-fee gold extraction experiment design is tested by building a preliminary CBR knowledge model to recommend treatments for gold extraction. Publications on cyanide-free leaching were analyzed and metallurgical researchers were interviewed in order to define the necessary attributes and their value ranges to be included in the model. We report the challenges encountered while building the CBR knowledge model, discuss its functionality and make suggestions for future research on the topic.

Maria Leikola, Lotta Rintala, Christian Sauer, Thomas Roth-Berghofer, Mari Lundström

Competence Guided Casebase Maintenance for Compositional Adaptation Applications

Abstract

A competence guided casebase maintenance algorithm retains a case in the casebase if it is useful to solve many problems and ensures that the casebase is highly competent in the global sense. In this paper, we address the compositional adaptation process (of which single case adaptation is a special case) during casebase maintenance by proposing a case competence model for which we propose a measure called retention score to estimate the retention quality of a case. We also propose a revised algorithm based on the retention score to estimate the competent subset of the casebase. We used regression datasets to test the effectiveness of the competent subset obtained from the proposed model. We also applied this model in a tutoring application and analyzed the competent subset of concepts in tutoring resources. Empirical results show that the proposed model is effective and overcomes the limitation of footprint based competence model in compositional adaptation applications.

Ditty Mathew, Sutanu Chakraborti

On the Transferability of Process-Oriented Cases

Abstract

This paper studies the feasibility of using transfer learning for process-oriented case-based reasoning. The work introduces a novel approach to transfer workflow cases from a loosely related source domain to a target domain. The idea is to develop a representation mapper based on workflow generalization, workflow abstraction, and structural analogy between the domain vocabularies. The approach is illustrated by a pair of sample domains in two sub-fields of customer relationship management that have similar process objectives but different tasks and data to fulfill them. An experiment with expert ratings of transferred cases is conducted to test the feasibility of the approach with promising results for workflow modeling support.

Mirjam Minor, Ralph Bergmann, Jan-Martin Müller, Alexander Spät

Case Completion of Workflows for Process-Oriented Case-Based Reasoning

Abstract

Cases available in real world domains are often incomplete and sometimes lack important information. Using incomplete cases in a CBR system can be harmful, as the lack of information can result in inappropriate similarity computations or incompletely generated adaptation knowledge. Case completion aims to overcome this issue by inferring missing information. This paper presents a novel approach to case completion for process-oriented case-based reasoning (POCBR). In particular, we address the completion of workflow cases by adding missing or incomplete dataflow information. Therefore, we combine automatically learned domain specific completion operators with generic domain-independent default rules. The empirical evaluation demonstrates that the presented completion approach is capable of deriving complete workflows with high quality and a high degree of completeness.

Gilbert Müller, Ralph Bergmann

Refinement-Based Similarity Measures for Directed Labeled Graphs

Abstract

This paper presents a collection of similarity measures based on refinement operators for directed labeled graphs (DLGs). We build upon previous work on refinement operators for other representation formalisms such as feature terms and description logics. Specifically, we present refinement operators for DLGs, which enable the adaptation of three similarity measures to DLGs: the anti-unification-based, \(S_{\lambda }\), the property-based, \(S_{\pi }\), and the weighted property-based, \(S_{w\pi }\), similarities. We evaluate the resulting measures empirically comparing them to existing similarity measures for structured data.

Santiago Ontañón, Ali Shokoufandeh

FEATURE-TAK - Framework for Extraction, Analysis, and Transformation of Unstructured Textual Aircraft Knowledge

Abstract

This paper describes a framework for semi-automatic knowledge extraction for case-based diagnosis in the aircraft domain. The available data on historical problems and their solutions contain structured and unstructured data. To transform these data into knowledge for CBR systems, methods and algorithms from natural language processing and case-based reasoning are required. Our framework integrates different algorithms and methods to transform the available data into knowledge for vocabulary, similarity measures, and cases. We describe the idea of the framework as well as the different tasks for knowledge analysis, extraction, and transformation. In addition, we give an overview of the current implementation, our evaluation in the application context, and future work.

Pascal Reuss, Rotem Stram, Cedric Juckenack, Klaus-Dieter Althoff, Wolfram Henkel, Daniel Fischer, Frieder Henning

Knowledge Extraction and Annotation for Cross-Domain Textual Case-Based Reasoning in Biologically Inspired Design

Abstract

Biologically inspired design (BID) is a methodology for designing technological systems by analogy to designs of biological systems. Given that knowledge of many biological systems is available mostly in the form of textual documents, the question becomes how can we extract design knowledge about biological systems from textual documents for potential use in designing engineering systems? In earlier work, we described how annotating biology articles with partial Structure-Behavior-Function models helps users access documents relevant to a given design problem and understand the biological systems for potential transfer of their causal mechanisms to engineering problems. In this paper, we present an automated technique instantiated in the IBID system for extracting partial SBF models of biological systems from their natural language documents for potential use in biologically inspired design.

Spencer Rugaber, Shruti Bhati, Vedanuj Goswami, Evangelia Spiliopoulou, Sasha Azad, Sridevi Koushik, Rishikesh Kulkarni, Mithun Kumble, Sriya Sarathy, Ashok Goel

Predicting the Electricity Consumption of Buildings: An Improved CBR Approach

Abstract

Case-based reasoning has recently been used to predict the hourly electricity consumption of institutional buildings. Past measurements of the building’s operation are modeled as cases and, combined with forecast weather information, used to predict the electricity demand for the next six hours. Elaborating on this idea, we present an improved CBR approach that yields more accurate predictions of energy consumption. In particular, we develop improved (local) similarity measures specifically tailored for this kind of application, and combine these measures with a regression-based method for similarity learning. Moreover, we incorporate a simple procedure for case adaptation. Experimental results for a real case study confirm a significant improvement in predictive accuracy compared to previous approaches.

Aulon Shabani, Adil Paul, Radu Platon, Eyke Hüllermeier

Case Representation and Retrieval Techniques for Neuroanatomical Connectivity Extraction from PubMed

Abstract

PubMed is a comprehensive database of abstracts and references of a large number of publications in the biomedical domain. Curation of structured connectivity databases creates an easy access point to the wealth of neuroanatomical connectivity information reported in the literature over years. Manual curation of such databases is time consuming and labor intensive. We present a Case Based Reasoning (CBR) approach to automatically compile connectivity status between brain region mentions in text. We focus on the Case Retrieval part of the CBR cycle and present three Instance based learning techniques to retrieve similar cases from the case base. These techniques use varied case representations ranging from surface level features to richer syntax based features. We have experimented with diverse similarity measures and feature weighting schemes for each technique. The three techniques have been evaluated and compared using a benchmark dataset from PubMed and it was found that the one using deep syntactic features gives the best trade off between Precision and Recall. In this study, we have explored issues pertaining to representation of, and retrieval over textual cases. It is envisaged that the ideas presented in the paper can be adapted to needs of other textual CBR domains as well.

Ashika Sharma, Ankit Sharma, Dipti Deodhare, Sutanu Chakraborti, P. Sreenivasa Kumar, P. Partha Mitra

Compositional Adaptation of Explanations in Textual Case-Based Reasoning

Abstract

When problem solving systems are deployed in real life, it is usually not enough to provide only a solution without any explanation. Users need an explanation in order to trust the system’s decisions. At the same time, explanations may also function internally in the system’s own reasoning process. One way to come up with an explanation for a new problem is to adapt an explanation from a similar problem encountered earlier, which is the idea behind the case-based explanation approach introduced by [29]. The original approach relies on manual construction of cases with explanations, which is difficult to scale up. In earlier work, therefore, we developed a system for automatic acquisition of cases with explanations from textual reports, including retrieval and adaptation of such cases [32, 33]. In this paper, we improve the adaptation method by combining explanations from more than one case, which we call compositional adaptation. The method is evaluated on an incident analysis task where the goal is to identify the root causes of a transportation incident, explaining it in terms of the information contained in the incident description. The evaluation results show that the proposed approach increases both the recall and the precision of the system.

Gleb Sizov, Pinar Öztürk, Erwin Marsi

Relevance Matrix Generation Using Sensitivity Analysis in a Case-Based Reasoning Environment

Abstract

Relevance matrices are a way to formalize the contribution of each attribute in a classification task. Within the CBR paradigm these matrices can be used to improve the global similarity function that outputs the similarity degree of two cases, which helps facilitate retrieval. In this work a sensitivity analysis method was developed to optimize the relevance values of each attribute of a case in a CBR environment, thus allowing an improved comparison of cases. The process begins with a statistical analysis of the values in a given dataset, and continues with an incremental update of the relevance of each attribute.

The method was tested on two datasets and it was shown that the statistical analysis performs better than evenly distributed relevance values, making it a suitable initial setting for the incremental update, and that updating the values over time gives better results than the statistical analysis.

Rotem Stram, Pascal Reuss, Klaus-Dieter Althoff, Wolfram Henkel, Daniel Fischer

Combining Case-Based Reasoning and Reinforcement Learning for Tactical Unit Selection in Real-Time Strategy Game AI

Abstract

This paper presents a hierarchical approach to the problems inherent in parts of real-time strategy games. The overall game is decomposed into a hierarchy of sub-problems and an architecture is created that addresses a significant number of these through interconnected machine-learning (ML) techniques. Specifically, individual modules that use a combination of case-based reasoning (CBR) and reinforcement learning (RL) are organised into three distinct yet interconnected layers of reasoning. An agent is created for the RTS game StarCraft and individual modules are devised for the separate tasks that are described by the architecture. The modules are individually trained and subsequently integrated in a micromanagement agent that is evaluated in a range of test scenarios. The experimental evaluation shows that the agent is able to learn how to manage groups of units to successfully solve a number of different micromanagement scenarios.

Stefan Wender, Ian Watson

Defining the Initial Case-Base for a CBR Operator Support System in Digital Finishing

A Methodological Knowledge Acquisition Approach

Abstract

Case-based reasoning (CBR) literature defines the process of defining a case-base as a hard and time-demanding task though the same literature does not report in detail on how to build your initial case base. The main contribution of this paper is the description of the methods that we used in order to build the initial case-base including the steps taken in order to make sure that the quality of the initial case set is appropriate. We first present the domain and argue why CBR is an appropriate solution for our application. Then we detail how we created the case base and show how the cases are validated.

Leendert W. M. Wienhofen, Bjørn Magnus Mathisen

Backmatter