Bayesian inference for an adaptive Ordered Probit model: An application to Brain Computer Interfacing

Abstract

This paper proposes an algorithm for adaptive, sequential classification in systems with unknown labeling errors, focusing on the biomedical application of Brain Computer Interfacing (BCI). The method is shown to be robust in the presence of label and sensor noise. We focus on the inference and prediction of target labels under a nonlinear and non-Gaussian model. In order to handle missing or erroneous labeling, we model observed labels as a noisy observation of a latent label set with multiple classes ($\ge 2$). Whilst this paper focuses on the method’s application to BCI systems, the algorithm has the potential to be applied to many application domains in which sequential missing labels are to be imputed in the presence of uncertainty. This dynamic classification algorithm combines an Ordered Probit model and an Extended Kalman Filter (EKF). The EKF estimates the parameters of the Ordered Probit model sequentially with time. We test the performance of the classification approach by processing synthetic datasets and real experimental EEG signals with multiple classes (2, 3 and 4 labels) for a Brain Computer Interfacing (BCI) experiment.

Introduction

Adaptive classification is an important problem in many application domains including biomedical engineering. Previous research in adaptive classification applied to biomedical engineering has used state space modeling of the time series (Lowne et al., 2010, Penny et al., 2000, Sykacek et al., 2004). The classification of an observed time series can often require extensive training before reliable results are obtained, as shown for biomedical applications (Pfurtscheller et al., 1993, Vidaurre et al., 2006, Wolpaw et al., 1991). Adaptive sequential classification approaches have been developed in order to reduce the overheads of training, to track non-stationarity and to cope rapidly with new observations. Such adaptive approaches differ from the typical methodology, in which an algorithm is trained off-line on retrospective data, in so far that the process of learning is continuously taking place rather than being confined to a section of training data.

In particular, Yoon et al., 2008a, Yoon et al., 2008b have investigated the use of the Extended Kalman Filter (EKF) to process and analyze brain signals with the assumption that an observed label is informative (there is little uncertainty), i.e. there are no ‘bit errors’ in the label stream. In general we do not, however, know the ground truth of the labels — we have only imperfect and noisy labels which we use as a reference. The assumption of error-free labels is hence invalid. In order to address this problem, we have developed a classification algorithm augmented by an auxiliary latent variable which corresponds to the uncorrupted underlying labels using Rao–Blackwellized Sequential Monte Carlo (Rao–Blackwellized SMC) (Yoon, Roberts, Dyson, & Gan, 2009). All these approaches have employed a standard Probit regression kernel for two class, binary classification. In this paper we develop a multi-class classifier.

Only two class labels may be distinguished by a sequential algorithm which employs a standard probit model (Aitchison & Silvey, 1957) with a threshold 0 so as to partition into two classes. The Probit model can be generalized by allowing multiple classes. Multi-class Probit models have been studied and applied in the economic (Geweke et al., 1994, McCulloch and Rossi, 1994) and clinical domains (Jung et al., 2006, Zhou et al., 2006). The multinomial Probit model is a well-known approach for unordered multi-class problems (Albert & Chip, 1993). Alternatively, for ordinal data, we can use the Ordered Probit model (Cameron and Trivedi, 2006, Munkin and Trivedi, 2007, Weiss, 1997).

It is common to find many application domains for ordinal classification, such as heights and weights of a population. For an ordinal dataset, the basic concept of the Ordered Probit model is that there is a latent continuous metric underlying the ordinal responses, and that thresholds partition the cumulative distribution function (c.d.f.) into a series of regions corresponding to the ordinal categories. Most literature in the Ordered Probit model focuses on off-line estimation of the thresholds for partitioning and on the estimation of their parameters and hyper-parameters (using, for example, Maximum Likelihood, Markov Chain Monte Carlo and Gibbs sampling) (Munkin and Trivedi, 2007, Ronning and Kukuk, 1996). However, the metrics (the thresholds) for ordinal classification may be adapted rather than kept fixed. In this paper, we propose an adaptive classifier using the Ordered Probit model which makes estimation of the non-stationary thresholds using hidden parameters whose values are dynamically inferred using an Extended Kalman Filter (EKF).

This paper is organized as follows. Section 2 presents the mathematical models for the proposed algorithms. We present the mathematical forms of the likelihood function and prior information with the Ordered Probit model based on a hierarchical framework. In Section 3 the algorithms are described in terms of prediction and filtering steps of the state space models. In Section 4 we present results from application to synthetic datasets (with two and three labels) and real experimental datasets composed of two and four label problems in the classification of human electroencephalogram (EEG) signals used for a Brain Computer Interfacing system.