A detection metric designed for O’Connell effect eclipsing binaries

Several theories propose to explain the effect, including starspots, gas stream impact, and circumstellar matter (McCartney 1999). The work by Wilsey and Beaky (2009) outlines each of these theories and demonstrates how the observed effects are generated by the underlying physics.

In the study (McCartney 1999), the authors limited the sample to only six star systems: GSC 03751-00178, V573 Lyrae, V1038 Herculis, ZZ Pegasus, V1901 Cygni, and UV Monocerotis. Researchers have used standard eclipsing binary simulations (Wilson and Devinney 1971) to demonstrate the proposed explanations for each light curve instance and estimate the parameters associated with the physics of the system. Wilsey and Beaky (2009) noted other cases of the O’Connell effect in binaries, which have since been described physically; in some cases, the effect varied over time, whereas in other cases, the effect was consistent over years of observation and over many orbits. The effect has been found in both overcontact, semidetached, and near-contact systems.

While one of the key visual differentiators of the O’Connell effect is \(\Delta m_{\mathrm{max}}\), this heuristic feature alone could not be used as a general mean for detection, as the targets trained on or applied to are not guaranteed to be (a) eclipsing binaries and (b) periodic. One of the goals we are attempting to highlight is the transformation of expert qualitative target selection into quantitative machine learning methods.

We develop a detection methodology for a specific target of interest—OEEB—defined as an eclipsing binary where the light curve (LC) maxima are consistently at different amplitudes over the span of observation. Beyond differences in maxima, and a number of published examples, little is defined as a requirement for identifying the O’Connell effect (Wilsey and Beaky 2009; Knote 2019).

McCartney (1999) provide some basic indicators/measurements of interest in relation to OEEB binaries: the O’Connell effect ratio (OER), the difference in maximum amplitudes (Δm), the difference in minimum amplitudes, and the light curve asymmetry (LCA). The metrics are based on the smoothed phased light curves. The OER is calculated as Equation (1): where the min-max amplitude (i.e. normalized flux) measurements for each star are grouped into phase bins (\(n=500\)), where the mean amplitude in each bin is \(I_{i}\). An \(\mathrm{OER}>1\) corresponds to the front half of the light curve having more total flux; note that for the procedure we present here, \(I_{1}=0\). The difference in max amplitude is calculated as Equation (2): where we have estimated the maximum in each half of the phased light curve. The LCA is calculated as Equation (3):

As opposed to the measurement of OER, LCA measures the deviance from symmetry of the two peaks. Defining descriptive metrics or functional relationships (i.e., bounds of distribution) requires a larger sample than is presently available. An increased number of identified targets of interest is required to provide the sample size needed for a complete statistical description of the O’Connell effect. The quantification of these functional statistics allows for the improved understanding of not just the standard definition of the targeted variable star but also the population distribution as a whole. These estimates allow for empirical statements to be made regarding the differences in light curve shapes among the variable star types investigated. The determination of an empirically observed distribution, however, requires a significant sample to generate meaningful descriptive statistics for the various metrics.

In this effort, we highlight the functional shape of the phased light curve as our defining feature of OEEB stars. The prior metrics identified are selected or reduced measures of this functional shape. We propose here that, as opposed to training a detector on the preceding indicators, we use the functional shape of the phased light curve by way of the distribution field to construct our automated system.

Leveraging the Kepler pipeline already in place, and using the data from the Villanova Eclipsing Binary catalog (Kirk et al. 2016), this study focuses on a set of predetermined eclipsing binaries identified from the Kepler catalog. From this catalog, we developed an initial, expertly derived, labeled data set of proposed targets “of interest” identified as OEEB. Likewise, we generated a set of targets identified as “not of interest” based on our expert definitions, i.e., intuitive inference.

We have labeled our two populations “of interest” and “not of interest” to represent those targets in the initial training set that a user has found to either be interesting to their research (identified OEEBs) or otherwise. The labeling “of interest” here is specifically used, as we are dependent on the expert selections.

Using the Eclipsing Binary catalog (Kirk et al. 2016), we identified a set of 30 targets “of interest” (see Table 1) and 121 targets of “not of interest” (see Table 2) via expert analysis—by-eye selection based on researchers’ interests. Specific target identification is listed in a supplementary digital file at the project repository.¹ We use this set of 151 light curves for training and testing.

The 2000+ eclipsing binaries left in the Kepler Eclipsing Binary catalog are left as unlabeled targets. We use our described detector to “discover” targets of interest, i.e., OEEB. The full set of Kepler data is accessible via the Villanova Eclipsing Binary website (http://​keplerebs.​villanova.​edu/​).

For analyzing the proposed algorithm design, the LINEAR data set is also leveraged as an unknown “unlabeled” data set ripe for OEEB discovery (Sesar et al. 2011; Palaversa et al. 2013). From the starting sample of 7194 LINEAR variables, we used a clean sample of 6146 time series data sets for detection. Stellar class type is limited further to the top five most populous classes—RR Lyr (ab), RR Lyr (c), Delta Scuti / SX Phe, Contact Binaries, and Algol-Like Stars with two minima—resulting in a set of 5,086 observations.

Unlike the Kepler Eclipsing Binary catalog, the LINEAR data set contains targets other than (but does include) eclipsing binaries; the data set we used (Johnston and Peter 2017) includes Algols (287), Contact Binaries (1805), Delta Scuti (68), and RR Lyr (ab-2189, c-737). The light curves are much more poorly sampled; this uncertainty in the functional shape results from lower SNR (ground survey) and poor sampling. The distribution of stellar classes is presented in Table 3.

The full data sets used at the time of this publication from the Kepler and LINEAR surveys are available from the associated public repository.²

Many prior studies on time-domain variable star classification (Debosscher 2009; Barclay et al. 2011; Blomme et al. 2011; Dubath et al. 2011; Pichara et al. 2012; Pichara and Protopapas 2013; Graham et al. 2013; Angeloni et al. 2014; Masci et al. 2014) rely on periodicity domain feature space reductions. Debosscher (2009) and Templeton (2004) review a number of feature spaces and a number of efforts to reduce the time-domain data, most of which implement Fourier techniques, primarily the Lomb–Scargle (L-S) method (Lomb 1976; Scargle 1982), to estimate the primary periodicity (Eyer and Blake 2005; Deb and Singh 2009; Richards et al. 2012; Park and Cho 2013; Ngeow et al. 2013).

The studies on classification of time-domain variable stars often further reduce the folded time-domain data into features that provide maximal-linear separability of classes. These efforts include expert selected feature efforts (Debosscher 2009; Sesar et al. 2011; Richards et al. 2012; Graham et al. 2013; Armstrong et al. 2016; Mahabal et al. 2017; Hinners et al. 2018), automated feature selection efforts (McWhirter et al. 2017; Naul et al. 2018), and unsupervised methods for feature extraction (Valenzuela and Pichara 2018; Modak et al. 2018). The astroinformatics community-standard features include quantification of statistics associated with the time-domain photometric data, Fourier decomposition of the data, and color information in both the optical and IR domains (Nun et al. 2015; Miller et al. 2015). The number of individual features commonly used is upward of 60 and growing (Richards et al. 2011) as the number of variable star types increases and as a result of further refinement of classification definitions (Samus’ et al. 2017). Curiously, aside from efforts to construct a classification algorithm from the time-domain data directly (McWhirter et al. 2017), few efforts in astroinformatics have looked at features beyond those described here—mostly Fourier domain transformations or time domain statistics. Considering the depth of possibility for time-domain transformations (Fu 2011; Grabocka et al. 2012; Cassisi et al. 2012; Fulcher et al. 2013), it is surprising that the community has focused on just a few transforms. Additionally, there has been recent work in exo-planet detection using whole phased waveform data (smoothed), in combination with neural network classification (Pearson et al. 2017), and with local linear embedding (Thompson et al. 2015). Similar to the design proposed here, these methods use the classifier to optimize the feature space (the phased waveform) for the purposes of detection.

Here we propose an implementation that simplifies the traditional design: limiting ourselves to a one versus all approach (Johnston and Oluseyi 2017) targeting a variable type of interest; limiting ourselves to a singular feature space—the distribution field of the phased light curve—based on (Helfer et al. 2015) as a representation of the functional shape; and introducing a classification/detection scheme that is based on similarity with transparent results (Bellet et al. 2015) that can be further extended, allowing for the inclusion of an anomaly detection algorithm.

As stated, this analysis focuses on detecting OEEB systems based on their light curve shape. The OEEB signature has a cyclostationary signal, a functional shape that repeats with a consistent frequency. The signature can be isolated using a process of period finding, folding, and phasing (Graham et al. 2013); the Villanova catalog provides the estimated “best period.” The proposed feature space transformation will focus on the quantification or representation of this phased functional shape. This particular implementation design makes the most intuitive sense, as visual inspection of the phased light curve is the way experts identify these unique sources.

As discussed, prior research on time-domain data identification has varied between generating machine-learned features (Gagniuc 2017), implementing generic features (Masci et al. 2014; Palaversa et al. 2013; Richards et al. 2012; Debosscher 2009), and looking at shape- or functional-based features (Haber et al. 2015; Johnston and Peter 2017; Park and Cho 2013). This analysis will leverage the distribution field transform to generate a feature space that can be operated on; a distribution field (DF) is defined as (Helfer et al. 2015; Sevilla-Lara and Learned-Miller 2012) Equation (5): where N is the number of samples in the phased data, and \([\: ]\) is the Iverson bracket (Iverson 1962), given as and \(y_{j}\) and \(x_{i}\) are the corresponding normalized amplitude and phase bins, respectively, where \(x_{i} = {0, 1/n_{x}, 2/n_{x}, \dots , 1}\), \(y_{i} = {0, 1/n_{y}, 2/n_{y}, \dots , 1}\), \(n_{x}\) is the number of time bins, and \(n_{y}\) is the number of amplitude bins. The result is a right stochastic matrix, i.e., the rows sum to 1. Bin number, \(n_{x}\) and \(n_{y}\), is optimized by cross-validation as part of the classification training process. Smoothed phased data—generated from SUPERSMOOTHER—are provided to the DF algorithm.

We found this implementation to produce a more consistent classification process. We found that the min-max scaling normalization—normalized flux—if applied by itself without smoothing, when outliers are present can produce final patterns that focus more on the outlier than the general functionality of the light curve. Likewise, we found that using the unsmoothed data in the DF algorithm resulted in a classification that was too dependent on the scatter of the phased light curve. Although at first glance, that would not appear to be an issue, this implementation resulted in light curve resolution having a large impact on the classification performance—in fact, a higher impact than the shape itself. An example of this transformation is given in Fig. 3.

Though the DF exhibits properties that a detection algorithm can use to identify specific variable stars of interest, it alone is not sufficient for our ultimate goal of automated detection. Rather than vectorizing the DF matrix and treating it as a feature vector for standard classification techniques, we treat the DF as the matrix-valued feature that it is (Helfer et al. 2015). This allows for the retention of row and column dependence information that would normally be lost in the vectorization process (Ding and Dennis Cook 2018).

At its core, the proposed detector is based on the definition of similarity and, more formally, a definition of distance. Consider the example triplet “x is more similar to y than to z,” i.e., the distance between x and y in the feature space of interest is smaller than the distance between x and z. The field of metric learning focuses on defining this distance in a given feature space to optimize a given goal, most commonly the reduction of error rate associated with the classification process. Given the selected feature space of DF matrices, the distance between two matrices X and Y (Bellet et al. 2015; Helfer et al. 2015) is defined as Equation (7): M is the metric that we will be attempting to optimize, where \(M\succeq 0\) (positive semi-definite). The PMML procedure outlined in (Helfer et al. 2015) is similar to the metric learning methodology LMNN (Weinberger et al. 2009), save for its implementation on matrix-variate data as opposed to vector-variate data. We summarize it here. The developed objective function is given in Equation (8): where \(N_{c}\) is the number of training data in class c; λ and γ are variables to control the importance of push versus pull and regularization, respectively. Bellet et al. (2015) define the triplet \(\{ \mathrm{DF}_{c}^{i},\mathrm{DF}_{c}^{j}, \mathrm{DF}_{c}^{k} \} \) as the relationship between similar and dissimilar observations, i.e., \(\mathrm{DF}_{c}^{i}\) is similar to \(\mathrm{DF}_{c}^{j}\) and dissimilar to \(\mathrm{DF}_{c}^{k}\). Note, the summation over i and j is the summation over similar observations in the training data, and the summation i and k is the summation over dissimilar observations.

There are three basic components: a pull term, which is small when the distance between similar observations is small; a push term, which is small when the distance between dissimilar observations is larger; and a regularization term, which is small when the Frobenius norm (\(\Vert M \Vert _{F}^{2} = \sqrt{\operatorname{Tr}(MM^{H})}\)) of M is small. Thus the algorithm attempts to bring similar distribution fields closer together, while pushing dissimilar ones farther apart, while attempting to minimize the complexity of the metric M. The regularizer on the metric M guards against overfitting and consequently enhances the algorithm’s ability to generalize, i.e., allow for operations across data sets. This regularization strategy is similar to popular regression techniques like lasso and ridge (Hastie et al. 2009).

Additional parameters λ and γ weight the importance of the push–pull terms and metric regularizer, respectively. These free parameters are typically tuned via standard cross-validation techniques on the training data. The objective function represented by Equation (8) is quadratic in the unknown metric M; hence it is possible to obtain the following closed-form solution to the minimization of Equation (8) as:

Equation (9) does not guarantee that M is positive semi-definite (PSD). To ensure this property, we can apply the following straightforward projection step after calculating M to ensure the requirement of \(M\succeq 0\):

If M is not PSD, then the distance axioms are not held up, and therefore our similarity that we are using \(d(X,Y)= \Vert X-Y \Vert _{M}^{2}\) would not be a true distance, specifically if M is not symmetric then \(d(x_{i}, x_{j}) \neq d(x_{j}, x_{i})\). This projected metric is used in the classification algorithm. The metric learned from this push–pull methodology is used in conjunction with a standard k-nearest neighbor (k-NN) classifier.

The traditional k-NN algorithm is a nonparametric classification method; it uses a voting scheme based on an initial training set to determine the estimated label (Altman 1992). For a given new observation, the \(L_{2}\) Euclidean distance is found between the new observation and all points in the training set. The distances are sorted, and the k closest training sample labels are used to determine the new observed sample estimated label (majority rule). Cross-validation is used to find an optimal k value, where k is any integer greater than zero.

The k-NN algorithm estimates a classification label based on the closest samples provided in training. For our implementation, the distance between a new pattern \(\mathrm{DF}^{i}\) and each pattern in the training set is found, using the optimized metric instead of the standard identity metric that would have been used in \(L_{2}\) Euclidean distance. The new pattern is classified depending on the majority of the closest k class labels. The distance between patterns is in Equation (7), using the learned metric M.

Optimized feature dimensions and parameters used in the classification process were estimated via five-fold cross-validation (Duda et al. 2012). Our procedure is as follows: the original set of 151 was split into two groups, 76 for training and 75 for testing. The training dataset is partitioned into 5 groups of equal sizes (14), with no replication. To evaluate the optimal DF dimensions we loop over a range of both x and y resolutions. We also include a loop for number of k-Neighbors. Within these loops we evaluate the performance of three classifiers that have limited input parameters to train using our partitioned data. This cycling over the partitioned data is the 5-fold cross-validation, within this loop we cycle over each partition, leaving it out, using the other four for training, and comparing the classifier trained on the four against the one left out.

The resulting error average over the five cycles is used as the performance estimate for the selection of x/y/k resolution. After the resulting analysis, we have three error estimates, per x/y/k resolution pairing. We select the “optimal” x/y resolution pairing based on a minimization of the PMML classifier, for all classifiers trained (the optimal resolutions for the other classifier methods resulted in roughly the same resolution at the PMML one). These optimal resolutions are then used to generate the DF features from all of the training data. This training data is then used to train the classifier selected, and those trained classifiers are then applied to the testing data that was originally set aside. The resulting misclassification rates are provided in the Table 6.

We make the following general notes/caveats about the process used:

The minimization of misclassification rate is used to optimize floating parameters in the design, such as the number of x-bins, the number of y-bins, and k-values (i.e. k number of neighbors). Some parameters are more sensitive than others; often this insensitivity is related to the loss function or the feature space, or the data themselves. For example, the γ and λ values weakly affected the optimization, while the bin sizes and k-values had a stronger effect (γ and λ values tested both spanned the range 0.0 to 1.0, k-Values were tested for odd values between 1 and 13).

The cross-validation process was then reduced to optimizing the \(n_{x}\), \(n_{y}\), and k values; \(n_{x}\) and \(n_{y}\) values were tested over values 20–40 in steps of 5. The set of optimized parameters is given as γ=1.0, \(\lambda =0.75\), \(n_{x} = 25\), \(n_{y} = 35\), and \(\mathrm{k}=3\). Given the optimization of these floating variables in all three algorithms, the testing data are then applied to the optimal designs (testing results provided in Sect. 5.2.2).

The algorithm is applied to the Villanova Eclipsing Binary catalog entries that were not identified as either “Of Interest” or “Not of Interest,” i.e., unlabeled for the purposes of our targeted goal. The trained and tested data sets are combined into a single training set for application; the primary method (push–pull metric classification) is used to optimize a metric based on the optimal parameters found during cross-validation and to apply the system to the entire Villanova Eclipsing Binary data (2875 curves).

We further demonstrate the algorithm with an application to a separate independent survey. Machine learning methods have been applied to classifying variable stars observed by the LINEAR survey (Sesar et al. 2011), and while these methods have focused on leveraging Fourier domain coefficients and photometric measurements \(\{ u,g,r,i,z \} \) from SDSS, the data also include best estimates of period, as all of the variable stars trained on had cyclostationary signatures. It is then trivial to extract the phased light curve for each star and apply our Kepler trained detector to the data to generate “discovered” targets of interest.

The discovered targets are aligned, and the smoothed light curves are presented in Fig. 7. Note that the LINEAR IDs are presented in Table 5 and as a supplementary digital file at the project repository.⁵

Application of our Kepler trained detector to LINEAR data results in 24 “discovered” OEEBs. These include four targets with a negative O’Connell effect. Similar to the Kepler discovered data set, we plot \(\mathrm{OER}/\Delta m\) features using lower-resolution phased binnings (\(n=20\)) and see that the distribution and relationship from McCartney (1999) hold here as well (see Fig. 8).

The pairing of DF feature space and push–pull matrix metric learning represents a novel design; thus it is difficult to draw conclusions about performance of the design, as there are no similar studies that have trained on this particular data set, targeted this particular variable type, used this feature space, or used this classifier. As we discussed earlier, classifiers that implement matrix-variate features directly are few and far between and almost always not off the shelf. We have developed here two hybrid designs—off-the-shelf classifiers mixed with feature space transform—to provide context and comparison.

These two additional classification methodologies implement more traditional and well-understood features and classifiers: k-NN using \(L^{2}\) distance applied to the phased light curves (Method A) and k-means representation with quadratic discriminant analysis (QDA) (Method B). Method A is similar to the UCR (Chen et al. 2015) time series data baseline algorithm, reported as part of the database. Provided here is a direct k-NN classification algorithm applied directly to the smoothed, aligned, regularly sampled phased light curve. This regular sampling is generated via interpolation of the smoothed data set and is required because of the nature of the nearest neighbor algorithm requiring one-to-one distance. Standard procedures can then be followed (Hastie et al. 2009). Method B borrows from Park et al. (2003), transforming the matrix-variate data into vector-variate data via estimation of distances between our training set and a smaller set of exemplar means DFs that were generated via unsupervised learning. Distances were found using the Frobenius norm of the difference between the two matrices.

Whereas Method A uses neither the DF feature representation nor the metric learning methodology, Method B uses DF feature space but not the metric learning methodology. This presents a problem, however, as most standard out-of-the-box classification methods require a vector input. Indeed, many methodologies, even when faced with a matrix input, choose to vectorize the matrix. An alternative to this implementation is a secondary transformation into a lower-dimensional feature space. Following the work of Park et al. (2003), we implement a matrix distance k-means algorithm (e.g., k-means with a Frobenius norm) to generate estimates of clusters in the DF space. Observations are transformed by finding the Euclidean distance between each training point and each of the k-mean matrices discovered. The resulting set of k-distances is treated as the input pattern, allowing the use of the standard QDA algorithm (Duda et al. 2012). The performances of both the proposed methodology and the two comparative methodologies are presented in Table 6. The algorithms are available as open source code, along with our novel implementation, at the project repository.

We present the performance of the main novel feature space/classification pairing as well as the two additional implementations that rely on more standard methods. Here we have evaluated performance based on misclassification rate, i.e., 1-accuracy given by Fawcett (2006) as \(1 - \mathrm{correct}/{\mathrm{total}}\). The method we propose has a marginally better misclassification rate (Table 6) and has the added benefit of (1) not requiring unsupervised clustering, which can be inconsistent, and (2) providing nearest neighbor estimates allowing for demonstration of direct comparison. These performance estimate values are dependent on the initial selected training and testing data. They have been averaged and optimized via cross-validation; however, with so little initial training data and with the selection process for which training and testing data are randomized, performance estimates may vary. Of course, increases in training data will result in increased confidence in performance results.

We have not included computational times as part of this analysis, as they tend to be dependent on the system operated on. We can anecdotally discuss that, on the system implemented as part of this research (MacBook Pro, 2.5 GHz Intel i7, 8 GB RAM), the training optimization of our proposed feature extraction and PPML classification total took less than 5–10 min to run—variation depending on whatever else was running in the background. Use of the classifiers on unlabeled data resulted in a classification in fractions of seconds per star. However, we should note that this algorithm will speed up if it is implemented on a parallel processing system, as much of the time taken in the training process resulted from linear algebra operations that can be parallelized.

The DF representation maps deterministic, functional stellar variable observations to a stochastic matrix, with the rows summing to unity. The inherently probabilistic nature of DFs provides a robust way to model interclass variability and handle irregular sampling rates associated with stellar observations. Because the DF feature is indifferent to sampling density so long as all points along the functional shape are represented, the trained detection algorithm we generate and demonstrate in this article can be trained on Kepler data but directly applied to the LINEAR data, as shown in Sect. 5.3.

The algorithm, including comparison methodologies, designed feature space transformations, classifiers, utilities, and so on, is publicly available at the project repository (see OCD, Johnston et al. 2019);⁶ all code was developed in MATLAB and was run on MATLAB 9.3.0.713579 (R2017b). The operations included here can be executed either via calling individual functions or using the script provided (ImplementDetector.m). Likewise, a Java version of all of the individual computational functions has been generated (see JVarStar, Johnston et al. 2019) and is included in the project repository.⁷

This design is modular enough to be applied as is to other types of stars and star systems that are cyclostationary in nature. With a change in feature space, specifically one that is tailored to the target signatures of interest and based on prior experience, this design can be replicated for other targets that do not demonstrate a cyclostationary signal (i.e., impulsive, nonstationary, etc.) and even to targets of interest that are not time variable in nature but have a consistent observable signature (e.g., spectrum, photometry, image point-spread function, etc.). One of the advantages of attempting to identify the O’Connell effect Eclipsing Binary is that one only needs the phased light curve—and thus the dominant period allowing a phasing of the light curve—to perform the feature extraction and thus the classification. The DF process here allows for a direct transformation into a singular feature space that focuses on functional shape.

For other variable stars, a multiview approach might be necessary; either descriptions of the light curve signal across multiple transformations (e.g., Wavelet and DF), or across representations (e.g. polarimetry and photometry) or across frequency regimes (e.g. optical and radio) would be required in the process of properly defining the variable star type. The solution to this multiview problem is neither straightforward nor well understood (Akaho 2006). Multiple options have been explored to resolve this problem: combination of classifiers, canonical correlation analysis, postprobability blending, and multimetric classification. The computational needs of the algorithm have only been roughly studied, and a more thorough review is necessary in the context of the algorithm proposed and the needs of the astronomy community. The k-NN algorithm dependence on pairwise difference, while one of its strong suits is also one of the more computationally demanding parts of the algorithm. Functionality such as \(k-d\) trees as well as other feature space partitioning methods have been shown to reduce the computational requirements.