Complete Stacked Denoising Auto-Encoders for Regression

Complete modified stacked denoising auto-encoder (CMSDAE) machines constitute a version of stacked auto-encoders in which a target estimate is included at the input, and are trained layer-by-layer by minimizing a convex combination of the errors corresponding to the input sample and the target. This permits to carry out the transformation of the observation space without forgetting what the target is. It has been shown in recent publications that this method produces a clear performance advantage in classification tasks. The above facts motivate to explore whether CMSDAE machines also offer performance improvements in regression problems, and in particular for time series prediction where conventional discriminative machines find difficulties: The layer-by-layer reconstruction of the target (together with the input) can help to reduce these difficulties. This contribution presents the CMSDAE regression/prediction machines and their design, showing experimental evidence of their frequent superior performance —never lower— with respect to other machine architectures. Some subsequent research directions are indicated together with the conclusions.