1 Introduction
2 Methods
2.1 PCA
2.2 Cardinality-constrained PCA
2.3 Penalized PCA
3 Simulation study
3.1 Design
R
software for statistical computing (R Core Team 2020), and SPCA with LARS using the elastic net R
-package (Zou and Hastie 2018). Both algorithms were run with one initial value, based on the SVD decomposition of the data. We supplied the analysis with the actual number of components. The tuning parameter of the ridge penalty for SPCA was left at the default value of \(10^{-6}\); this is a small value such that the focus remains on comparing the cardinality constraint to the LASSO penalty as a means to sparsify the PCA problem in (1).3.2 Results
Sparsity = 20% | Sparsity = 80% | ||||||
---|---|---|---|---|---|---|---|
\(J=50\) | \(J=100\) | \(J=500\) | \(J=50\) | \(J=100\) | \(J=500\) | ||
Noise level = 5% | |||||||
MAB | CCPCA | 0.1082 | 0.0760 | 0.0339 | 0.0669 | 0.0474 | 0.0211 |
SPCA | 0.0999 | 0.0701 | 0.0314 | 0.0571 | 0.0401 | 0.0171 | |
MVAR | CCPCA | 0.0198 | 0.0099 | 0.0020 | 0.0214 | 0.0105 | 0.0021 |
SPCA | 0.0198 | 0.0099 | 0.0020 | 0.0198 | 0.0099 | 0.0020 | |
MSE | CCPCA | 0.0410 | 0.0200 | 0.0037 | 0.0408 | 0.0212 | 0.0041 |
SPCA | 0.0427 | 0.0196 | 0.0040 | 0.0386 | 0.0205 | 0.0041 | |
Noise level = 20% | |||||||
MAB | CCPCA | 0.1084 | 0.0762 | 0.0341 | 0.0663 | 0.0480 | 0.0212 |
SPCA | 0.1005 | 0.0702 | 0.0315 | 0.0539 | 0.0399 | 0.0171 | |
MVAR | CCPCA | 0.0198 | 0.0099 | 0.0020 | 0.0211 | 0.0106 | 0.0021 |
SPCA | 0.0198 | 0.0099 | 0.0020 | 0.0198 | 0.0099 | 0.0020 | |
MSE | CCPCA | 0.0422 | 0.0206 | 0.0041 | 0.0417 | 0.0214 | 0.0043 |
SPCA | 0.0411 | 0.0206 | 0.0042 | 0.0392 | 0.0208 | 0.0041 | |
Noise level = 80% | |||||||
MAB | CCPCA | 0.1108 | 0.0783 | 0.0345 | 0.0828 | 0.0573 | 0.0220 |
SPCA | 0.1018 | 0.0717 | 0.0315 | 0.0569 | 0.0396 | 0.0172 | |
MVAR | CCPCA | 0.0198 | 0.0099 | 0.0020 | 0.0279 | 0.0134 | 0.0022 |
SPCA | 0.0198 | 0.0099 | 0.0020 | 0.0198 | 0.0099 | 0.0020 | |
MSE | CCPCA | 0.0403 | 0.0204 | 0.0038 | 0.0489 | 0.0236 | 0.0041 |
SPCA | 0.0403 | 0.0202 | 0.0039 | 0.0400 | 0.0200 | 0.0039 |
Sparsity = 20% | Sparsity = 80% | ||||||
---|---|---|---|---|---|---|---|
\(J=50\) | \(J=100\) | \(J=500\) | \(J=50\) | \(J=100 \) | \(J=500\) | ||
Noise level = 5% | |||||||
MAB | CCPCA | 0.0764 | 0.0533 | 0.0234 | 0.0526 | 0.0365 | 0.0160 |
SPCA | 0.1016 | 0.0710 | 0.0295 | 0.0681 | 0.0495 | 0.0221 | |
MVAR | CCPCA | 0.0536 | 0.0321 | 0.0077 | 0.0345 | 0.0176 | 0.0047 |
SPCA | 0.0198 | 0.0099 | 0.0020 | 0.0198 | 0.0099 | 0.0020 | |
MSE | CCPCA | 0.0468 | 0.0263 | 0.0064 | 0.0173 | 0.0090 | 0.0030 |
SPCA | 0.0104 | 0.0050 | 0.0008 | 0.0091 | 0.0045 | 0.0008 | |
Noise level = 20% | |||||||
MAB | CCPCA | 0.0769 | 0.0532 | 0.0242 | 0.0529 | 0.0375 | 0.0162 |
SPCA | 0.1019 | 0.0715 | 0.0297 | 0.0560 | 0.0483 | 0.0218 | |
MVAR | CCPCA | 0.0377 | 0.0220 | 0.0045 | 0.0275 | 0.0151 | 0.0035 |
SPCA | 0.0198 | 0.0099 | 0.0020 | 0.0198 | 0.0099 | 0.0020 | |
MSE | CCPCA | 0.0267 | 0.0167 | 0.0033 | 0.0107 | 0.0070 | 0.0018 |
SPCA | 0.0118 | 0.0053 | 0.0009 | 0.0050 | 0.0048 | 0.0008 | |
Noise level = 80% | |||||||
MAB | CCPCA | 0.0820 | 0.0577 | 0.0260 | 0.0824 | 0.0574 | 0.0210 |
SPCA | 0.1053 | 0.0747 | 0.0300 | 0.1053 | 0.0729 | 0.0242 | |
MVAR | CCPCA | 0.0279 | 0.0141 | 0.0028 | 0.0282 | 0.0135 | 0.0023 |
SPCA | 0.0198 | 0.0099 | 0.0020 | 0.0198 | 0.0099 | 0.0020 | |
MSE | CCPCA | 0.0439 | 0.0189 | 0.0018 | 0.0435 | 0.0167 | 0.0009 |
SPCA | 0.0369 | 0.0152 | 0.0011 | 0.0368 | 0.0152 | 0.0011 |
4 Empirical application
5 Conclusion
R
-software. When using them, it is essential to consider that both methods are subject to local minima. It is a common practice to implement a multi-start procedure and select the solution with the smallest objective function, but the obtained solutions will still be subject to local optima. For future work, it would be interesting to analyze the conditions for optimality for sparse PCA methods.