Determining an Adequate Number of Principal Components
The problem of choosing the number of PCs to retain is analyzed in the context of <i>model selection</i>, using so-called <i>model selection criteria</i> (MSCs). For a prespecified set of models, indexed by k=1,2,…,K, these model selection criteria (MSCs) take the form MSC<i><sub>k</sub></i>=<i><sub...
Saved in:
| Main Author: | |
|---|---|
| Format: | Book Chapter |
| Language: | English |
| Published: |
IntechOpen
2022
|
| Subjects: | |
| Online Access: | https://www.intechopen.com/chapters/81645 |
| Tags: |
No Tags, Be the first to tag this record!
|
| Summary: | The problem of choosing the number of PCs to retain is analyzed in the context of <i>model selection</i>, using so-called <i>model selection criteria</i> (MSCs). For a prespecified set of models, indexed by k=1,2,…,K, these model selection criteria (MSCs) take the form MSC<i><sub>k</sub></i>=<i><sub>n</sub>LL<sub>k</sub></i>+<i>a<sub>n</sub>m<sub>k</sub></i>, where, for model <i>k</i>, <i>LL<sub>k</sub></i> is the maximum log likelihood, <i>m<sub>k</sub></i> is the number of independent parameters, and the constant <i>a<sub>n</sub></i> is <i>a<sub>n</sub></i>=ln<i>n</i> for BIC and <i>a<sub>n</sub></i>=2 for AIC. The maximum log likelihood <i>LL<sub>k</sub></i> is achieved by using the maximum likelihood estimates (MLEs) of the parameters. In Gaussian models, <i>LL<sub>k</sub></i> involves the logarithm of the mean squared error (MSE). The main contribution of this chapter is to show how to best use BIC to choose the number of PCs, and to compare these results to <i>ad hoc</i> procedures that have been used. Findings include the following. These are stated as they apply to the eigenvalues of the correlation matrix, which are between 0 and <i>p</i> and have an average of 1. For considering an additional PC<i><sub>k + 1</sub></i>, with AIC, inclusion of the additional PC<i><sub>k + 1</sub></i> is justified if the corresponding eigenvalue <i>λ<sub>k+1</sub></i> is greater than exp(−2/<i>n</i>). For BIC, the inclusion of an additional PC<i><sub>k + 1</sub></i> is justified if <i>λ<sub>k+1</sub></i> > <i>n<sup>1/n</sup></i>, which tends to 1 for large <i>n</i>. Therefore, this is in approximate agreement with the average eigenvalue rule for correlation matrices, stating that one should retain dimensions with eigenvalues larger than 1. |
|---|---|
| ISBN: | 978-1-80355-765-6 |
| DOI: | 10.5772/intechopen.104534 |