Multivariate Statistical Analysis Anderson
Content
An Introduction to Multivariate Statistical Analysis
Second Edition
T. W. ANDERSON Professor of Statistics and Economics Stanford University
JOHN WILEY & SONS New York Chichester Brisbane
Toronto
Singapore
Contents
CHAPTER 1
Introduction 1.1. Multivariate Statistical Analysis 1.2. The Multivariate Normal Distribution
1 1 3
CHAPTER 2
The Multivariate Normal Distribution Introduction Notions of Multivariate Distributions The Multivariate Normal Distribution The Distribution of Linear Combinations of Normally Distributed Variates; Independence of Variates; Marginal Distributions 2.5. Conditional Distributions and Multiple Correlation Coefficient 2.6. The Characteristic Function; Moments Problems
CHAPTER 3
6 6 7 14
2.1. 2.2. 2.3. 2.4.
24 35 43 50
Estimation of the Mean Vector and the Covariance Matrix 3.1. Introduction 3.2. The Maximum Likelihood Estimators of the Mean Vector and the Covariance Matrix 3.3. The Distribution of the Sample Mean Vector; Inference Concerning the Mean When the Covariance Matrix Is Known
XI
59 59 60 68
Xll
CONTENTS
3.4. Theoretical Properties of Estimators of the Mean Vector 3.5. Improved Estimation of the Mean Problems
CHAPTER 4
77 86 96
The Distributions and Uses of Sample Correlation Coefficients 4.1. 4.2. 4.3. 4.4. Introduction Correlation Coefficient of a Bivariate Sample Partial Correlation Coefficients; Conditional Distributions The Multiple Correlation Coefficient Problems
102 102 103 125 134 149
CHAPTER 5
The Generalized r 2 -Statistic 5.1. 5.2. 5.3. 5.4. Introduction Derivation of the Generalized r2-Statistic and Its Distribution Uses of the T2-Statistic The Distribution of T2 Under Alternative Hypotheses; The Power Function 5.5. The Two-Sample Problem with Unequal Covariance Matrices 5.6. Some Optimal Properties of the T2-Test Problems
156 156 157 164 173 175 181 190
CHAPTER 6
Classification of Observations 6.1. The Problem of Classification 6.2. Standards of Good Classification 6.3. Procedures of Classification into One of Two Populations with Known Probability Distributions 6.4. Classification into One of Two Known Multivariate Normal Populations 6.5. Classification into One of Two Multivariate Normal Populations When the Parameters Are Estimated 6.6. Probabilities of Misclassification 6.7. Classification into One of Several Populations
195 195 196 199 204 208 217 224
CONTENTS
Xlll
6.8. Classification into One of Several Multivanate Normal Populations 6.9. An Example of Classification into One of Several Multivanate Normal Populations 6.10. Classification into One of Two Known Multivanate Normal Populations with Unequal Covariance Matrices Problems
CHAPTER 7
228 231 234 241
The Distribution of the Sample Covariance Matrix and the Sample Generalized Variance 7.1. 7.2. 7.3. 7.4. 7.5. 7.6. Introduction The Wishart Distribution Some Properties of the Wishart Distribution Cochran's Theorem The Generalized Variance Distribution of the Set of Correlation Coefficients When the Population Covariance Matrix Is Diagonal 7.7. The Inverted Wishart Distribution and Bayes Estimation of the Covariance Matrix 7.8. Improved Estimation of the Covariance Matrix Problems
CHAPTER 8
244 244 245 252 257 259 266 268 273 279
Testing the General Linear Hypothesis; Multivariate Analysis of Variance 8.1. Introduction 8.2. Estimators of Parameters in Multivariate Linear Regression 8.3. Likelihood Ratio Criteria for Testing Linear Hypotheses About Regression Coefficients 8.4. The Distribution of the Likelihood Ratio Criterion When the Hypothesis Is True 8.5. An Asymptotic Expansion of the Distribution of the Likelihood Ratio Criterion 8.6. Other Criteria for Testing the Linear Hypothesis 8.7. Tests of Hypotheses About Matrices of Regression Coefficients and Confidence Regions
285 285 287 292 298 311 321 333
r
XIV
CONTENTS
8.8. Testing Equality of Means of Several Normal Distributions with Common Covariance Matrix 8.9. Multivariate Analysis of Variance 8.10. Some Optimal Properties of Tests Problems
CHAPTER 9
338 342 349 369
Testing Independence of Sets of Variates 9.1. Introduction 9.2. The Likelihood Ratio Criterion for Testing Independence of Sets of Variates 9.3. The Distribution of the Likelihood Ratio Criterion When the Null Hypothesis Is True 9.4. An Asymptotic Expansion of the Distribution of the Likelihood Ratio Criterion 9.5. Other Criteria 9.6. Step-down Procedures 9.7. An Example 9.8. The Case of Two Sets of Variates 9.9. Admissibility of the Likelihood Ratio Test 9.10. Monotonicity of Power Functions of Tests of Independence of Sets Problems
CHAPTER 10
376 376 376 381 385 387 389 392 394 397 399 402
Testing Hypotheses of Equality of Covariance Matrices and Equality of Mean Vectors and Covariance Matrices 10.1. Introduction 10.2. Criteria for Testing EquaUty of Several Covariance Matrices 10.3. Criteria for Testing That Several Normal Distributions Are Identical 10.4. Distributions of the Criteria 10.5. Asymptotic Expansions of the Distributions of the Criteria 10.6. The Case of Two Populations 10.7. Testing the Hypothesis That a Covariance Matrix Is Proportional to a Given Matrix; The Sphericity Test
404 404 405 408 410 419 422 427
CONTENTS
XV
10.8. Testing the Hypothesis That a Covariance Matrix Is Equal to a Given Matrix 10.9. Testing the Hypothesis That a Mean Vector and a Covariance Matrix Are Equal to a Given Vector and Matrix 10.10. Admissibility of Tests Problems
CHAPTER 11
434 440 443 446
Principal Components 11.1. Introduction 11.2. Definition of Principal Components in the Population 11.3. Maximum Likelihood Estimators of the Principal Components and Their Variances 11.4. Computation of the Maximum Likelihood Estimates of the Principal Components 11.5. An Example 11.6. Statistical Inference 11.7. Testing Hypotheses about the Characteristic Roots of a Covariance Matrix Problems CHAPTER 12 Canonical Correlations and Canonical Variables 12.1. 12.2. 12.3. 12.4. 12.5. 12.6. 12.7. Introduction Canonical Correlations and Variates in the Population Estimation of Canonical Correlations and Variates Statistical Inference An Example Linearly Related Expected Values Simultaneous Equations Models Problems
451 451 452 460 462 465 468 473 477
480 480 481 492 497 500 502 509 519
CHAPTER 13
The Distributions of Characteristic Roots and Vectors 13.1. Introduction 13.2. The Case of Two Wishart Matrices
521 521 522
XVI
CONTENTS
13.3. 13.4. 13.5. 13.6.
The Case of One Nonsingular Wishart Matrix Canonical Correlations Asymptotic Distributions in the Case of One Wishart Matrix Asymptotic Distributions in the Case of Two Wishart Matrices Problems
532 538 540 544 548
CHAPTER 14 Factor Analysis 14.1. Introduction 14.2. The Model 14.3. Maximum Likelihood Estimators for Random Orthogonal Factors 14.4. Estimation for Fixed Factors 14.5. Factor Interpretation and Transformation 14.6. Estimation for Identification by Specified Zeros 14.7. Estimation of Factor Scores Problems
APPENDIX A
550 550 551 557 569 570 574 575 576
Matrix Theory A.l. A.2. A.3. A.4. A.5. Definition of a Matrix and Operations on Matrices Characteristic Roots and Vectors Partitioned Vectors and Matrices Some Miscellaneous Results Gram-Schmidt Orthogonalization and the Solution of Linear Equations
579 579 587 591 596 605
APPENDIX B
Tables 1. Wilks' Likelihood Criterion: Factors C(p, m, M) to Adjust to X2pm where M = n - p + 1 2. Tables of Signiflcance Points for the Lawley-Hotelling Trace Test 3. Tables of Signiflcance Points for the Bartlett-Nanda-Pillai Trace Test 4. Tables of Signiflcance Points for the Roy Maximum Root Test
609 609 616 630 634
CONTENTS
XV11
5. Tables of Significance Points for the Modified Likelihood Ratio Test of Equality of Covariance Matrices Based on Equal Sample Sizes 6. Correction Factors for Significance Points for the Sphericity Test 7. Significance Points for the Modified Likelihood Ratio Test 2 = 2 0 References Index
638 639 641 643 667
Second Edition
T. W. ANDERSON Professor of Statistics and Economics Stanford University
JOHN WILEY & SONS New York Chichester Brisbane
Toronto
Singapore
Contents
CHAPTER 1
Introduction 1.1. Multivariate Statistical Analysis 1.2. The Multivariate Normal Distribution
1 1 3
CHAPTER 2
The Multivariate Normal Distribution Introduction Notions of Multivariate Distributions The Multivariate Normal Distribution The Distribution of Linear Combinations of Normally Distributed Variates; Independence of Variates; Marginal Distributions 2.5. Conditional Distributions and Multiple Correlation Coefficient 2.6. The Characteristic Function; Moments Problems
CHAPTER 3
6 6 7 14
2.1. 2.2. 2.3. 2.4.
24 35 43 50
Estimation of the Mean Vector and the Covariance Matrix 3.1. Introduction 3.2. The Maximum Likelihood Estimators of the Mean Vector and the Covariance Matrix 3.3. The Distribution of the Sample Mean Vector; Inference Concerning the Mean When the Covariance Matrix Is Known
XI
59 59 60 68
Xll
CONTENTS
3.4. Theoretical Properties of Estimators of the Mean Vector 3.5. Improved Estimation of the Mean Problems
CHAPTER 4
77 86 96
The Distributions and Uses of Sample Correlation Coefficients 4.1. 4.2. 4.3. 4.4. Introduction Correlation Coefficient of a Bivariate Sample Partial Correlation Coefficients; Conditional Distributions The Multiple Correlation Coefficient Problems
102 102 103 125 134 149
CHAPTER 5
The Generalized r 2 -Statistic 5.1. 5.2. 5.3. 5.4. Introduction Derivation of the Generalized r2-Statistic and Its Distribution Uses of the T2-Statistic The Distribution of T2 Under Alternative Hypotheses; The Power Function 5.5. The Two-Sample Problem with Unequal Covariance Matrices 5.6. Some Optimal Properties of the T2-Test Problems
156 156 157 164 173 175 181 190
CHAPTER 6
Classification of Observations 6.1. The Problem of Classification 6.2. Standards of Good Classification 6.3. Procedures of Classification into One of Two Populations with Known Probability Distributions 6.4. Classification into One of Two Known Multivariate Normal Populations 6.5. Classification into One of Two Multivariate Normal Populations When the Parameters Are Estimated 6.6. Probabilities of Misclassification 6.7. Classification into One of Several Populations
195 195 196 199 204 208 217 224
CONTENTS
Xlll
6.8. Classification into One of Several Multivanate Normal Populations 6.9. An Example of Classification into One of Several Multivanate Normal Populations 6.10. Classification into One of Two Known Multivanate Normal Populations with Unequal Covariance Matrices Problems
CHAPTER 7
228 231 234 241
The Distribution of the Sample Covariance Matrix and the Sample Generalized Variance 7.1. 7.2. 7.3. 7.4. 7.5. 7.6. Introduction The Wishart Distribution Some Properties of the Wishart Distribution Cochran's Theorem The Generalized Variance Distribution of the Set of Correlation Coefficients When the Population Covariance Matrix Is Diagonal 7.7. The Inverted Wishart Distribution and Bayes Estimation of the Covariance Matrix 7.8. Improved Estimation of the Covariance Matrix Problems
CHAPTER 8
244 244 245 252 257 259 266 268 273 279
Testing the General Linear Hypothesis; Multivariate Analysis of Variance 8.1. Introduction 8.2. Estimators of Parameters in Multivariate Linear Regression 8.3. Likelihood Ratio Criteria for Testing Linear Hypotheses About Regression Coefficients 8.4. The Distribution of the Likelihood Ratio Criterion When the Hypothesis Is True 8.5. An Asymptotic Expansion of the Distribution of the Likelihood Ratio Criterion 8.6. Other Criteria for Testing the Linear Hypothesis 8.7. Tests of Hypotheses About Matrices of Regression Coefficients and Confidence Regions
285 285 287 292 298 311 321 333
r
XIV
CONTENTS
8.8. Testing Equality of Means of Several Normal Distributions with Common Covariance Matrix 8.9. Multivariate Analysis of Variance 8.10. Some Optimal Properties of Tests Problems
CHAPTER 9
338 342 349 369
Testing Independence of Sets of Variates 9.1. Introduction 9.2. The Likelihood Ratio Criterion for Testing Independence of Sets of Variates 9.3. The Distribution of the Likelihood Ratio Criterion When the Null Hypothesis Is True 9.4. An Asymptotic Expansion of the Distribution of the Likelihood Ratio Criterion 9.5. Other Criteria 9.6. Step-down Procedures 9.7. An Example 9.8. The Case of Two Sets of Variates 9.9. Admissibility of the Likelihood Ratio Test 9.10. Monotonicity of Power Functions of Tests of Independence of Sets Problems
CHAPTER 10
376 376 376 381 385 387 389 392 394 397 399 402
Testing Hypotheses of Equality of Covariance Matrices and Equality of Mean Vectors and Covariance Matrices 10.1. Introduction 10.2. Criteria for Testing EquaUty of Several Covariance Matrices 10.3. Criteria for Testing That Several Normal Distributions Are Identical 10.4. Distributions of the Criteria 10.5. Asymptotic Expansions of the Distributions of the Criteria 10.6. The Case of Two Populations 10.7. Testing the Hypothesis That a Covariance Matrix Is Proportional to a Given Matrix; The Sphericity Test
404 404 405 408 410 419 422 427
CONTENTS
XV
10.8. Testing the Hypothesis That a Covariance Matrix Is Equal to a Given Matrix 10.9. Testing the Hypothesis That a Mean Vector and a Covariance Matrix Are Equal to a Given Vector and Matrix 10.10. Admissibility of Tests Problems
CHAPTER 11
434 440 443 446
Principal Components 11.1. Introduction 11.2. Definition of Principal Components in the Population 11.3. Maximum Likelihood Estimators of the Principal Components and Their Variances 11.4. Computation of the Maximum Likelihood Estimates of the Principal Components 11.5. An Example 11.6. Statistical Inference 11.7. Testing Hypotheses about the Characteristic Roots of a Covariance Matrix Problems CHAPTER 12 Canonical Correlations and Canonical Variables 12.1. 12.2. 12.3. 12.4. 12.5. 12.6. 12.7. Introduction Canonical Correlations and Variates in the Population Estimation of Canonical Correlations and Variates Statistical Inference An Example Linearly Related Expected Values Simultaneous Equations Models Problems
451 451 452 460 462 465 468 473 477
480 480 481 492 497 500 502 509 519
CHAPTER 13
The Distributions of Characteristic Roots and Vectors 13.1. Introduction 13.2. The Case of Two Wishart Matrices
521 521 522
XVI
CONTENTS
13.3. 13.4. 13.5. 13.6.
The Case of One Nonsingular Wishart Matrix Canonical Correlations Asymptotic Distributions in the Case of One Wishart Matrix Asymptotic Distributions in the Case of Two Wishart Matrices Problems
532 538 540 544 548
CHAPTER 14 Factor Analysis 14.1. Introduction 14.2. The Model 14.3. Maximum Likelihood Estimators for Random Orthogonal Factors 14.4. Estimation for Fixed Factors 14.5. Factor Interpretation and Transformation 14.6. Estimation for Identification by Specified Zeros 14.7. Estimation of Factor Scores Problems
APPENDIX A
550 550 551 557 569 570 574 575 576
Matrix Theory A.l. A.2. A.3. A.4. A.5. Definition of a Matrix and Operations on Matrices Characteristic Roots and Vectors Partitioned Vectors and Matrices Some Miscellaneous Results Gram-Schmidt Orthogonalization and the Solution of Linear Equations
579 579 587 591 596 605
APPENDIX B
Tables 1. Wilks' Likelihood Criterion: Factors C(p, m, M) to Adjust to X2pm where M = n - p + 1 2. Tables of Signiflcance Points for the Lawley-Hotelling Trace Test 3. Tables of Signiflcance Points for the Bartlett-Nanda-Pillai Trace Test 4. Tables of Signiflcance Points for the Roy Maximum Root Test
609 609 616 630 634
CONTENTS
XV11
5. Tables of Significance Points for the Modified Likelihood Ratio Test of Equality of Covariance Matrices Based on Equal Sample Sizes 6. Correction Factors for Significance Points for the Sphericity Test 7. Significance Points for the Modified Likelihood Ratio Test 2 = 2 0 References Index
638 639 641 643 667
