Multivariate Analysis of Variance

Multivariate analysis of variance
Multivariate analysis of variance (MANOVA) is a generalized form of univariate analysis of variance (ANOVA). It is used when there are two or more dependent variables. It helps to answer : 1. do changes in the independent variable(s) have significant effects on the dependent variables; 2. what are the interactions among the dependent variables and 3. among the independent variables.[1] Where sums of squares appear in univariate analysis of variance, in multivariate analysis of variance certain positive-definite matrices appear. The diagonal entries are the same kinds of sums of squares that appear in univariate ANOVA. The off-diagonal entries are corresponding sums of products. Under normality assumptions about error distributions, the counterpart of the sum of squares due to error has a Wishart distribution. Analogous to ANOVA, MANOVA is based on the product of model variance matrix, Σmodel and inverse of the error variance matrix, , or . The hypothesis that Σmodel = Σresidual implies that the product A∼I[2] . Invariance considerations imply the MANOVA statistic should be a measure of magnitude of the singular value decomposition of this matrix product, but there is no unique choice owing to the multidimensional nature of the alternative hypothesis. The most common[3][4] statistics are summaries based on the roots (or eigenvalues) λp of the A matrix:
 ΛWilks =

Samuel Stanley Wilks'

∏
1... p

(1 / (1 + λp))

distributed as lambda (Λ)
 ΛPillai =

the Pillai-M. S. Bartlett trace,

∑
1... p

(1 / (1 + λp))

 ΛLH =

the Lawley-Hotelling trace,
p

∑ (λ )
1... p 

Roy's greatest root (also called Roy's largest root), ΛRoy = maxp(λp)

Discussion continues over the merits of each, though the greatest root leads only to a bound on significance which is not generally of practical interest. A further complication is that the distribution of these statistics under the null hypothesis is not straightforward and can only be approximated except in a few low-dimensional cases. The bestknown approximation for Wilks' lambda was derived by C. R. Rao. In the case of two groups, all the statistics are equivalent and the test reduces to Hotelling's T-square.
[edit]References

1. 2. 3. 4.

^ Stevens, J. P. (2002). Applied multivariate statistics for the social sciences. Mahwah, NJ:

Lawrence Erblaum. ^ Carey, Gregory. "Multivariate Analysis of Variance (MANOVA): I. Theory". Retrieved 2011-03-22. ^ Garson, G. David. "Multivariate GLM, MANOVA, and MANCOVA". Retrieved 2011-03-22. ^ UCLA: Academic Technology Services, Statistical Consulting Group.. "Stata Annotated Output --

MANOVA". Retrieved 2011-03-22.

[edit]

© Gregory Carey, 1998 MANOVA: I - 1 Multivariate Analysis of Variance (MANOVA): I. Theory Introduction The purpose of a t test is to assess the likelihood that the means for two groups are sampled from the same sampling distribution of means. The purpose of an ANOVA is to test whether the means for two or more groups are taken from the same sampling distribution. The multivariate equivalent of the t test is Hotelling’s T 2 . Hotelling’s T2 tests whether the two vectors of means for the two groups are sampled from the same sampling distribution. MANOVA is the multivariate analogue to Hotelling's T2

. The purpose of MANOVA is to test whether the vectors of means for the two or more groups are sampled from the same sampling distribution. Just as Hotelling's T2 will provide a measure of the likelihood of picking two random vectors of means out of the same hat, MANOVA gives a measure of the overall likelihood of picking two or more random vectors of means out of the same hat. There are two major situations in which MANOVA is used. The first is when there are several correlated dependent variables, and the researcher desires a single, overall statistical test on this set of variables instead of performing multiple individual tests. The second, and in some cases, the more important purpose is to explore how independent variables influence some patterning of response on the dependent variables. Here, one literally uses an analogue of contrast codes on the dependent variables to test hypotheses about how the independent variables differentially predict the dependent variables. MANOVA also has the same problems of multiple post hoc comparisons as ANOVA. An ANOVA gives one overall test of the equality of means for several groups for a single variable. The ANOVA will not tell you which groups differ from which other groups. (Of course, with the judicious use of a priori contrast coding, one can overcome this problem.) The MANOVA gives one overall test of the equality of mean vectors for several groups. But it cannot tell you which groups differ from which other groups on their mean vectors. (As with ANOVA, it is also possible to overcome this problem through the use of a priori contrast coding.) In addition, MANOVA will not tell you which variables are responsible for the differences in mean vectors. Again, it is possible to overcome this with proper contrast coding for the dependent variables. In this handout, we will first explore the nature of multivariate sampling and then explore the logic behind MANOVA. 1. MANOVA: Multivariate Sampling To understand MANOVA and multivariate sampling, let us first examine a MANOVA design. Suppose a researcher in psychotherapy interested in the treatment efficacy of depression randomly assigned clinic patients into four conditions: (1) a placebo control group who received typical clinic psychotherapy and a placebo drug; (2) a placebo cognitive therapy group who received the placebo medication and systematic cognitive psychotherapy; (3) an active antidepressant medication group who received the© Gregory Carey, 1998 MANOVA: I – 2 typical clinic psychotherapy; and (4) an active medication group who received cognitive therapy. This is a (2 x 2) factorial design with medication (placebo versus drug) as one

factor and type of psychotherapy (clinic versus cognitive) as the second factor. Studies such as this one typically collect a variety of measures before treatment, during treatment, and after treatment. To keep the example simple, we will focus only on three outcome measures, say, Beck Depression Index scores (a self-rated depression inventory), Hamilton Rating Scale scores (a clinician rated depression inventory), and Symptom. Checklist for Relatives (a rating scale that a relative completes on the patient--it was made up for this example). High scores on all these measures indicate more depression; low scores indicate normality. The data matrix would look like this: Person Drug Psychotherapy BDI HRS SCR Sally placebo cognitive 12 9 6 Mortimer drug clinic 10 13 7 Miranda placebo clinic 16 12 4 . . . . . Waldo drug cognitive 8 3 2

For simplicity, assume the design is balanced with equal numbers of patients in all four conditions. A univariate ANOVA on any single outcome measure would contain three effects, a main effect for psychotherapy, a mean effect for medication, and an interaction between psychotherapy and medication. The MANOVA will also contain the same three effects. The univariate ANOVA main effect for psychotherapy tells whether the clinic versus the cognitive therapy groups have different means, irrespective of their medication. The MANOVA main effect for psychotherapy tells whether the clinic versus the cognitive therapy group have different mean vectors irrespective of their medication; the vectors in this case are the (3 x 1) column vectors of (BDI, HRS, and SCR) means. The univariate ANOVA for medication tells whether the placebo group has a different mean from the drug group irrespective of psychotherapy. The MANOVA main effect for medication tells whether the placebo group has a different mean vector from the drug group irrespective of psychotherapy. The univariate ANOVA interaction tells whether the four means for a single variable differ from the value predicted from knowledge of the main effects of psychotherapy and drug. The MANOVA interaction term tells whether the four mean vectors differ from the vector predicted from knowledge of the main effects of psychotherapy and drug. If you are coming to the impression that a MANOVA has all the properties as an ANOVA, you are correct. The only difference is that an ANOVA deals with a (1 x 1) mean vector for any group while a MANOVA deals with a (p x 1) vector for any group, p being the number of dependent variables, 3 in our example. Now let's think for a minute. What is the variance-covariance matrix for a single variable? It is a (1 x 1) matrix that has only one element, the variance of the variable. What is the variance-covariance matrix for p variables? It is now a (p x p) matrix with the variances on the diagonal and the covariances© Gregory Carey, 1998 MANOVA: I – 3 on the off diagonals. The

ANOVA partitions the (1 x 1) covariance matrix into a part due to error and a part due to hypotheses (the two main effects and the interaction term as described above). Or for our example, we can write Vt = Vp + Vm + V(p*m) + Ve (1.1) Equation (1.1) states that the total variability (Vt ) is the sum of the variability due to psychotherapy (Vp), the variability due to medication (Vm), the variability due to the interaction of psychotherapy with medication (V(p*m) ), and error variability (Ve ). The MANOVA likewise partitions its (p x p) covariance matrix into a part due to error and a part due to hypotheses. Consequently, the MANOVA for our example will have a (3 x 3) covariance matrix for total variability, a (3 x 3) covariance matrix due to psychotherapy, a (3 x 3) covariance matrix due to edication, a (3 x 3) covariance matrix due to the interaction of psychotherapy with medication, and a (3 x 3) covariance matrix for error. Or we can now write Vt = Vp + Vm + V(p*m) + Ve (1.2) where V now stands for the appropriate (3 x 3) matrix. Note how equation (1.2) equals (1.1) except that (1.2) is in matrix form. Actually, if we considered all the variances in a univariate ANOVA as (1 by 1) matrices and wrote equation (1.1) in matrix form, we would have equation (1.2). Let's now interpret what these matrices in (1.2) mean. The Ve matrix will look like this BDI HRS CSR BDI Ve1 cov(e1,e2) cov(e1,e3) HRS cov(e2,e1) Ve2 cov (e2,e3) CSR cov(e3,e1) cov(e3,e2) Ve3 The diagonal elements are the error variances. They have the same meaning as the error variances in their univariate ANOVAs. In a univariate ANOVA the error variance is an average variability within the four groups. The error variance in MANOVA has the same meaning. That is, if we did a univariate ANOVA for the Beck Depression Inventory, the error variance would be the mean squares within groups. ve1 would be the same means squares within groups; it would literally be the same number as in the univariate analysis. The same would hold for the mean squares within groups for the HRS and the CSR. The only difference then is in the off diagonals. The off diagonal covariances for the error matrix must all be interpreted as within group covariances. That is, cov(e1,e2) tells us the extent to which individuals within a group who have high BDI scores also tend to have high HRS scores. Because this is a covariance matrix, the matrix can be scaled to correlations to ease inspection. For example, corr(e1,e2) = cov(e1,e2) (ve1ve2) -1/2 .© Gregory Carey, 1998 MANOVA: I – 4 I suggest that you always have this error covariance matrix and its correlation matrix printed and inspect the correlations. In theory, if you could measure a sufficient number of factors, covariates, etc. so that the only remaining variability is due to random noise, then these correlations should all go to 0. Inspection of these correlations is often a sobering experience because it demonstrates how far we have to go to be able to predict behavior. What about the other matrices? In MANOVA, they will all have their analogues in the univariate ANOVA. For example, the variance in BDI due to psychotherapy calculated from a univariate ANOVA of the BDI would be the first diagonal element in the Vp

matrix. The variance of HRS calculated from a univariate ANOVA is the second diagonal element in Vp. The variance in CSR due to the interaction between psychotherapy and drug as calculated from a univariate ANOVA will be the third diagonal element in V(p*m) . The off diagonal elements are all covariances and should be interpreted as between group covariances. That is, in Vp, cov(1,2) = cov(BDI, HRS) tells us whether the psychotherapy group with the highest mean score on BDI also has the highest mean score on the HRS. If, for example, the cognitive therapy were more efficacious than the clinic therapy, then we should expect that all the covariances in Vp be large and positive. Again, cov(2,3) = cov(HRS, CSR) in the V(p*n) matrix has the following interpretation: if we control for the main effects of psychotherapy and medication, then do groups with high average scores on the Hamilton also tend to have high average scores on the relative's checklist? In theory, if there were no main effect for psychotherapy on any of the measures, then all the elements of Vp will be 0. If there were no main effect for medication, then Vm will be all 0's. And if there were no interaction, then all of V(p*m) would be 0's. It makes sense to have these matrices calculated and printed out so that you can inspect them. However, just as most programs for univariate ANOVAs do not give you the variance components, most computer programs for MANOVA do not give you the variance component matrices. You can, however, calculate them by hand 2. Understanding MANOVA Understanding of MANOVA requires understanding of three basic principles. They are: · An understanding of univariate ANOVA. · Remembering that mathematicians work very hard to be lazy. · A little bit of matrix algebra. Each of these topics will now be convered in detail. 2.1 Understanding Univariate ANOVA. Let us review univariate ANOVA by examining the expected results of a simple oneway ANOVA under the null hypothesis. The overall logic of ANOVA is to obtain two different estimates of the population variance; hence, the term analysis of variance.© Gregory Carey, 1998 MANOVA: I – 5 The first estimate is based on the variance within groups. The second estimate of the variance is based on the variance of the means of the groups. Let us examine the logic behind this by referring to an example of a oneway ANOVA. Suppose we select four groups of equal size and measure them on a single variable. Let n denote the sample size within each group. The null hypothesis assumes that the scores for individuals in each of the four groups are all sampled from a single normal distribution with mean m and variance s 2 . Thus, the expected value for the mean of the first group is m and the expected value of the variance of the first group is s 2 , the expected value for the mean of the second group is m and the expected value of the variance of the second group is s 2 , etc. The observed statistics and their expected values for this ANOVA are given in Table 1. Table 1. Observed and expected means and variances of four groups in a oneway ANOVA under the null hypothesis. Group 1 2 3 4 Sample Size: n n n n Means: Observed X 1

X 2 X 3 X 4 Expected m m m m Variances: Observed s 1 2 s 2 2 s 3 2 s 4 2 Expected s2s2s2s2 Now concentrate on the rows for the variances in Table 1. There are four observed variances, one for each group, and they all have expected values of s2. Instead of having four different estimates of s2, we can obtain a better overall estimate of s 2 by simply taking the average of the four estimates. That is, ) ss22=s12+ s22+ s32+ s424(2.1)The notation) ss22 is used to denote that this is the estimate of s2 derived from the observed variances. (Note that the only reason we could add up the four variances is because the four groups are independent. Were they dependent in some way, this could not be done.)Now examine the four means. A very famous theorem in statistics, the central limit theorem, states that when scores are normally distributed with mean m and variance s 2, then the the sampling distribution of means based on size n will be normally© Gregory Carey, 1998 MANOVA: I – 6 distributed with an overall mean of m and variance s2/n. Thus, the four means in Table 1 will be sampled from a single normal distribution with mean m and variance s2/n. A second estimate of s2 may now be obtained by going through three steps: (1) treat the four means as raw scores, (2) calculate the variance of the four means, and (3) multiplyingthe result by n. Let x denote the overall mean of the four means, then sx2= n (xi - x )2i=14å4 -1(2.2)where )sx2 denotes the estimate of s2 based on the means and not the variance of the means. We now have two separate estimates of s 2, the first based on the within group variances ()ss22) and the second based on the means ()sx2 sx 2 ). If we take a ratio of the two estimates, we expect a value close to 1.0, or E) s x 2 ) s s 2 2 æ è ç ö

ø ÷ » 1. (2.3) This is the logic of the simple oneway ANOVA, although it is most often expressed in different terms. The estimate

) s s 2 2 from equation (2.1) is the mean squares within groups. The estimate

) s x 2 from (2.2) is the mean squares between groups. And the ratio in (2.3) is the F ratio expected under the null hypothesis. In order to generalize to any number of groups, say g groups, we would perform the same step but substitute g in place of 4 in Equation (2.2). Now the above derivations all pertain to the null hypothesis. The alternative hypothesis states that at least one of the four groups has been sampled from a different normal distribution than the other three means. Here, for the sake of exposition, it is

assumed that scores in the four groups are sampled from different normal distributions with respective means of m1, m2, m3, and m4, but with the same variance, s 2 . Because the variances do not differ, the estimate derived from the observed group variances in equation (2.1), or

) s s 2 2 , will remain a valid estimate of s 2 . However, the variance derived from the means using equation (2.2) is no longer an estimate of s 2 . If we performed the calculation on the right hand side of Equation (2.2), the expected results would be

En (x i-x) 2

i =1 4 å 4 -1 æ è ç ö ø ÷ ÷ ÷ = nsm 2 + ) s x 2 . (2.4)© Gregory Carey, 1998 MANOVA: I - 7 Consequently, the expectation of the F ratio becomes E(F) = nsm 2 + )

s x 2 ) s s 2 2 = ) s x 2 ) s s 2 2 + nsm 2 ) s s 2 2

»1 + nsm 2 ) s s 2 2 . (2.5) Instead of having an expectation around 1.0 (which F has under the null hypothesis), the expectation under the alternative hypothesis will be sometihing greater than 1.0. Hence, the larger the F statistic, the more likely that the null hypothesis is false. 2.2 Mathematicians Are Lazy The second step in understanding MANOVA is the recognition that mathematicians are lazy. Mathematical indolence, as applied to MANOVA, is apparent because mathematical statisticians do not bother to express the information in terms of the estimates of variance that were outlined above. Instead, all the information is expressed in terms of sums of squares and cross products. Although this may seem quite foreign to the student, it does save steps in calculations--something that was important in the past when computers were not available. We can explore this logic by once again returning to a simple oneway ANOVA. Recall that the two estimates of the population variance in a oneway ANOVA are

termed mean squares. Computationally, mean squares denote the quantity resulting from dividing the sum of squares by its associated degrees of freedom. The between group mean squares--which will now be called the hypothesis mean squares, is

MSh = SSh df h and the within group mean squares--which will now be called the error mean squares, is

MSe = SSe df e . The F statistic for any hypothesis is defined as the mean squares for the hypothesis divided by the mean squares for error. Using a little algebra, the F statistic can be shown to be the product of two ratios. The first ratio is the degrees of freedom for error divided by the degrees of freedom for the hypothesis. The second ratio is the sum of squares for the hypothesis divided by the sum of squares for error. That is,© Gregory Carey, 1998 MANOVA: I - 8

F=

MSh MSe = SS h df h SSe df e = df e df h SSh SSe . The role of mathematical laziness comes about by developing a new statistic-called the A statistic here--that simplifies matters by removing the step of calculating the mean squares. The A statistic is simply the F statistic multiplied by the degrees of freedom for the hypothesis and divided by the degrees of freedom for error, or

A= df h df

e F= df h df e df e df h æ è ç ö ø ÷ SSh SSe = SSh SSe . The chief advantage of using the A statistic is that one never has to go through the trouble of calculating mean squares. Hence, given the overriding principle of mathematical indolence, that, of course, must be adhered to at all costs in statistics, it is preferable to

develop ANOVA tables using the A statistic instead of the F statistic and to develop tables for the critical values of the A statistic to replace tables of critical values for the F statistic. Accordingly, we can refer to traditional ANOVA tables as “stupid,” and to ANOVA tables using the F statistic as “enlightened.” The following tables demonstrate the difference between a traditional, “stupid” ANOVA and a modern, “enlightened” ANOVA. Stupid ANOVA Table Source Sums of Squares Degrees of Freedom Mean Squares F p Hypothesis SSh dfh MSh Error SSe df e MSe Total SSt df t Critical Values of F (a = .05) (Stupid) df e (numerator) dfh 1 2 3 4 5 1 161.45 199.50 215.71 224.58 230.16 2 18.51 19.00 19.16 19.25 19.30

3 10.13 9.55 9.28 9.12 9.01© Gregory Carey, 1998 MANOVA: I - 9 Enlightened ANOVA Table Source Sums of Squares Degrees of Freedom A p Hypothesis SSh dfh Error SSe df e Total SSt df t Critical Values of A (a = .05)(Enlightened) df e (numerator) dfh 1 2 3 4 5 1 161.45 399.00 647.13 898.32 1150.80 2 9.26 19.00 28.74 38.50 48.25 3 3.38 6.37 9.28 12.16 15.02 2.3. Vectors and Matrices The final step in understanding MANOVA is to express the information in terms of vectors and matrices. Assume that instead of a single dependent variance in the oneway ANOVA, there are three dependent variables. Under the null hypothesis, it is assumed that scores on the three variables for each of the four groups are sampled from a trivariate normal distribution mean vector

m= m1 m2 m3 æ è ö ø and variance-covariance matrix

S= s1 2 r12s1s2 r13s1s3 r12s1s2 s2 2 r23s2s3 r13s1s3 r23s2s3 s3 2 æ è ç ç ö ø

. (Recall that the quantity

r12s1s2 equals the covariance between variables 1 and 2.) Under the null hypothesis, the scores for all those in group 1 will be sampled from this distribution, as will the scores for all those individuals in groups 2, 3, and 4. The observed and expected values for the four groups are given in Table 2.© Gregory Carey, 1998 MANOVA: I - 10 Table 2. Observed and expected statistics for the mean vectors and the variancecovariance matrices of four groups in a oneway MANOVA under the null hypothesis. Group 1234 Sample Size: n n n n Mean Vector: Observed x 1 x 2 x 3 x 4 Expected m m m m Covariance Matrix: Observed S1 S2 S3 S4 Expected S S S S Note the resemblance between Tables 1 and 2. The only difference is that Table 2 is written in matrix notation. Indeed, if we consider the elements in Table 1 as (1 by 1) vectors or (1 by 1) matrices and then rewrite Table 1, we would get Table 2! Once again, how can we obtain different estimates of S? Again, concentrate on the rows marked variances in Table 2. The easiest way to estimate S is to add up the covariance matrices for the four groups and divide by 4--or, in other words, take the average observed covariance matrix:

S ˆ w= S1 + S2 + S3 + S4 4 . (2.6) Note how (2.6) is identical to (2.1) except that (2.6) is expressed in matrix notation. What about the means? Under the null hypothesis, the means will be sampled from a trivariate normal distribution with mean vector m and covariance matrix S/n. Consequently, to obtain an estimate of S based on the mean vectors for the four groups, we proceed with the same logic as that in the oneway ANOVAiven in section 2.1, but now apply it to the vectors of means. That is, treat the means as if they were raw scores and calculate the covariance matrix for the three “variables;” then multiply this result by n. Let

X ij denote the mean for the ith group on the jth variable. The data would look like this: X 11 X 12 X 13 X

21 X 22 X 23 X 31 X 32 X 33 X 41 X 42 X 43© Gregory Carey, 1998 MANOVA: I - 11 This is identical to a data matrix with four observations and three variables. In this case, the observations are the four groups and the “scores” or numbers are the means. Let SSCPX denote the sums of squares and cross products matrix used to calculate the covariance matrix for these three variables. Then the estimate of S based on the means of the four groups will be ˆ S b=n SSCPX 4 -1 . (2.7) Note how (2.7) is identical in form to (2.2) except that (2.7) is expressed in matrix notation.

We now have two different estimates of S. The first,

ˆ S w , is derived from the average covariance matrix within the groups, and the second,

ˆ S b , is derived from the covariance matrix for the group mean vectors. Because both of them measure the same quantity, we expect E( ˆ S b ˆ S w -1 ) = I. (2.8) or an identity matrix. Note that (2.8) is identical to (2.3) except that (2.8) is written in matrix notation. If the matrices in (2.8) were simply (1 by 1) matrices then their

expectation would be a (1 by 1) identity matrix or simply 1.0. Just what we got in (2.3)! Now examine the alternative hypothesis. Under the alternative hypothesis it is assumed that each group is sampled from a multivariate normal distribution that has the same covariance matrix, say S. Consequently, the average of the covariance matrices for the four groups in equation (2.6) remains an estimate of S. However, under the alternate hypothesis, the mean vector for at least one group is sampled from a multivariate normal with a different mean vector than the other groups. Consequently, the covariance matrix for the observed mean vectors will reflect the covariance due to true mean differences, say Sm , plus the covariance matrix for sampling error, S/n. Thus, multiplying the covariance matrix of the means by n now gives

nSm + S . (2.9) Once again, (2.9) is the same as (2.4) except for its expression in matrix notation. What is the expectation of the estimate from the observed means postmultiplied by the estimate from the observed covariance matrices under the alternative hypothesis? Substituting the expression in (2.9) for

ˆ S

b in Equation (2.8) gives E( ˆ S b ˆ S w -1 ) = (nSm + S)S -1 = nSmS + SS -1 = nSmS + I . (2.10)© Gregory Carey, 1998 MANOVA: I - 12 Although it is a bit difficult to see how, (2.10) is the same as (2.5) except that (2.10) is expressed in matrix notation. Note that (2.10) is the sum of an identity matrix and another term. The identity matrix will have 1's on the diagonal, so the diagonals of the result will always be "1 + something." That "something" will always be a positive number [for technical reasons in matrix algebra]. Consequently, the diagonals in (2.10) will always be greater than 1.0. Again, if we consider the matrices in (2.10) as (1 by 1) matrices, we can verify that the expectation will always be greater than 1.0, just as we found in (2.5). This exercise demonstrates how MANOVA is a natural extension of ANOVA.

The only remaining point is to add the laziness of mathematical statisticians. In ANOVA, we saw how this laziness saved a compuational step by avoiding calculations of the means squares (which is simply another term for the variance) and expressing all the information in terms of the sums of squares. The same applies to MANOVA, instead of calculating the mean squares and mean products matrix (which is simply another term for a covariance matrix), MANOVA avoids this step by expressing all the information in terms of the sums of squares and cross products matrices. The A statistic developed in section 2.2 was simply the ratio of the sum of squares for an hypothesis and the sum of squares for error. Let H denote the hypothesis sums of squares and cross products matrix, and let E denote the error sums of squares and cross products matrix. The multivariate equivalent of the A statistic is the matrix A which is A = HE -1 (2.11) Verify how Equation 2.11 is the matrix analogue of the A statistic given in section 2.2. Notice how mean squares (or, in other terms, covariance matrices) disappear from MANOVA just as they did for ANOVA. All hypothesis tests may be performed on matrix A. Parenthetically, note that because both H and E are symmetric, HE -1 =E -1

H. This is one special case where the order of matrix multiplication does not matter. 3.0. Hypothesis Testing in MANOVA All current MANOVA tests are made on A = E -1 H. That's the good news. The bad news is that there are four different multivariate tests that are made on E -1 H. Each of the four test statistics has its own associated F ratio. In some cases the four tests give an exact F ratio for testing the null hypothesis and in other cases the F ratio is approximated. The reason for four different statistics and for approximations is that the mathematics of MANOVA get so complicated in some cases that no one has ever been able to solve them. (Technically, the math folks can't figure out the sampling distribution of the F statistic in some multivariate cases.) To understand MANOVA, it is not necessary to understand the derivation of the statistics. Here, all that is mentioned is their names and some properties. In terms of notation, assume that there are q dependent variables in the MANOVA, and let li denote the ith eigenvalue of matrix A which, of course, equals HE -1 .© Gregory Carey, 1998 MANOVA: I - 13 The first statistic is Pillai's trace. Some statisticians consider it to be the most powerful and most robust of the four statistics. The formula is

Pillai's trace = trace[H(H + E) -1 ]= li 1+ l i i=1 q å . (3.1) The second test statistic is Hotelling-Lawley's trace. Hotelling-Lawley's trace = trace(A ) = trace(HE -1 ) = li i=1 q å . (3.2) The third is Wilk's lambda (L). (Here, the upper case, Greek L is used for Wilk’s lambda to avoid confusion with the lower case, Greek l often used to denote an eigenvalue. However, many texts use the lower case lambda as the notation for Wilk’s lambda.) Wilk’s L was the first MANOVA test statistic developed and is very important for several multivariate procedures in addition to MANOVA. Wilk's lambda = L = |E| |H+E| = 1 1+ li

i=1 q Õ . (3.3) The quantity (1 - L) is often interpreted as the proportion of variance in the dependent variables explained by the model effect. However, this quantity is not unbiased and can be quite misleading in small samples. The fourth and last statistic is Roy's largest root. This gives an upper bound for the F statistic. Roy's largest root = max(li ). (3.4) or the maximum eigenvalue of A = HE -1 . (Recall that a "root" is another name for an eigenvalue.) Hence, this statistic could also be called Roy's largest eigenvalue. (In case you know where Roy's smallest root is, please let him know.) Note how all the formula in equations (3.1) through (3.4) are based on the eigenvalues of A = HE -1 . This is the major reason why statstical programs such as SAS print out the eigenvalues and eigenvectors of A = HE -1 . Once the statistics in (3.1) through (3.4) are obtained, they are translated into F statistics in order to test the null hypothesis. The reason for this translation is identical

to the reason for converting Hotelling's T 2 --the easy availability of published tables of the F distribution. The important issue to recognize is that in some cases, the F statistic is exact and in other cases it is approximate. Good statistical packages will inform you whether the F is exact or approximate. In some cases, the four will generate identical F statistics and identical probabilities. In other's they will differ. When they differ, Pillai's trace is often used because it is the most powerful and robust. Because Roy's largest root is an upper bound© Gregory Carey, 1998 MANOVA: I - 14 on F, it will give a lower bound estimate of the probability of F. Thus, Roy's largest root is generally disregarded when it is significant but the others are not significant.

Multivariate Analysis of Variance (MANOVA) Aaron French, Marcelo Macedo, John Poulsen, Tyler Waterson and Angela Yu Keywords: MANCOVA, special cases, assumptions, further reading, computations Introduction Multivariate analysis of variance (MANOVA) is simply an ANOVA with several dependent variables. That is to say, ANOVA tests for the difference in means between two or more groups, while MANOVA tests for the difference in two or more vectors of means. For example, we may conduct a study where we try two different textbooks, and we are interested in the students' improvements in math and physics. In that case, improvements in math and physics are the two dependent variables, and our

hypothesis is that both together are affected by the difference in textbooks. A multivariate analysis of variance (MANOVA) could be used to test this hypothesis. Instead of a univariate F value, we would obtain a multivariate F value (Wilks' λ) based on a comparison of the error variance/covariance matrix and the effect variance/covariance matrix. Although we only mention Wilks' λ here, there are other statistics that may be used, including Hotelling's trace and Pillai's criterion. The "covariance" here is included because the two measures are probably correlated and we must take this correlation into account when performing the significance test. Testing the multiple dependent variables is accomplished by creating new dependent variables that maximize group differences. These artificial dependent variables are linear combinations of the measured dependent variables. Research Questions The main objective in using MANOVA is to determine if the response variables (student improvement in the example mentioned above), are altered by the observer’s manipulation of the independent variables. Therefore, there are several types of research questions that may be answered by using MANOVA: 1) What are the main effects of the independent variables? 2) What are the interactions among the independent variables? 3) What is the importance of the dependent variables?4) What is the strength of association between dependent variables? 5) What are the effects of covariates? How may they be utilized? Results If the overall multivariate test is significant, we conclude that the respective effect (e.g., textbook) is significant. However, our next question would of course be whether only math skills improved, only physics skills improved, or both. In fact, after obtaining a significant multivariate test for a particular main effect or interaction, customarily one would examine the univariate F tests for each variable to interpret

the respective effect. In other words, one would identify the specific dependent variables that contributed to the significant overall effect. MANOVA is useful in experimental situations where at least some of the independent variables are manipulated. It has several advantages over ANOVA. First, by measuring several dependent variables in a single experiment, there is a better chance of discovering which factor is truly important. Second, it can protect against Type I errors that might occur if multiple ANOVA’s were conducted independently. Additionally, it can reveal differences not discovered by ANOVA tests. However, there are several cautions as well. It is a substantially more complicated design than ANOVA, and therefore there can be some ambiguity about which independent variable affects each dependent variable. Thus, the observer must make many potentially subjective assumptions. Moreover, one degree of freedom is lost for each dependent variable that is added. The gain of power obtained from decreased SS error may be offset by the loss in these degrees of freedom. Finally, the dependent variables should be largely uncorrelated. If the dependent variables are highly correlated, there is little advantage in including more than one in the test given the resultant loss in degrees of freedom. Under these circumstances, use of a single ANOVA test would be preferable. Assumptions Normal Distribution: - The dependent variable should be normally distributed within groups. Overall, the F test is robust to non-normality, if the non-normality is caused by skewness rather than by outliers. Tests for outliers should be run before performing a MANOVA, and outliers should be transformed or removed. Linearity - MANOVA assumes that there are linear relationships among all pairs of dependent variables, all pairs of covariates, and all dependent variable-covariate pairs in each cell. Therefore, when the relationship deviates from linearity, the power

of the analysis will be compromised. Homogeneity of Variances: - Homogeneity of variances assumes that the dependent variables exhibit equal levels of variance across the range of predictor variables. Remember that the error variance is computed (SS error) by adding up the sums of squares within each group. If the variances in the two groups are different from each other, then adding the two together is not appropriate, and will not yield an estimate of the common within-group variance. Homoscedasticity can be examined graphically or by means of a number of statistical tests. Homogeneity of Variances and Covariances: - In multivariate designs, with multiple dependent measures, the homogeneity of variances assumption described earlier also applies. However, since there are multiple dependent variables, it is also required that their intercorrelations (covariances) are homogeneous across the cells of the design. There are various specific tests of this assumption. Special Cases Two special cases arise in MANOVA, the inclusion of within-subjects independent variables and unequal sample sizes in cells. Unequal sample sizes - As in ANOVA, when cells in a factorial MANOVA have different sample sizes, the sum of squares for effect plus error does not equal the total sum of squares. This causes tests of main effects and interactions to be correlated. SPSS offers and adjustment for unequal sample sizes in MANOVA. Within-subjects design - Problems arise if the researcher measures several different dependent variables on different occasions. This situation can be viewed as a withinsubject independent variable with as many levels as occasions, or it can be viewed as separate dependent variables for each occasion. Tabachnick and Fidell (1996) provide examples and solutions for each situation. This situation often lends itself to the use of profile analysis, which is explained below.

Additional Limitations Outliers - Like ANOVA, MANOVA is extremely sensitive to outliers. Outliers may produce either a Type I or Type II error and give no indication as to which type of error is occurring in the analysis. There are several programs available to test for univariate and multivariate outliers. Multicollinearity and Singularity - When there is high correlation between dependent variables, one dependent variable becomes a near-linear combination of the other dependent variables. Under such circumstances, it would become statistically redundant and suspect to include both combinations. MANCOVA MANCOVA is an extension of ANCOVA. It is simply a MANOVA where the artificial DVs are initially adjusted for differences in one or more covariates. This can reduce error "noise" when error associated with the covariate is removed. For Further Reading: Cooley, W.W. and P. R. Lohnes. 1971. Multivariate Data Analysis. John Wiley & Sons, Inc. George H. Dunteman (1984). Introduction to multivariate analysis. Thousand Oaks, CA: Sage Publications. Chapter 5 covers classification procedures and discriminant analysis. Morrison, D.F. 1967. Multivariate Statistical Methods. McGraw-Hill: New York. Overall, J.E. and C.J. Klett. 1972. Applied Multivariate Analysis. McGraw-Hill: New York. Tabachnick, B.G. and L.S. Fidell. 1996. Using Multivariate Statistics. Harper Collins College Publishers: New York. Webpages: Site Link

Statsoft text entry on MANOVA http://www.statsoft.com/textbook/stathome.html EPA Statistical Primer http://www.epa.gov/bioindicators/primer/html/manova.html Introduction to MANOVA http://ibgwww.colorado.edu/~carey/p7291dir/handouts/manova1.pdf Practical guide to MANOVA for SAS http://ibgwww.colorado.edu/~carey/p7291dir/handouts/manova2.pdf

Computations First, the total sum-of-squares is partitioned into the sum-of-squares between groups (SSbg) and the sum-of-squares within groups (SSwg): SStot = SSbg + SSwg This can be expressed as: The SSbg is then partitioned into variance for each IV and the interactions between them. In a case where there are two IVs (IV1 and IV2), the equation looks like this: Therefore, the complete equation becomes: Because in MANOVA there are multiple DVs, a column matrix (vector) of values for

each DV is used. For two DVs (a and b) with n values, this can be represented: Similarly, there are column matrices for IVs - one matrix for each level of every IV. Each matrix of IVs for each level is composed of means for every DV. For "n" DVs and "m" levels of each IV, this is written: Additional matrices are calculated for cell means averaged over the individuals in each group. Finally, a single matrix of grand means is calculated with one value for each DV averaged across all individuals in matrix. Differences are found by subtracting one matrix from another to produce new matrices. From these new matrices the error term is found by subtracting the GM matrix from each of the DV individual scores: Next, each column matrix is multiplied by each row matrix: These matrices are summed over rows and groups, just as squared differences are summed in ANOVA. The result is an S matrix (also known as: "sum-of-squares and cross-products," "cross-products," or "sum-of-products" matrices.) For a two IV, two DV example:

Stot = SIV1 + SIV2 + Sinteraction + Swithin-group error

Determinants (variance) of the S matrices are found. Wilks’ λ is the test statistic preferred for MANOVA, and is found through a ratio of the determinants: An estimate of F can be calculated through the following equations:

Where,

Finally, we need to measure the strength of the association. Since Wilks’ λ is equal to the variance not accounted for by the combined DVs, then (1 – λ) is the variance that is accounted for by the best linear combination of DVs.

However, because this is summed across all DVs, it can be greater than one and therefore less useful than:

Other statistics can be calculated in addition to Wilks’ λ. The following is a short list of some of the popularly reported test statistics for MANOVA: • Wilks’ λ = pooled ratio of error variances to effect variance plus error variance • This is the most commonly reported test statistic, but not always the best choice. • Gives an exact F-statistic • Hotelling’s trace = pooled ratio of effect variance to error variance ∑ = = s i Ti 1 λ

• Pillai-Bartlett criterion = pooled effect variances • Often considered most robust and powerful test statistic. • Gives most conservative F-statistic. ∑ = + = S ii i V 1 1λ λ • Roy’s Largest Root = largest eigenvalue o Gives an upper-bound of the F-statistic. o Disregard if none of the other test statistics are significant. MANOVA works well in situations where there are moderate correlations between DVs. For very high or very low correlation in DVs, it is not suitable: if DVs are too correlated, there is not enough variance left over after the first DV is fit, and if DVs are uncorrelated, the multivariate test will lack power anyway, so why sacrifice degrees of freedom? Multivariate analysis of variance (MANOVA) Aaron French and John Poulsen Keywords: MANCOVA, special cases, assumptions, further reading,

computations Introduction Multivariate analysis of variance (MANOVA) is simply an ANOVA with several dependent variables. For example, we may conduct a study where we try two different textbooks, and we are interested in the students' improvements in math and physics. In that case, improvements in math and physics are the two dependent variables, and our hypothesis is that both together are affected by the difference in textbooks. A multivariate analysis of variance (MANOVA) could be used to test this hypothesis. Instead of a univariate F value, we would obtain a multivariate F value (Wilks' lambda) based on a comparison of the error variance/covariance matrix and the effect variance/covariance matrix. Although we only mention Wilks' lambda here, there are other statistics that may be used, including Hotelling's trace and Pillai's criterion. The "covariance" here is included because the two measures are probably correlated and we must take this correlation into account when performing the significance test. Testing the multiple dependent variables is accomplished by creating new dependent variables that maximize group differences. These artificial dependent variables are linear combinations of the measured dependent variables. Results If the overall multivariate test is significant, we conclude that the respective effect (e.g., textbook) is significant. However, our next question would of course be whether only math skills improved, only physics skills improved, or both. In fact, after obtaining a significant multivariate test for a particular main effect or interaction, customarily one would examine the univariate F tests for each variable to interpret the respective effect. In other words, one would identify the specific dependent variables that contributed to the significant overall effect.

MANOVA is useful in experimental situations where at least some of the independent variables are manipulated. It has several advantages over ANOVA. First, by measuring several dependent variables in a single experiment, there is a better chance of discovering which factor is truly important. Second, it can protect against Type I errors that might occur if multiple ANOVA’s were conducted independently. Additionally, it can reveal differences not discovered by ANOVA tests. However, there are several cautions as well. It is a substantially more complicated design than ANOVA, and therefore there can be some ambiguity as to which independent variable affects each dependent variable. Moreover, one degree of freedom is lost for each dependent variable that is added. The gain of power obtained from decreased SS error may be offset by the loss in these degrees of freedom. Finally, the dependent variables should be largely uncorrelated. If the dependent variables are highly correlated, there is little advantage in including more than one in the test given the resultant loss in degrees of freedom. Assumptions Normal Distribution: The dependent variable should be normally distributed within groups. Overall, the F test is robust to non-normality if it is caused by skewness rather than outliers. Tests for outliers should be run before performing a MANOVA, and outliers should be transformed or removed. Homogeneity of Variances: Homogeneity of variances assumes that the dependent variables exhibit equal levels of variance across the range of predictor variables. Remember that the error variance is computed (SS error) by adding up the sums of squares within each group. If the variances in the two groups are different from each other, then

adding the two together is not appropriate, and will not yield an estimate of the common within-group variance. Homoscedasticity can be examined graphically or by means of a number of statistical tests. Homogeneity of Variances and Covariances: In multivariate designs, with multiple dependent measures, the homogeneity of variances assumption described earlier also applies. However, since there are multiple dependent variables, it is also required that their intercorrelations (covariances) are homogeneous across the cells of the design. There are various specific tests of this assumption. Special Cases Two special cases arise in MANOVA, the inclusion of within-subjects independent variables and unequal sample sizes in cells. Unequal sample sizes As in ANOVA, when cells in a factorial MANOVA have different sample sizes, the sum of squares for effect plus error does not equal the total sum of squares.This causes tests of main effects and interactions to be correlated. SPSS offers and adjustment for unequal sample sizes in MANOVA. Within-subjects design Problems arise if the researcher measures several different dependent variables on different occasions. This situation can be viewed as a within-subject independent variable with as many levels as occasions. Or, it can be viewed as a separate dependent variables for each occasion. Tabachnick and Fidell (1996) provide examples and solutions for each situation. MANCOVA MANCOVA is an extension of ANCOVA. It is simply a MANOVA where the artificial DVs are initially adjusted for differences in one or more covariates. This can reduce error "noise" when error associated with the covariate is removed.

For Further Reading: Cooley, W.W. and P. R. Lohnes. 1971. Multivariate Data Analysis. John Wiley & Sons, Inc. George H. Dunteman (1984). Introduction to multivariate analysis. Thousand Oaks, CA: Sage Publications. Chapter 5 covers classification procedures and discriminant analysis. Morrison, D.F. 1967. Multivariate Statistical Methods. McGraw-Hill: New York. Overall, J.E. and C.J. Klett. 1972. Applied Multivariate Analysis. McGraw-Hill: New York. Tabachnick, B.G. and L.S. Fidell. 1996. Using Multivariate Statistics. Harper Collins College Publishers: New York. Webpages: www.statsoft.com/textbook/stathome.html Computations First, the total sum-of-squares is partitioned into the sum-of-squares between groups (SSbg) and the sum-of-squares within groups (SSwg): SStot = SSbg + SSwg This can be expressed as:The SSbg is then partitioned into variance for each IV and the interactions between them. In a case where there are two IVs (IV1 and IV2), the equation looks like this: Therefore, the complete equation becomes: Because in MANOVA there are multiple DVs, a column matrix (vector) of values for each DV is used. For two DVs (a and b) with n values, this can be represented: Similarly, there are column matrices for IVs - one matrix for each level of every

IV. Each matrix of IVs for each level is composed of means for every DV. For "n" DVs and "m" levels of each IV, this is written:Additional matrices are calculated for cell means averaged over the individuals in each group. Finally, a single matrix of grand means is calculated with one value for each DV averaged across all individuals in matrix. Differences are found by subtracting one matrix from another to produce new matrices. From these new matrices the error term is found by subtracting the GM matrix from each of the DV individual scores: Next, each column matrix is multiplied by each row matrix: These matrices are summed over rows and groups, just as squared differences are summed in ANOVA. The result is an S matrix (also known as: "sum-ofsquares and cross-products," "cross-products," or "sum-of-products" matrices.) For a two IV, two DV example: Stot = SIV1 + SIV2 + Sinteraction + Swithin-group errorDeterminants (variance) of the S matrices are found. Wilks’ Lambda is the test statistic preferred for MANOVA, and is found through a ratio of the determininants: An estimate of F can be calculated through the following equations: Where, Finally, we need to measure the strength of the association. Since Wilks’ Lambda is equal to the variance not accounted for by the combined DVs, then (1 – Lambda) is the variance that is accounted for by the best linear combination of DVs. However, because this is summed across all DVs, it can be greater than one and therefore less useful than:Other statistics can be calculated in addition to Wilks’ Lambda. The following is a

short list of some of the popularly reported test statistics for MANOVA: · Wilks’ Lambda = pooled ratio of error variances to effect variance plus error variance · Hotelling’s trace = pooled ratio of effect variance to error variance · Pillai’s criterion = pooled effect variances Use Wilks’ Lambda because it is the most commonly available and reported, however Pillai’s criterion is more robust and therefore more appropriate when there are small or unequal sample sizes. MANOVA works well in situations where there are moderate correlations between DVs. For very high or very low correlation in DVs, it is not suitable: if DVs are too correlated, there isn’t enough variance left over after the first DV is fit, and if DVs are uncorrelated, the multivariate test will lack power anyway, so why sacrifice degrees of freedom?

Multivariate Analysis of Variance (Manova)
description | simple example | MAIA example | how it works | caveats Description: Manova creates a linear combination of the dependent variables (DV's) and then tests for differences in the new variable using methods similar to Anova. The independent variable (IV) used to group the cases is categorical. Manova tests whether the categorical variable explains a significant amount of variability in the new dependent variable. {2 or more DV's} = f (1 or more categorical IV's). Simple example: Suppose you want to test whether stream size and shape differ across ecoregions. The independent categorical variable would be ecoregion and the set of dependent variables might include {width, depth, flow, and gradient}. The dependent variables are correlated which is appropriate for Manova. Manova constructs new variables from {width, depth, flow, and gradient} and tests whether the new composite variables differs across ecoregions. MAIA example: For the Maryland fish IBI, Roth et al. (1999) first grouped stream sites using cluster analysis of fish species data. The cluster analysis computed the distance between each set of species for

every pair of sites and yielded a dendogram that grouped sites by cluster. Cluster analysis does not have statistical testing associated with it, but the authors were interested in determining which clusters were significantly different. To test for significance, they used a Manova model. For the Manova model, the relative abundances of the different fish species were the dependent variables; and they used cluster assignment as the independent variable. They tested each branching point successively down the cluster tree for statistical significance.

Figure

Figure: Schematic representation of the Manova analysis for the Maryland fish index. (The complete cluster tree was too complicated to illustrate here.) In the sketch above, the first branch of the cluster analysis was significant, A and B included significantly different fish assemblages. At the next level, AA and AB were not significantly different but BA and BB were. Successive branches are not pictured, but the same algorithm was applied down each branch. Thus, this diagram illustrates three significant clusters, sites grouped as A, BA and BB.

How the method works: A new variable is created that combines all the dependent variables on the left hand side of the equation such that the differences between group means are maximized. (The fstatistic from Anova is maximized, that is, the ratio of explained variance to error variance). The simplest significance test treats the first, new variable just like a single dependent variable in Anova, and uses the tests as in Anova. Additional, multivariate tests can also be computed that involve multiple new variables derived from the initial set of dependent variables. Assumptions/limitations: Dependent variables can be correlated or independent of each other. Like Anova, Manova isn't too bothered by slight departures from normality, but extreme outliers can be more of a problem. Manova can require rather large sample sizes for complicated models because the number of cases in each category must be larger than the number of dependent variables. Manova also prefers that the

groups have a similar number of cases in each group. In addition, Manova expects that the variance of dependent variables and the correlation between them are similar within groups.

2 - Manova 4.3.05 25 Multivariate Analysis of Variance What Multivariate Analysis of Variance is The general purpose of multivariate analysis of variance (MANOVA) is to determine whether multiple levels of independent variables on their own or in combination with one another have an effect on the dependent variables. MANOVA requires that the dependent variables meet parametric requirements. When do you need MANOVA? MANOVA is used under the same circumstances as ANOVA but when there are multiple dependent variables as well as independent variables within the model which the researcher wishes to test. MANOVA is also considered a valid alternative to the repeated measures ANOVA when sphericity is violated. What kinds of data are necessary? The dependent variables in MANOVA need to conform to the parametric assumptions. Generally, it is better not to place highly correlated dependent variables in the same model for two main reasons. First, it does not make scientific sense to place into a model two or three dependent variables which the researcher knows measure the same aspect of 2 - Manova 4.3.05 26 outcome. (However, this is point will be influenced by the hypothesis which the researcher is testing. For example, subscales from the same questionnaire may all be included in a MANOVA to overcome problems associated with multiple testing.

Subscales from most questionnaires are related but may represent different aspects of the dependent variable.) The second reason for trying to avoid including highly correlated dependent variables is that the correlation between them can reduce the power of the tests. If MANOVA is being used to reduce multiple testing, this loss in power needs to be considered as a trade-off for the reduction in the chance of a Type I error occurring. Homogeneity of variance from ANOVA and t tests becomes homogeneity of variance covariance in MANOVA models. The amount of variance within each group needs to be comparable so that it can be assumed that the groups have been drawn from a similar population. Furthermore it is assumed that these results can be pooled to produce an error value which is representative of all the groups in the analysis. If there is a large difference in the amount of error within each group the estimated error measure for the model will be misleading. How much data? There needs to be more participants than dependent variables. If there were only one participant in any one of the combination of conditions, it would be impossible to determine the amount of variance within that combination (since only one data point would be available). Furthermore, the statistical power of any test is limited by a small sample size. (A greater amount of variance will be attributed to error in smaller sample 2 - Manova 4.3.05 27 sizes, reducing the chances of a significant finding.) A value known as Box’s M, given by

most statistical programs, can be examined to determine whether the sample size is too small. Box’s M determines whether the covariance in different groups is significantly different and must not be significant if one wishes to demonstrate that the sample sizes in each cell are adequate. An alternative is Levene’s test of homogeneity of variance which tolerates violations of normality better than Box's M. However, rather than directly testing the size of the sample it examines whether the amount of variance is equally represented within the independent variable groups. In complex MANOVA models the likelihood of achieving robust analysis is intrinsically linked to the sample size. There are restrictions associated with the generalizability of the results when the sample size is small and therefore researchers should be encouraged to obtain as large a sample as possible. Example of MANOVA Considering an example may help to illustrate the difference between ANOVAs and MANOVAs. Kaufman and McLean (1998) used a questionnaire to investigate the relationship between interests and intelligence. They used the Kaufman Adolescent and Adult Intelligence Test (KAIT) and the Strong Interest Inventory (SII) which contained six subscales on occupational themes (GOT) and 23 Basic Interest Scales (BISs). Kaufman et al. used a MANOVA model which had four independent variables: age, gender, KAIT IQ and Fluid-Crystal intelligence (F-C). The dependent variables were the six occupational theme subscales (GOT) and the twenty-three Basic Interest Scales (BIS). 2 - Manova 4.3.05 28 In Table 2.1 the dependent variables are listed in columns 3 and 4. The independent

variables are listed in column 2, with the increasingly complex interactions being shown below the main variables. If an ANOVA had been used to examine these data, each of the GOT and BIS subscales would have been placed in a separate ANOVA. However, since the GOT and BIS scales are related, the results from separate ANOVAs would not be independent. Using multiple ANOVAs would increase the risk of a Type I error (a significant finding which occurs by chance due to repeating the same test a number of times). Kaufman and McLean used the Wilks’ lambda multivariate statistic (similar to the F values in univariate analysis) to consider the significance of their results and reported only the interactions which were significant. These are shown as Sig in Table 2.1. The values which proved to be significant are the majority of the main effects and one of the 2-way interactions. Note that although KAIT IQ had a significant main effect none of the interactions which included this variable were significant. On the other hand, age and gender show a significant interaction in the effect which they have on the dependent variables. What a multivariate analysis of variance does Like an ANOVA, MANOVA examines the degree of variance within the independent variables and determines whether it is smaller than the degree of variance between the independent variables. If the within subjects variance is smaller than the between subjects variance it means the independent variable has had a significant effect on the dependent 2 - Manova 4.3.05 29 Table 2.1. The different aspects of the data considered by the MANOVA model used by

Kaufman and McLean (1998). Level Independent variables 6 GOT subscales 23 BIS Age Sig Gender Sig Sig KAIT IQ Sig Sig Main Effects F-C Age x Gender Sig Sig Age x KAIT IQ Age x F-C Gender x KAIT IQ Gender x F-C 2-way Interactions KAIT IQ x F-C Age x Gender x KAIT IQ Age x Gender x F-C Age x KAIT IQ x F-C Gender x KAIT IQ x F-C 3-way Interactions Age KAIT IQ x F-C 4-way Interactions Age x Gender x KAIT IQ x F-C 2 - Manova 4.3.05 30 variables. There are two main differences between MANOVAs and ANOVAs. The first is that MANOVAs are able to take into account multiple independent and multiple

dependent variables within the same model, permitting greater complexity. Secondly, rather than using the F value as the indicator of significance a number of multivariate measures (Wilks’ lambda, Pillai’s trace, Hotelling trace and Roy’s largest root) are used. (An explanation of these multivariate statistics is given below). MANOVA deals with the multiple dependent variables by combining them in a linear manner to produce a combination which best separates the independent variable groups. An ANOVA is then performed on the newly developed dependent variable. In MANOVAs the independent variables relevant to each main effect are weighted to give them priority in the calculations performed. In interactions the independent variables are equally weighted to determine whether or not they have an additive effect in terms of the combined variance they account for in the dependent variable/s. The main effects of the independent variables and of the interactions are examined with “all else held constant”. The effect of each of the independent variables is tested separately. Any multiple interactions are tested separately from one another and from any significant main effects. Assuming there are equal sample sizes both in the main effects and the interactions, each test performed will be independent of the next or previous calculation (except for the error term which is calculated across the independent variables). There are two aspects of MANOVAs which are left to researchers: first, they decide which variables are placed in the MANOVA. Variables are included in order to address a 2 - Manova 4.3.05 31 particular research question or hypothesis, and the best combination of dependent

variables is one in which they are not correlated with one another, as explained above. Second, the researcher has to interpret a significant result. A statistical main effect of an independent variable implies that the independent variable groups are significantly different in terms of their scores on the dependent variable. (But this does not establish that the independent variable has caused the changes in the dependent variable. In a study which was poorly designed, differences in dependent variable scores may be the result of extraneous, uncontrolled or confounding variables.) To tease out higher level interactions in MANOVA, smaller ANOVA models which include only the independent variables which were significant can be used in separate analyses and followed by post hoc tests. Post hoc and preplanned comparisons compare all the possible paired combinations of the independent variable groups e.g. for three ethnic groups of white, African and Asian the comparisons would be: white v African, white v Asian, African v Asian. The most frequently used preplanned and post hoc tests are Least Squares Difference (LSD), Scheffe, Bonferroni, and Tukey. The tests will give the mean difference between each group and a p value to indicate whether the two groups differ significantly. The post hoc and preplanned tests differ from one another in how they calculate the p value for the mean difference between groups. Some are more conservative than others. LSD perform a series of t tests only after the null hypothesis (that there is no overall difference between the three groups) has been rejected. It is the most liberal of the post

hoc tests and has a high Type I error rate. The Scheffe test uses the F distribution rather than the t distribution of the LSD tests and is considered more conservative. It has a high 2 - Manova 4.3.05 32 Type II error rate but is considered appropriate when there are a large number of groups to be compared. The Bonferroni approach uses a series of t tests but corrects the significance level for multiple testing by dividing the significance levels by the number of tests being performed (for the example given above this would be 0.05/3). Since this test corrects for the number of comparisons being performed, it is generally used when the number of groups to be compared is small. Tukey's Honesty Significance Difference test also corrects for multiple comparisons, but it considers the power of the study to detect differences between groups rather than just the number of tests being carried out i.e. it takes into account sample size as well as the number of tests being performed. This makes it preferable when there are a large number of groups being compared, since it reduces the chances of a Type I error occurring. The statistical packages which perform MANOVAs produce many figures in their output, only some of which are of interest to the researcher. Sum of Squares: The sum of squares measure found in a MANOVA, like that reported in the ANOVA, is the measure of the squared deviations from the mean both within and between the independent variable. In MANOVA, the sums of squares are controlled for

covariance between the independent variables. There are six different methods of calculating the sum of squares. Type I, hierarchical or sequential sums of squares, is appropriate when the groups in the MANOVA are of equal sizes. Type I sum of squares provides a breakdown of the sums of squares for the whole model used in the MANOVA but it is particularly sensitive to the order in which the independent variables are placed in the model. If a variable is entered first, it is not 2 - Manova 4.3.05 33 adjusted for any of the other variables; if it is entered second, it is adjusted for one other variable (the first one entered); if it is placed third, it will be adjusted for the two other variables already entered. Type II, the partially sequential sum of squares, has the advantage over Type I in that it is not affected by the order in which the variables are entered. It displays the sum of squares after controlling for the effect of other main effects and interactions but is only robust where there are even numbers of participants in each group. Type III sum of squares can be used in models where there are uneven group sizes, although there needs to be at least one participant in each cell. It calculates the sum of squares after the independent variables have all been adjusted for the inclusion of all other independent variables in the model. Type IV sum of squares can be used when there are empty cells in the model but it is generally thought more suitable to use Type III sum of squares under these conditions since Type IV is not thought to be good at testing lower order effects.

Type V has been developed for use where there are cells with missing data. It has been designed to examine the effects according to the degrees of freedom which are available and if the degrees of freedom fall below a given level these effects are not taken into account. The cells which remain in the model have at least the degrees of freedom the full model would have without any cells being excluded. For those cells which remain in the model the Type III sum of squares are calculated. However, the Type V sum of squares 2 - Manova 4.3.05 34 are sensitive to the order in which the independent variables are placed in the model and the order in which they are entered will determine which cells are excluded. Type VI sum of squares is used for testing hypotheses where the independent variables are coded using negative and positive signs e.g. +1 = male, -1 = female. Type III sum of squares is the most frequently used as it has the advantages of Types IV, V and VI without the corresponding restrictions. Mean Squares: The mean square is the sum of squares divided by the appropriate degrees of freedom. Multivariate Measures: In most of the statistical programs used to calculate MANOVAs there are four multivariate measures: Wilks’ lambda, Pillai's trace, Hotelling-Lawley trace and Roy’s largest root. The difference between the four measures is the way in which they combine the dependent variables in order examine the amount of variance in the data. Wilks’ lambda demonstrates the amount of variance accounted for in the dependent

variable by the independent variable; the smaller the value, the larger the difference between the groups being analyzed. 1 minus Wilks’ lambda indicates the amount of variance in the dependent variables accounted for by the independent variables. Pillai's trace is considered the most reliable of the multivariate measures and offers the greatest protection against Type I errors with small sample sizes. Pillai's trace is the sum of the variance which can be explained by the calculation of discriminant variables. It calculates the amount of variance in the dependent variable which is accounted for by the greatest separation of the independent variables. The Hotelling-Lawley trace is generally 2 Manova 4.3.05 35 converted to the Hotelling’s T-square. Hotelling’s T is used when the independent variable forms two groups and represents the most significant linear combination of the dependent variables. Roy’s largest root, also known as Roy’s largest eigenvalue, is calculated in a similar fashion to Pillai's trace except it only considers the largest eigenvalue (i.e. the largest loading onto a vector). As the sample sizes increase the values produced by Pillai’s trace, Hotelling-Lawley trace and Roy’s largest root become similar. As you may be able to tell from these very broad explanations, the Wilks’ lambda is the easiest to understand and therefore the most frequently used measure. Multivariate F value: This is similar to the univariate F value in that it is representative of the degree of difference in the dependent variable created by the independent variable. However, as well as being based on the sum of squares (as in ANOVA) the calculation for F used in MANOVAs also takes into account the covariance of the variables.

Example of Output of SPSS The data analysed in this example came from a large-scale study with three dependent variables which were thought to measure distinct aspects of behaviour. The first was total score on the Dissociative Experiences Scale, the second was reaction time when completing a signal detection task and the third was a measure of people’s hallucinatory experiences (the Launay Slade Hallucinations Scale, LSHS). The independent variable was the degree to which participants were considered prone to psychotic experiences (labeled PRONE in the output shown in Fig 2.1), divided into High, Mean and Low. 2 Manova 4.3.05 36 The first part of the output is shown in Fig 2.1. The Between-Subjects Factors table displays the independent variable levels. Here there is only one independent variable with three levels. The number of participants in each of the independent variable groups are displayed in the column on the far right. The Multivariate Tests table displays the multivariate values: Pillai’s Trace, Wilks’ Lambda, Hotelling’s Trace and Roy’s Largest Root. These are the multivariate values for the model as a whole. The F values for the Intercept, shown in the first part of the table are all the same. Those for the independent variable labeled PRONE are all different, but they are all significant above the 1% level (shown by Sig being .000), indicating that on the dependent variables there is a significant difference between the three proneness groups. The Tests of Between-Subjects Effects table gives the sum of squares, degrees of

freedom, Mean Square value, the F values and the significance levels for each dependent variable. (The Corrected Model is the variance in the dependent variables which the independent variables accounts for without the intercept being taken into consideration.) The section of the table which is of interest is where the source under consideration is the independent variable, the row for PRONE. In this row it can be seen that two (DES, Launay Slade Hallucinations Scale) out of the three dependent variables included in the model are significant (p<.05), meaning that three proneness groups differ significantly in their scores on the DES and the LSHS. ******** Insert Fig 2.1*********** 2 - Manova 4.3.05 37 Figure 2.1. Example of main output from SPSS for MANOVA. To determine which of the three independent variable groups differ from one another on the dependent variables, Least Squares difference comparisons were performed by selecting the relevant option in SPSS. The output is shown in Fig 2.2. The table is split into broad rows, one for each dependent variable, and within each row the three groups of high, mean or low Proneness are compared one against the other. The mean difference between the two groups under consideration are given in one column and then separate columns show the Standard Error, Significance and the lower and upper 95% Confidence intervals. For the DES dependent variable, the High group is significantly different from the Low group (p=0.004) but not from the Mean group (p=0.076). Similarly, for the Total

mean reaction time dependent variable, the High group differs significantly from the Low group but not from the Mean group. For both these dependent variables, the Mean group do not differ from the High group nor from the Low group. For the Launay scale, the High group differs significantly from the Mean and Low groups, both of which also differ from each other. ******** Insert Fig 2.2 *********** Figure 2.2. The Least Squares Difference output from SPSS MANOVA analysis. 2 Manova 4.3.05 38 The means and standard deviations on the DES and Launay scales for the three proneness groups are displayed in Fig 2.3, which was produced by SPSS. Tables such as this assist in interpreting the results. The MANOVA above could also have included another independent variable, such as gender. The interaction between the two independent variables on the dependent variables would have been reported in the Multivariate Statistics and Tests of Between Subjects Effects tables. ******* Insert Fig 2.3 ********* Figure 2.3. Means for the three psychosis proneness groups on the Dissociative Experiences Scale and the Launay Slade Hallucinations Scale. Example of the use of MANOVA From health Snow and Bruce (2003) explored the factors involved in Australian teenage girls smoking, collecting data from 241 participants aged between 13 and 16 years of age. Respondents completed a number of questionnaires including an Adolescent Coping

Scale. They were also asked to indicate how frequently they smoked cigarettes and the responses were used to divide the respondents into three smoking groups (current smokers, experimental smokers, never smokers). In their analysis, Snow and Bruce used 2 - Manova 4.3.05 39 MANOVA with smoking group as the independent variable. In one of the MANOVAs, the dependent variables were the measures on three different coping strategies. Snow and Bruce were only interested in the main effects of smoking group on the dependent variables so they converted the Wilk’s lambda to F values and significance levels. They used Scheffe post hoc analysis to determine which of the three smoking groups differed significantly on the dependent variables, and found that on the 'productive' coping strategy there was a significant difference between the current and experimental smokers, on the 'non-productive' coping strategy there was a difference between the current smokers and those who had never smoked, and on the 'rely on others' coping strategy there was a difference between current and experimental smokers. FAQs How do I maximise the power in a MANOVA? Power refers to the sensitivity of your study design to detect true significant findings when using statistical analysis. Power is determined by the significance level chosen, the effect size and the sample size. (There are many free programmes available on the internet which can be used to calculate effect sizes from previous studies by placing means and sample sizes into the equations.) A small effect size will need a large sample size for significant differences to be detected, while a large effect size will need a

relatively small sample to be detected. In general, the power of your analysis will increase the larger the effect size and sample size. Taking a practical approach, obtaining as large a sample as possible will maximise the power of your study. 2 - Manova 4.3.05 40 What do I do if I have groups with uneven sample sizes? Having unequal sample sizes can affect the integrity of the analysis. One possibility is to recruit more participants into the groups which are under represented in the data set, another is to randomly delete cases from the more numerous groups until they are equal to the least numerous one. What do I do if I think there may be a covariate with the dependent variables included in a MANOVA model? When a covariate is incorporated into a MANOVA it is usually referred to as a MANCOVA model. The ‘best’ covariate for inclusion in a model should be highly correlated with the dependent variables but not related to the independent variables. The dependent variables included in a MANCOVA are adjusted for their association with the covariate. Some experimenters include baseline data in as a covariate to control for any individual differences in scores since even randomisation to different experimental conditions does not completely control for individual differences. Summary MANOVA is used when there are multiple dependent variables as well as independent variables in the study. MANOVA combines the multiple dependent variables in a linear

manner to produce a combination which best separates the independent variable groups. An ANOVA is then performed on the newly developed dependent variable. 2 Manova 4.3.05 41 Glossary Additive: the effect of the independent variables on one another when placed in multi- or univariate analysis of variance. Interaction: the combined effect of two or more independent variables on the dependent variables. Main Effect: the effect of one independent variable on the dependent variables, examined in isolation from all other independent variables. Mean Squares: sum of squares expressed as a ratio of the degrees of freedom either within or between the different groups. Sum of Squares: the squared deviations from the mean within or between the groups. In MANOVA they are controlled for covariance. Wilks' lambda: a statistic which can vary between 0 and 1 and which indicates whether the means of groups differ. A value of 1 indicates the groups have the same mean. References Kaufman, A.S. and McLean, J.E. (1998). An investigation into the relationship between interests and intelligence. Journal of Clinical Psychology, 54, 279-295. 2 - Manova 4.3.05 42 Snow, P.C. and Bruce, D.D. (2003). Cigarette smoking in teenage girls: exploring the role of peer reputations, self-concept and coping. Health Education Research Theory and Practice, 18, 439-452.

Further Reading Anderson, T.W. (2003). An Introduction to Multivariate Statistical Analysis. New York: Wiley. Rees, D.G. (2000). Essential Statistics. (4th ed.) London: Chapman and Hall/CRC. Internet Sources www.statsoftinc.com/textbook/ www2.chass.ncsu.edu/garson/pa765/manova.htm
Multivariate Analysis and MANOVA
The MANOVA (multivariate analysis of variance) is a type of multivariate analysis used to analyze data that involves more than one dependent variable at a time. MANOVA allows us to test hypotheses regarding the effect of one or more independent variables on two or more dependent variables. A MANOVA analysis generates a p-value that is used to determine whether or not the null hypothesis can be rejected. See Statistical Data Analysis for more information.

MANOVA Example
Suppose we have a hypothesis that a new teaching style is better than the standard method for teaching math. We may want to look at the effect of teaching style (independent variable) on the average values of several dependent variables such as student satisfaction, number of student absences and math scores. A MANOVA procedure allows us to test our hypothesis for all three dependent variables at once.

More About MANOVA
Like the example above, a MANOVA is often used to detect differences in the average values of the dependent variables between the different levels of the independent variable. Interestingly, in addition to detecting differences in the average values, a MANOVA test can also detect differences in correlations among the dependent variables between the different levels of the independent variable. MANOVA is simply one of many multivariate analyses that can be performed using SPSS. The SPSS MANOVA procedure is a standard, well accepted means of performing this analysis. Multiple Linear Regression is another type of multivariate analysis, which is described in its own tutorial topic.

Get the Statistics Help you need
When you hire me to do the data analysis for your dissertation Results Chapter, I will determine whichstatistical methods are appropriate for your hypotheses. I also provide ongoing statistical help as needed to ensure that you fully understand all of the statistics that I used for your dissertation. Simply contact me by phone or email to get started. Steve Creech 1-800-357-0321 or 1-630-705-9482 | [email protected] Statistical Consultant Overview