Introduction

Before considering using ANOVA, you should check that you have a dependent (outcome) variable which is on a continuous or ratio scale (such as time in days to withdrawal of life support) and that duyou have at least one categorical variable (such as age-group). The categorical variable is referred to as a factor in the analysis, while the different categories for this variable (e.g. ‘<25′, ’25-45′, ’46-60’ and ‘>60’ represent the levels of this factor. This terminology should help when consulting the comprehensive flowchart for ANOVA discussed below. If you are also unfamiliar with terminology, such as continuous and categorical, for different types of variable  you should find it helpful to consult the  Key Learning in Statistics tutorial Types of data – a prelude to hypothesis testing.

· Q 1. Where should I refer to in doing some useful background reading about ANOVA?

A. You can make a really good start by consulting the free ebook

General Linear Models: Univariate GLM, Anova/Ancova, Repeated Measures by G. David Garson.

On scrolling down to the page at the link to the above title, you will find various means of obtaining a copy of the book.

Alternatively, you may prefer to consult the following video:

Using GLM Univariate to perform an analysis of covariance – ANCOVA 

However, you would also be well-advised to consider some supplementary reading when progressing to perform analyses using SPSS, as the above video, assume you already understand why you should be clicking certain buttons .  Useful sources include:

SPSS Survival Manual: A Step by Step Guide to Data Analysis Using IBM SPSS (7th edition) [contains very useful worked examples for you to follow with your own data]

To assist you in deciding which chapters are of relevance to your particular research design, you should carefully consult a comprehensive flowchart on ANOVA (see second of two flowcharts on page linked to this topic).

· Q 2. I have consulted the above flowchart and am interested in performing an F-test to assess the statistical effect of age group (‘< 25′, ’25 – 45′, ’46 – 60’ and ‘>60’) on time to withdrawal of life support. How may I test for Normality of residuals and equality of variances?

A. Regarding testing for Normality, please note that you should first capture the residuals in a column within your spreadsheet. This can easily be achieved by clicking the button Save in the SPSS dialogue box you are using to perform your ANOVA and then choosing Standardized from the available options under Residuals. Having completed the ANOVA procedure, you can then use the advice under Q. 6 of the StatsforMedics WordPress page TESTS OF NORMALITY  to assess whether or not the residuals are Normally distributed.

Regarding testing for equality of variances, have a look at a video on Levene’s test.

· Q 3.  I have consulted the above flowchart and noted that, as my data do not satisfy the requirements for the F-test for a between subjects ANOVA, I should consider performing the Kruskal-Wallis test instead. Where can I find instructions on performing this test using SPSS?

A. If the assumption of Normality of residuals is violated, you may wish to consider the Kruskal-Wallis test, in which case move on to the advice below. However, if it is only the assumption of eqality of variances that is violated, you should consider Welch’s ANOVA. A helpful tutorial on conducting Welch’s ANOVA using SPSS and in progressing to the corresponding pairwise comparisons using the Games-Howell test can be found at ANOVA with Games-Howell Post Hoc Test in SPSS.

Instructions relating to the Kruskal-Wallis test

Have a look at Non-parametric ANOVA using SPSS.

If you progress to look at the additional information on pairwise comparisons in this resource, please be aware that you are obliged to correct your corresponding p-values for chance.

A simple procedure would be to use the Bonferroni correction, which requires you to multiply each of the p-values from a pairwise comparison by the number of pairwse comparisons you have made. Therefore you should think very carefully in advance about how many pairwise comparisons are necessary for the purpose of your study. The more you carry out the higher the correction factor! There is no need to correct your p-value for the Kruskal-Wallis test in this context.

When quoting your final results it would be a good idea to tabulate the p-values obtained for your pairwise comparisons with and without the above correction and leave the reader to draw their own conclusions.

Perhaps more importantly, you should note that use of the a Bonferroni-corrected Mann-Whitney U-test is very conservative and in more recent versions of SPSS (version 19 onwards), functionality has been provided to generate the results for a Bonferroni-corrected Dunn’s test (sometimes referred to as the Dunn-Bonferroni test). This functionality is available provided the Kruskal-Wallis test result proves significance. This makes, sense as only once statistical significance is determined across all groups does it makes sense to start probing deeper in an attempt to assess which pairs of groups contribute most to this result.  To find out how to access this functionality, please refer to the following resource Can SPSS perform a Dunn’s nonparametric comparison for post hoc testing after a Kruskal-Wallis test?

· Q 5.  Can you recommend an effect size estimate for use with the Kruskal-Wallis test?

A. There are a few possiblities. One such estimate which is easy to calculate and grasp conceptually is eta-squared. For more details, please refer to  the article The need to report effect size estimates revisited: An overview of some recommended measures of effect size.

· Q 6. As my data did not satisfy the requirements for the F-test, I performed the Kruskal-Wallis test for the case study in Q. 2. The corresponding p-value was not > 0.05. Therefore, I was able to conclude that there was statistical evidence at the 5% significance level to conclude that variation in time to withdrawal of life support could be explained in terms of patient age group. I also noted, however, from the output that the values of the mean rank for time to withdrawal of life-support provided in the SPSS output decreased with increasing age as defined by the age categories.  Is there a test which I can apply as a complement to the Kruskal-Wallis test to test specifically for a trend in time to withdrawal of life support according to age?

A. Yes, indeed there is. SPSS provides the Jonckheere-Terpstra test. Please refer to the resource Does SPSS offer a non-parametric trend test for K ordered independent samples? for more details. When following the menu command instructions within this resource, please note that with the current version of SPSS, there is now an extra sub-menu command Legacy Dialogs immedately after the option Non-parametric tests which you will need to select in order to access the recommended option k independent samples.

· Q 7. What if I wish to include additional factors such as gender and comorbidities?

A. A one-way between subjects ANOVA can be extended to a two- way between subjects ANOVA or a three-way between subjects ANOVA for independent factors. Details of how to perform a two-way between subjects ANOVA using SPSS (together with how to test for interactions between factors) are provided in the movie Two-way ANOVA in SPSS. These details can easily be adapted to perform a three-way between subjects ANOVA.  Note that a three-way between subjects ANOVA probably represents the upper limit in terms of factors that ought to be included in a between subjects ANOVA  while still hoping to find the model output comprehensible. 

Please also refer to the solution to Q. 8, below for assistance with interpretation of graphical output for the the above sorts of models.

· Q 8.  By means of a one-way between subjects ANOVA, I found that there was statistical evidence to suggest that the factor age contributes to the variability in values for the dependent variable time to withdrawal of life-support. Can you recommend a significance test for identifying which age-groups are significantly different with respect to mean time to withdrawal of life support and can this test be performed using SPSS?

A.  Provided  your data satisfy the assumption of homogeneity of variances (see video on Levene’s test) and your do not intend exclusively to compare multiple groups against a baseline control, a good choice is Tukey’s test, which is available as an additional option in SPSS when performing a between subjects ANOVA.  See the movie

· Q 10. Is there an effective procedure for obtaining a graphical representation of main effects and interactions for a two-way or three-way between subjects ANOVA.

A. Yes, this can be achieved by means of profile plots, which are plots of estimated marginal means of the dependent variable for the different levels of the individual factors. In these plots, parallel lines point to no interactive effect between the factors in determining the values of the dependent variable. A hypothesis test is performed to assess whether a possible interactive effect reflected in the profile is statistically significant.   Some relevant steps and further interpretation of findings can be conveniently accessed by means of the following resources:

1. Two-way factorial ANOVA in PASW (SPSS) – covers two-way between subjects ANOVA

and

2. ANOVA/MANOVA  – covers both two-way and three-way between subjects ANOVA: click on the  header ‘Interaction Effects’ within the contents list at the top of the page and use the explanation in the example as a guide to assist you in presenting the interpretation of your own findings. Although your study involves different variables, with a little care, you should be able to recognize a direct analogy.

· Q 11. I wish to compare exam performance as a continuous variable for the same students over consecutive years. Having examined the distribution of the marks for each year, I recognize that my data are non-parametric and therefore do not satisfy the conditions for an F-test under a within-subjects ANOVA. I would still like to compare my data across years. How should I proceed?

A. In terms of investigating whether or not year group can account for variability in exam scores, you should consider the Friedman test. This test acknowledges that your data are related across year groups, as the students don’t change across these groups. Details of how to perform this test using SPSS, including how to present your data in your spreadsheet and interpret your findings, are available under Friedman Test in SPSS. While the advice on post-hoc tests for pairwise comparisons in this resource are of interest, please be advised that unless the distribution for each group in a pair that you wish to compare is symmetric in shape, you ought to use the Sign test rather than the Wilcoxon matched-pairs signed ranks test (while still bearing in mind the Bonferroni correction recommended in the above resource). You can read more about the Sign test (and the above advice on choice of test) by consulting the solution to Q. 11 on the StatsforMedics WordPress page HYPOTHESIS TESTS FOR COMPARING TWO GROUPS OF MEASUREMENT OR ORDINAL DATA.

· Q 12. For my experiment, I have multiple repeated measures but I am limited by a small sample size. Can you recommend suitable statistical procedures for comparing my measurements over time or methods of measurement?

A. Yes. You should take a thorough look through the article

Matthew JNS et al. Analysis of serial measurements in medical research. British Medical Journal 1990; 300: 230-235.

· Q 13. I have some repeated measurements over time within my laboratory-based study and am testing for the effect of time. However, from my reading, I understand that I need to test for Sphericity. My SPSS output includes a table with results for Mauchly’s Test of Sphericity. What should I do if this test shows that the requirement of homogeneity of covariance has been violated?

A. Mauchly’s test of sphericity has its limitations. Please consider reading the article

What is all this stuff about sphericity in my repeated measures ANOVA output?
to learn more.

Among other things, the material in this article recommends the use of a value called epsilon over and above Mauchly’s test for deciding whether the requirement of homogeneity of covariance has been violated. Incidentally, the same article also explains the notion of homogeneity of covariance for those who are interested! Estimates of the value of epsilon are included in the table of SPSS output with the header Mauchly’s Test of Sphercity.

The idea in the above resource of using the average of the estimates for epsilon offered by the Greenhouse-Geisser and Huynh-Feldt methods is well worth considering.  If you then discover epsilon is considerably below 1, then you may wish to opt for the corrected ANOVA results where sphericity is not assumed (see the table with header ‘Test of Within-Subjects Effects’ later in your SPSS output). Here, you ought to decide on whether to use the more conservative approach* (Greenhouse-Geisser) or the more liberal approach (Huynh-Feldt).  Bear in mind that with a conservative or liberal test you run the risk of a Type II or Type I error, respectively. In terms of knowing how low epsilon can go before you need to concern yourself about violation of homogeneity of variance, this depends on the number of levels for your within subject factor. Please refer to the recommended threshold values for different numbers of levels provided in the above resource before deciding whether your existing F-test results (including those for any interaction effects) need to corrected. Which would pose the biggest problem within the context of your study? If you need a reminder about the meaning of Type I and Type II errors, please consult the Key learning in Statistics tutorial 4. Fundamentals of Hypothesis Testing.

As is also explained in the first resource recommended above (see the section Multiple comparisons and contrasts), it is common practise to retain your original Bonferroni test results (for pairwise comparisons) irrespective of whether or not homogeneity of covariance has been violated, so in terms of possible adjustments, you need only focus on F-test results for overall main effects or interactions.

* Terminology: A “more conservative approach” in this case is one which applies a stronger correction to your original p-values, thus making statistical significance harder to achieve for the F-tests; in other words it affords a more stringent test.

· Q 14. I would like to assess the possible effect of different treatments on patients in terms of their depression levels (represented by a numeric variable) over time.   This will involve investigating differences across the treatment types and across time points, in addition to testing for an interactive effect between time and treatment type.  I understand from consulting the comprehensive flowchart on ANOVA provided on this WordPress site
that a mixed (or, split-plot) factorial ANOVA is an appropriate choice of ANOVA model for my above objectives.  However, I am a little daunted by the very name of the model! Can you recommend a suitable resource to help me make friends with this model and perhaps another one to help me gain a deeper understanding of the results generated and what they mean.

A. No problem!  You ought to find that the content of the video How to Use SPSS-Factorial Repeated Measures ANOVA (Split-Plot or Mixed Between-Within Subjects) makes you much less apprehensive.

If you are registered with the University of Edinburgh, you can check the loan status of hard copies of the above book via the University’s library discovery system, DiscoverEd.

· Q 15. I am carrying out a project in orthopaedics. I would like to compare post-operative outcomes for patients in two groups, one of which experienced transfer of care from one consultant to another and the other (the control group) which did not experience transfer of care. My primary outcome measures are the Oxford hip and knee scores and the SF36 score, all of which are provided as raw scores rather than categories. The patients were assigned to the different treatment options. Therefore, on comparing the two groups post-operatively, I wish to correct for differences in pre-op scores. How should I proceed?

A. You would do well to familiarize yourself with the statistical procedure known as ANCOVA (Analysis of COvariance). An excellent resource for this purpose is the chapter Analysis of Covariance in the statistical textbook SPSS Survival Manual by Julie Pallant. While this book is in its 5th edition, earlier editions also cover the above topic adequately.

If you are registered with the University of Edinburgh, you can check the loan status of hard copies of different editions, including the 5th edition, of above book via the University’s library discovery system, DiscoverEd.

The above  chapter provides a great deal of support regarding preliminary tests (including a test of multicollinearity) for ensuring your data are suitable for ANCOVA. The chapter also provides a clear interpretation of the SPSS output for these tests and the subsequent ANCOVA, where appropriate. Just as with ANOVA, there are multiple types of ANCOVA. Moreover, even with a one-way ANCOVA, you have the freedom to choose how many covariates (the factors to control for) which you wish to include. At the simplest level, you might wish to compare post-operative Oxford hip scores across your two groups while including pre-operative Oxford hip score as a covariate.  In the above chapter, there is an analogous example whereby scores on a Fear of Statstics  test are compared across two groups at Time 2, while controlling for scores on the same test at Time 1. However, more advanced examples are also provided to meet your needs.

To assist you to engage with the relevant procedures the SPSS data for all of the worked examples provided in this chapter are available at 

Data files and other resources for use with SPSS Survival Manual 5.

For supplementary advice on interpretation of SPSS output for ANCOVA, including output pertaining to pairwise comparisons,
I strongly recommend use of Chapter 13 of the text IBM SPSS Statistics 19 Made Simple by Gray, C.D & Kinnear, P.R.

If you are registered with the University of Edinburgh, you can check the loan status of hard copies of the above book via the University’s library discovery system, DiscoverEd.

In terms of suitable reading material for defending the the use of ANCOVA over and above considering arithmetic difference or percentage change in scores across pre- and post-op stages, I would recommended the BMJ article Analysing controlled trials with baseline and follow up measurements.  

· Q 16. I have a number of different different continous outcome variables which I wish to consider in relation to the same predictive categorical factors. I have heard that MANOVA could serve as a suitable approach to dealing with the different analyses in one sweep. However, I would first of all wish to verify whether this is appropriate.

A. MANOVA (Multivariate ANalysis of VAriance) as defined is designed for use instead of ANOVA where the assumptions of ANOVA are met but different continuous dependent variables are under consideration for the same factors. While the latter interpretation of MANOVA may justify its use from the point
of view of a textbook definition, the output presented is not particularly transparent and statisticians with considerable experience may advise you that a MANOVA is rarely preferrable over and above multiple individual ANOVAs (with the p-values corrected for multiple comparisons). To gain an insight into when to use MANOVA and the benefits of so doing, you may wish to consider the article Brief Overview of Manova. While this article goes on to refer to analysis of data in Stata (a package which you may not be using), the underlying explanation should be helpful.

For further reading, you might like to consider the article Multivariate Analysis of Variance (MANOVA) by French et al.

· Q 17. I have gained some insight into the underlying theory behind MANOVA. However, I would also like support in running a MANOVA for my data using a statistical package and ideally linking theory with application to interpret my findings soundly. Can you recommend any resources that are fit for purpose.

A.  The following references should be particularly useful in meeting your needs:

The chapter on MANOVA in Pallant’s SPSS Survival Manual (see solution to previous question for more details of this book);

Applied Multivariate Statistics for the Social Sciences by Keenan A Pituch and James P Stevens. The 6th edition of this book is available via Amazon.co.uk, while earlier versions are also available via the University of Edinburgh library catalogue (see DiscoverEd).

In terms of representing MANOVA models, the second of these two references is the more thorough. However, both are highly suitable for sound interpretation of model assumptions and findings.

Both books have associated webpages which allow access to datasets for working through real-life examples where MANOVA may prove to be a useful choice for analysing the data.

Pallant’s book is designed for use with SPSS, while Stevens’s book is designed for use with both SPSS and SAS.

Analysis of Variance (ANOVA), Analysis of Covariance (ANCOVA) and Multivariate Analysis of Variance (MANOVA) by Margaret MacDougall is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.