With a repeated measures ANOVA, as we are using the same subjects in each group, we can remove the variability due to the individual differences between subjects, referred to as \(SS_{subjects}\), from the within-groups variability (\(SS_w\)) by treating each subject as a block.
That is, each subject becomes a level of a factor called subjects. The ability to subtract \(SS_{subjects}\) will leave us with a smaller \(SS_{error}\) term (it will only reflect individual variability to each condition).
1.4 One-way versus two-way repeated measurements analyis
One-way repeated measures ANOVA is an extension of the paired-samples t-test for comparing the means of three or more levels of a within-subjects variable.
Two-way repeated measures ANOVA is used to evaluate simultaneously the effect of two within-subject factors on a continuous outcome variable.
2 F-statistic and error term
2.1 F-statistic
Following division by the appropriate degrees of freedom, a mean sum of squares for between-groups (\(MS_b\)) and within-groups (\(MS_w\)) is determined and an F-statistic is calculated as the ratio of \(MS_b\) to \(MS_w\) (or \(MS_{error}\)).
\[ F=\frac{MS_b}{MS_w}=\frac{MS_b}{MS_{error}}\] A repeated measures ANOVA calculates an F-statistic in a similar way:
This has the effect of increasing the value of the F-statistic due to the reduction of the denominator and leading to an increase in the power of the test to detect significant differences between means.
2.3 Manual examples: repeated measures ANOVA vs independent ANOVA
3 Assumptions and effect size
3.1 Assumptions
No significant outliers
Normality of the dependent variable (at each time point or condition)
We can use histograms and normality tests
Variance of the differences between groups should be equal (sphericity assumption)
We can use Mauchly’s test of Sphericity (if p>0.05, sphericity can be assumed).
3.2 Reminder about factors
Simple one-way ANOVA: there is only one factor, the independent variable that divides the data into groups.
Two-way ANOVA: there are two factors for which we examine their effects on the dependent variable, and also if there is any interaction between the factors.
Repeated measures ANOVA: the factors may include both between-subject factors (e.g., group membership) and within-subject factors (e.g., time points when measurements are taken on the same subject).
3.3 Effect size
The \(\eta_{partial}^2\) is specific to the factor \(i\), but if there are several factors, you cannot add the individual \(\eta_{partial}^2\) to form a total value because the denominator does not contain the total sum of squares (it includes only the variance related to a particular factor and its error, not the total variance). \(\eta_{partial}^2\) is where the the \(SS_{subjects}\) has been removed from the denominator:
\[\eta_{partial}^2=\frac{SS_{conditions}}{SS_{conditions}+SS_{error}}\] In contrast, total \(\eta^2\) considers the proportion of the total variance explained by each factor, which allows for a single value that accounts for all sources of variance.
4 Example in R
4.1 One-way repeated measure ANOVA
Let’s create a random dataset of 6 participants with three observation periods:
ID<-seq(1:6) # 6 participantsset.seed(123) # allow for the random sample to be reproducibleTime01<-c(sample(1:200,6,replace=T)) # 6 measurements at first time pointTime02<-c(sample(50:250,6,replace=T)) # 6 measurements at second time pointTime03<-c(sample(100:300,6,replace=T)) # 6 measurements at third time pointdf<-data.frame(ID,Time01,Time02,Time03)df$ID<-as.factor(df$ID)# convert into long formatDF <- tidyr::gather(df,Time,Measure,2:4)DF$Time <-as.factor(DF$Time)head(DF)
Effect DFn DFd SSn SSd F p p<.05 pes
1 (Intercept) 1 5 494680.89 29967.11 82.537 0.00027 * 0.943
2 Time 2 10 38812.11 24467.89 7.931 0.00900 * 0.613
Note: “ges” in the outputs indicates generalized \(\eta^2\) and “pes” indicates \(\eta_{partial}^2\).
4.3 Mauchly’s test
We can test whether the variance of the differences between groups is equal (sphericity assumption) using the Mauchly’s test of Sphericity:
mod$`Mauchly's Test for Sphericity`
Effect W p p<.05
1 Time 0.936 0.877
Interpretation: When Mauchly’s test for equality of variances fails to show significance, you have evidence that the data are suitable for the application of the One-way ANOVA repeated Measures test.
4.4 Corrections
In case Sphericity was violated, we can rely on two corrections based upon estimates of sphericity that are applied to the df used to assess the observed F-ratio:
Greenhous and Geisser (GG)
Huynh-Feldt (HFe)
In our example, the lower limit will be: \(1/(k-1)=1/(3-1)=1/2=0.5\)
Interpretation: In the output the calculated GGe value is 0.94 and, as this is closer to 1 than it is to the limit of 0.5, it does not represent a substantial deviation from sphericity.
In case of sphericity violation (and in case where both corrections give different results), a good practice is to calculate the average of the two corrections (GGe and HFe) and to take the average of the two significance values (p[GG] and p[HF]). If the average p-value is not significant, then the F-ratio is considered as non-significant.
4.5 Two-way repeated measure ANOVA
ID<-seq(1:18) # 18 participantsGroup<-rep(1:3,each=6) # 3 parallel groups with 6 participantsset.seed(123) # allow for the random sample to be reproducibleTime01<-c(sample(1:200,18,replace=T)) # 18 measurements at first time pointTime02<-c(sample(50:250,18,replace=T)) # 18 measurements at second time pointTime03<-c(sample(100:300,18,replace=T)) # 18 measurements at third time pointdf<-data.frame(ID,Group,Time01,Time02,Time03)df$ID<-as.factor(df$ID)df$Group<-factor(df$Group,levels=c(1,2,3),labels=c("Group 01", "Group 02","Group 03"))# convert into long formatDF<-tidyr::gather(df,Time,Measure,3:5)DF$Time<-as.factor(DF$Time)head(DF)
ID Group Time Measure
1 1 Group 01 Time01 159
2 2 Group 01 Time01 179
3 3 Group 01 Time01 14
4 4 Group 01 Time01 195
5 5 Group 01 Time01 170
6 6 Group 01 Time01 50
4.6 Methods for conducting two-way repeated measure ANOVA
ANOVA Table (type II tests)
Effect DFn DFd F p p<.05 ges
1 Group 2 45 0.187 0.830000 0.008
2 Time 2 45 10.671 0.000161 * 0.322
3 Group:Time 4 45 0.589 0.672000 0.050
Interpretation: We do not observed statistically significant two-way interactions between group and time.
4.7 Pairwise comparisons for Group
# Effect of group at each time pointone.way <- DF |> dplyr::group_by(Time) |> rstatix::anova_test(dv = Measure, formula = Measure ~ Group,wid = ID, within = Group) |> rstatix::get_anova_table() |> rstatix::adjust_pvalue(method="bonferroni")print(one.way[,c(1,2,5,6,8)])
# A tibble: 3 × 5
Time Effect F p ges
<fct> <chr> <dbl> <dbl> <dbl>
1 Time01 Group 0.919 0.42 0.109
2 Time02 Group 0.118 0.89 0.015
3 Time03 Group 0.215 0.809 0.028
# Comparisons between groups at each time pointpwc <- DF |> dplyr::group_by(Time) |> rstatix::pairwise_t_test( Measure ~ Group, paired =TRUE,p.adjust.method ="bonferroni")print(pwc[,c(1,3,4,7,9)])
# A tibble: 9 × 5
Time group1 group2 statistic p
<fct> <chr> <chr> <dbl> <dbl>
1 Time01 Group 01 Group 02 1.53 0.188
2 Time01 Group 01 Group 03 -0.152 0.885
3 Time01 Group 02 Group 03 -1.36 0.232
4 Time02 Group 01 Group 02 -0.612 0.568
5 Time02 Group 01 Group 03 0.0310 0.976
6 Time02 Group 02 Group 03 0.454 0.669
7 Time03 Group 01 Group 02 0.221 0.834
8 Time03 Group 01 Group 03 0.820 0.45
9 Time03 Group 02 Group 03 0.502 0.637
4.8 Pairwise comparisons for Time
# Effect of time at each level of groupone.way2 <- DF |> dplyr::group_by(Group) |> rstatix::anova_test(dv = Measure, formula = Measure ~ Time,wid = ID, within = Time) |> rstatix::get_anova_table() |> rstatix::adjust_pvalue(method ="bonferroni")print(one.way2[,c(1,2,5,6,8)])
# A tibble: 3 × 5
Group Effect F p ges
<fct> <chr> <dbl> <dbl> <dbl>
1 Group 01 Time 2.55 0.111 0.254
2 Group 02 Time 7.37 0.006 0.496
3 Group 03 Time 3.03 0.079 0.288
# Pairwise comparisons between time pointspwc2 <- DF |> dplyr::group_by(Group) |> rstatix::pairwise_t_test( Measure ~ Time, paired =TRUE,p.adjust.method ="bonferroni")print(pwc2[,c(1,3,4,7,9)])
# A tibble: 9 × 5
Group group1 group2 statistic p
<fct> <chr> <chr> <dbl> <dbl>
1 Group 01 Time01 Time02 -0.160 0.879
2 Group 01 Time01 Time03 -4.20 0.009
3 Group 01 Time02 Time03 -2.36 0.065
4 Group 02 Time01 Time02 -2.46 0.057
5 Group 02 Time01 Time03 -3.21 0.024
6 Group 02 Time02 Time03 -1.75 0.141
7 Group 03 Time01 Time02 -0.0604 0.954
8 Group 03 Time01 Time03 -3.25 0.023
9 Group 03 Time02 Time03 -2.39 0.062
4.9 Pairwise t-test comparison (if no significant interaction)
If the interaction is not significant, we can simply execute pairwise t-test comparison.
# Comparisons for group variableres1 = DF |> rstatix::pairwise_t_test( Measure ~ Group, paired =TRUE,p.adjust.method ="bonferroni")res1[,c(2,3,6,8,10)]
# A tibble: 3 × 5
group1 group2 statistic p p.adj.signif
<chr> <chr> <dbl> <dbl> <chr>
1 Group 01 Group 02 0.698 0.495 ns
2 Group 01 Group 03 0.338 0.739 ns
3 Group 02 Group 03 -0.340 0.738 ns
# Comparisons for the time variableres2 = DF |> rstatix::pairwise_t_test( Measure ~ Time, paired =TRUE,p.adjust.method ="bonferroni")res2[,c(2,3,6,8,10)]
