Repeated measures ANOVA presentation

Learn how to run the analysis in R

1 Repeated measure ANOVA (or within-subjects ANOVA)

1.1 Definition

Repeated measures ANOVA:

• is used for analyzing data where same subjects are measured more than once on the same outcome variable under different time points or conditions.
• is the equivalent of the one-way ANOVA, but for related, not independent groups, and is the extension of the dependent t-test.

In repeated measures ANOVA, the independent variable (also referred as the within-subject factor) has categories (called levels or groups).

1.2 ANOVA versus repeated measures ANOVA

See original image here

1.3 Individual differences between subjects

With a repeated measures ANOVA, as we are using the same subjects in each group, we can remove the variability due to the individual differences between subjects, referred to as \(SS_{subjects}\), from the within-groups variability (\(SS_w\)) by treating each subject as a block.

That is, each subject becomes a level of a factor called subjects. The ability to subtract \(SS_{subjects}\) will leave us with a smaller \(SS_{error}\) term (it will only reflect individual variability to each condition).

1.4 One-way versus two-way repeated measurements analyis

One-way repeated measures ANOVA is an extension of the paired-samples t-test for comparing the means of three or more levels of a within-subjects variable.

Two-way repeated measures ANOVA is used to evaluate simultaneously the effect of two within-subject factors on a continuous outcome variable.

2 F-statistic and error term

2.1 F-statistic

Following division by the appropriate degrees of freedom, a mean sum of squares for between-groups (\(MS_b\)) and within-groups (\(MS_w\)) is determined and an F-statistic is calculated as the ratio of \(MS_b\) to \(MS_w\) (or \(MS_{error}\)).

\[ F=\frac{MS_b}{MS_w}=\frac{MS_b}{MS_{error}}\] A repeated measures ANOVA calculates an F-statistic in a similar way:

\[ F=\frac{MS_{conditions}}{MS_{error}}=\frac{MS_{time}}{MS_{error}}\]

2.2 Error term

A repeated measures ANOVA can further partition the error term, thus reducing its size:

\[SS_{error}=SS_w-SS_{subjects} \] \[SS_{error}=SS_T-SS_{conditions}-SS_{subjects}\]

This has the effect of increasing the value of the F-statistic due to the reduction of the denominator and leading to an increase in the power of the test to detect significant differences between means.

2.3 Manual examples: repeated measures ANOVA vs independent ANOVA

3 Assumptions and effect size

3.1 Assumptions

• No significant outliers
• Normality of the dependent variable (at each time point or condition)
  • We can use histograms and normality tests
• Variance of the differences between groups should be equal (sphericity assumption)
  • We can use Mauchly’s test of Sphericity (if p>0.05, sphericity can be assumed).

3.2 Reminder about factors

• Simple one-way ANOVA: there is only one factor, the independent variable that divides the data into groups.
• Two-way ANOVA: there are two factors for which we examine their effects on the dependent variable, and also if there is any interaction between the factors.
• Repeated measures ANOVA: the factors may include both between-subject factors (e.g., group membership) and within-subject factors (e.g., time points when measurements are taken on the same subject).

3.3 Effect size

The \(\eta_{partial}^2\) is specific to the factor \(i\), but if there are several factors, you cannot add the individual \(\eta_{partial}^2\) to form a total value because the denominator does not contain the total sum of squares (it includes only the variance related to a particular factor and its error, not the total variance). \(\eta_{partial}^2\) is where the the \(SS_{subjects}\) has been removed from the denominator:

\[\eta_{partial}^2=\frac{SS_{conditions}}{SS_{conditions}+SS_{error}}\] In contrast, total \(\eta^2\) considers the proportion of the total variance explained by each factor, which allows for a single value that accounts for all sources of variance.

4 Example in R

4.1 One-way repeated measure ANOVA

Let’s create a random dataset of 6 participants with three observation periods:

ID<-seq(1:6) # 6 participants
set.seed(123) # allow for the random sample to be reproducible
Time01<-c(sample(1:200,6,replace=T)) # 6 measurements at first time point
Time02<-c(sample(50:250,6,replace=T)) # 6 measurements at second time point
Time03<-c(sample(100:300,6,replace=T)) # 6 measurements at third time point
df<-data.frame(ID,Time01,Time02,Time03)
df$ID<-as.factor(df$ID)
# convert into long format
DF <- tidyr::gather(df,Time,Measure,2:4)
DF$Time <- as.factor(DF$Time)
head(DF)

  ID   Time Measure
1  1 Time01     159
2  2 Time01     179
3  3 Time01      14
4  4 Time01     195
5  5 Time01     170
6  6 Time01      50

4.2 Methods for conducting one-way repeated measure ANOVA

# general
mod=rstatix::anova_test(DF,
dv=Measure,
wid=ID,
within=c(Time))
mod$ANOVA

  Effect DFn DFd     F     p p<.05   ges
1   Time   2  10 7.931 0.009     * 0.416

# detailed 
rstatix::anova_test(DF,
dv=Measure,
wid=ID,
within=c(Time),
detailed = T,
effect.size = "pes")$ANOVA

       Effect DFn DFd       SSn      SSd      F       p p<.05   pes
1 (Intercept)   1   5 494680.89 29967.11 82.537 0.00027     * 0.943
2        Time   2  10  38812.11 24467.89  7.931 0.00900     * 0.613

Note: “ges” in the outputs indicates generalized \(\eta^2\) and “pes” indicates \(\eta_{partial}^2\).

4.3 Mauchly’s test

We can test whether the variance of the differences between groups is equal (sphericity assumption) using the Mauchly’s test of Sphericity:

mod$`Mauchly's Test for Sphericity`

  Effect     W     p p<.05
1   Time 0.936 0.877      

Interpretation: When Mauchly’s test for equality of variances fails to show significance, you have evidence that the data are suitable for the application of the One-way ANOVA repeated Measures test.

4.4 Corrections

In case Sphericity was violated, we can rely on two corrections based upon estimates of sphericity that are applied to the df used to assess the observed F-ratio:

• Greenhous and Geisser (GG)
• Huynh-Feldt (HFe)

In our example, the lower limit will be: \(1/(k-1)=1/(3-1)=1/2=0.5\)

mod$`Sphericity Corrections`

  Effect  GGe    DF[GG] p[GG] p[GG]<.05   HFe      DF[HF] p[HF] p[HF]<.05
1   Time 0.94 1.88, 9.4  0.01         * 1.488 2.98, 14.88 0.009         *

Interpretation: In the output the calculated GGe value is 0.94 and, as this is closer to 1 than it is to the limit of 0.5, it does not represent a substantial deviation from sphericity.

In case of sphericity violation (and in case where both corrections give different results), a good practice is to calculate the average of the two corrections (GGe and HFe) and to take the average of the two significance values (p[GG] and p[HF]). If the average p-value is not significant, then the F-ratio is considered as non-significant.

4.5 Two-way repeated measure ANOVA

ID<-seq(1:18) # 18 participants
Group<-rep(1:3,each=6) # 3 parallel groups with 6 participants
set.seed(123) # allow for the random sample to be reproducible
Time01<-c(sample(1:200,18,replace=T)) # 18 measurements at first time point
Time02<-c(sample(50:250,18,replace=T)) # 18 measurements at second time point
Time03<-c(sample(100:300,18,replace=T)) # 18 measurements at third time point
df<-data.frame(ID,Group,Time01,Time02,Time03)
df$ID<-as.factor(df$ID)
df$Group<-factor(df$Group,levels=c(1,2,3),
                 labels=c("Group 01", "Group 02","Group 03"))
# convert into long format
DF<-tidyr::gather(df,Time,Measure,3:5)
DF$Time<-as.factor(DF$Time)
head(DF)

  ID    Group   Time Measure
1  1 Group 01 Time01     159
2  2 Group 01 Time01     179
3  3 Group 01 Time01      14
4  4 Group 01 Time01     195
5  5 Group 01 Time01     170
6  6 Group 01 Time01      50

4.6 Methods for conducting two-way repeated measure ANOVA

DF$Measure <- as.numeric(DF$Measure)
DF$ID <- as.factor(DF$ID)
DF$Time <- as.factor(DF$Time)
two_way <- rstatix::anova_test(data = DF,
                               formula = Measure ~ Group*Time,
                               dv = Measure, 
                               wid = ID,
                               within = c(Group, Time)
)
rstatix::get_anova_table(two_way)

ANOVA Table (type II tests)

      Effect DFn DFd      F        p p<.05   ges
1      Group   2  45  0.187 0.830000       0.008
2       Time   2  45 10.671 0.000161     * 0.322
3 Group:Time   4  45  0.589 0.672000       0.050

Interpretation: We do not observed statistically significant two-way interactions between group and time.

4.7 Pairwise comparisons for Group

# Effect of group at each time point
one.way <- DF |>
            dplyr::group_by(Time) |>
            rstatix::anova_test(dv = Measure, 
              formula = Measure ~ Group,
              wid = ID, 
              within = Group) |>
            rstatix::get_anova_table() |>
            rstatix::adjust_pvalue(method="bonferroni")
print(one.way[,c(1,2,5,6,8)])

# A tibble: 3 × 5
  Time   Effect     F     p   ges
  <fct>  <chr>  <dbl> <dbl> <dbl>
1 Time01 Group  0.919 0.42  0.109
2 Time02 Group  0.118 0.89  0.015
3 Time03 Group  0.215 0.809 0.028

# Comparisons between groups at each time point
pwc <- DF |>
        dplyr::group_by(Time) |>
        rstatix::pairwise_t_test(
        Measure ~ Group, 
        paired = TRUE,
        p.adjust.method = "bonferroni"
)
print(pwc[,c(1,3,4,7,9)])

# A tibble: 9 × 5
  Time   group1   group2   statistic     p
  <fct>  <chr>    <chr>        <dbl> <dbl>
1 Time01 Group 01 Group 02    1.53   0.188
2 Time01 Group 01 Group 03   -0.152  0.885
3 Time01 Group 02 Group 03   -1.36   0.232
4 Time02 Group 01 Group 02   -0.612  0.568
5 Time02 Group 01 Group 03    0.0310 0.976
6 Time02 Group 02 Group 03    0.454  0.669
7 Time03 Group 01 Group 02    0.221  0.834
8 Time03 Group 01 Group 03    0.820  0.450
9 Time03 Group 02 Group 03    0.502  0.637

4.8 Pairwise comparisons for Time

# Effect of time at each level of group
one.way2 <- DF |>
            dplyr::group_by(Group) |>
            rstatix::anova_test(dv = Measure, 
                       formula = Measure ~ Time,
                       wid = ID, 
                       within = Time) |>
            rstatix::get_anova_table() |>
            rstatix::adjust_pvalue(method = "bonferroni")
print(one.way2[,c(1,2,5,6,8)])

# A tibble: 3 × 5
  Group    Effect     F     p   ges
  <fct>    <chr>  <dbl> <dbl> <dbl>
1 Group 01 Time    2.55 0.111 0.254
2 Group 02 Time    7.37 0.006 0.496
3 Group 03 Time    3.03 0.079 0.288

# Pairwise comparisons between time points
pwc2 <- DF |>
        dplyr::group_by(Group) |>
        rstatix::pairwise_t_test(
        Measure ~ Time, 
        paired = TRUE,
        p.adjust.method = "bonferroni"
)
print(pwc2[,c(1,3,4,7,9)])

# A tibble: 9 × 5
  Group    group1 group2 statistic       p
  <fct>    <chr>  <chr>      <dbl>   <dbl>
1 Group 01 Time01 Time02   -0.160  0.879  
2 Group 01 Time01 Time03   -4.20   0.00850
3 Group 01 Time02 Time03   -2.36   0.0646 
4 Group 02 Time01 Time02   -2.46   0.0571 
5 Group 02 Time01 Time03   -3.21   0.0236 
6 Group 02 Time02 Time03   -1.75   0.141  
7 Group 03 Time01 Time02   -0.0604 0.954  
8 Group 03 Time01 Time03   -3.25   0.0228 
9 Group 03 Time02 Time03   -2.39   0.0625 

4.9 Pairwise t-test comparison (if no significant interaction)

If the interaction is not significant, we can simply execute pairwise t-test comparison.

# Comparisons for group variable
res1 = DF |>
  rstatix::pairwise_t_test(
    Measure ~ Group, 
    paired = TRUE,
    p.adjust.method = "bonferroni"
)
res1[,c(2,3,6,8,10)]

# A tibble: 3 × 5
  group1   group2   statistic     p p.adj.signif
  <chr>    <chr>        <dbl> <dbl> <chr>       
1 Group 01 Group 02     0.698 0.495 ns          
2 Group 01 Group 03     0.338 0.739 ns          
3 Group 02 Group 03    -0.340 0.738 ns          

# Comparisons for the time variable
res2 = DF |>
  rstatix::pairwise_t_test(
    Measure ~ Time, 
    paired = TRUE,
    p.adjust.method = "bonferroni"
)
res2[,c(2,3,6,8,10)]

# A tibble: 3 × 5
  group1 group2 statistic         p p.adj.signif
  <chr>  <chr>      <dbl>     <dbl> <chr>       
1 Time01 Time02     -1.32 0.203     ns          
2 Time01 Time03     -5.68 0.0000269 ****        
3 Time02 Time03     -3.88 0.00119   **