How does Welch’s t-test determine the degrees of freedom?
In the traditional two-sample \(t\)-test, the two groups are assumed to have the same population variance. This leads to a simple expression for the degrees of freedom:
\[\large df=n_1+n_2-2. \]
Welch’s \(t\)-test does not assume that the population variances are equal. As a result, the uncertainty in the estimated standard error is a little more complicated.
Welch’s test uses the Welch-Satterthwaite approximation. First, define
\[ v_1=\frac{s_1^2}{n_1},\qquad v_2=\frac{s_2^2}{n_2}, \]
where \(v_1\) and \(v_2\) are the estimated variances of the two sample means. The estimated variance of their difference is therefore
\[\large v_1+v_2. \]
The degrees of freedom are approximated by
\[\large df = \frac{ \left( v_1 + v_2 \right)^2 }{ \frac{ v_1^2 }{ n_1-1 } + \frac{ v_2^2 }{ n_2-1 } }. \]
Unlike \(n_1 + n_2 - 2\), this value does not have to be an integer.
What can we learn from the formula?
A few features of the approximation help explain how Welch’s test works:
Scaling both variance contributions by the same amount does not change the degrees of freedom. If \(v_1\) and \(v_2\) become \(cv_1\) and \(cv_2\), the factor \(c^2\) appears in both the numerator and denominator and cancels.
With equal sample sizes and equal variance contributions, we recover the usual degrees of freedom. If \(n_1=n_2=n\) and \(v_1=v_2=v\),
\[\large df = \frac{(2v)^2} {\dfrac{v^2}{n-1}+\dfrac{v^2}{n-1}} = 2(n-1) = n_1+n_2-2. \]
- As the variance contribution becomes concentrated in one group, the effective degrees of freedom decrease. In the extreme, if \(v_2\) approaches zero,
\[\large df \longrightarrow n_1-1. \]
- A large variance contribution from a smaller group can reduce the degrees of freedom particularly strongly. Its contribution to the denominator is \(v_i^2/(n_i-1)\), so estimating a large variance contribution from relatively few observations increases the denominator and reduces the effective degrees of freedom.
So Welch’s test makes an important distinction:
The amount of variability affects the standard error. How that variability is distributed between the two groups, together with their sample sizes, affects the effective degrees of freedom.
The degrees of freedom determine which \(t\) distribution is used to describe the uncertainty in the standardized statistic.
The smaller the effective degrees of freedom, the heavier the tails of the corresponding \(t\) distribution, reflecting greater uncertainty in the estimated standard error.
Further reading: Delacre, M., Lakens, D., & Leys, C. (2017). Why psychologists should by default use Welch’s t-test instead of Student’s t-test. International Review of Social Psychology, 30(1), 92–101. doi:10.5334/irsp.82.