| PreBDI | PosBDI | dBDI |
|---|---|---|
| 29 | 28 | 1 |
| 32 | 26 | 6 |
| 33 | 29 | 4 |
| 28 | 26 | 2 |
| 31 | 30 | 1 |
| 32 | 27 | 5 |
| 33 | 27 | 6 |
| 30 | 27 | 3 |
| 29 | 27 | 2 |
| 30 | 28 | 2 |
| 33 | 29 | 4 |
| 31 | 25 | 6 |
40 T-tests
40.1 One-sample T-tests
Application scenario
The application scenario of a one-sample T-test is characterized, as is well known, by considering \(n\) univariate data points from one sample (group) of randomized experimental units, which are assumed to be realizations of \(n\) independent and identically normally distributed random variables. With respect to the identical univariate normal distributions \(N\left(\mu, \sigma^{2}\right)\) of these random variables, both the expectation parameter \(\mu\) and the variance parameter \(\sigma^{2}\) are assumed to be unknown. Finally, one assumes an interest in an inferential comparison of the unknown expectation parameter \(\mu\) with a specified value \(\mu_{0}\), for example \(\mu_{0}:=0\).
Application example
For a concrete application example, we consider the analysis of pre-post intervention BDI difference scores for one group of \(n=12\) patients, as shown in Table 40.1. The first two columns of this table list the patient-specific BDI values before (PreBDI) and after (PosBDI) the psychotherapeutic intervention; the third column, dBDI, shows the corresponding PreBDI-PosBDI difference scores. A positive value here corresponds to an improvement in depressive symptoms, and a negative value to a worsening of depressive symptoms.
When applying a one-sample T-test to the dBDI data of this dataset, we thus assume that the dBDI data are realizations of \(n=12\) independently normally distributed random variables \(y_i \sim N\left(\mu, \sigma^{2}\right)\). We further assume that we are interested in quantifying our uncertainty in the inferential comparison of the true, but unknown, expectation parameter \(\mu\) with a comparison value \(\mu_{0}\) in terms of a hypothesis test.
Independently of this inferential procedure, we first consider the descriptive statistics of the dBDI data, as shown in Table 40.2. In particular, it is noticeable that the sample mean is relatively small compared with the standard deviation. On average, the PreBDI and PosBDI values therefore differ in a positive direction, which implies a reduction in depressive symptoms, but the data also vary considerably across patients, as is already apparent from Table 40.1.
| n | Max | Min | Median | Mean | Var | Std | |
|---|---|---|---|---|---|---|---|
| F2F | 12 | 6 | 1 | 3.5 | 3.5 | 3.73 | 1.93 |
Model formulation
We now define the one-sample T-test model as follows.
Definition 40.1 (One-sample T-test model) For \(i=1, \ldots, n\), let \(y_i\) be random variables that model the \(n\) data points of a one-sample T-test scenario. Then the one-sample T-test model has the structural form \[\begin{equation} y_i = \mu+\varepsilon_{i} \mbox{ with } \varepsilon_{i} \sim N\left(0, \sigma^{2}\right) \mbox{ i.i.d. for } i=1, \ldots, n \mbox{ with } \mu \in \mathbb{R} \mbox{ and } \sigma^{2}>0, \end{equation}\] the data-distribution form \[\begin{equation} y_i \sim N\left(\mu, \sigma^{2}\right) \mbox{ i.i.d. for } i=1, \ldots, n \mbox{ with } \mu \in \mathbb{R} \mbox{ and } \sigma^{2}>0, \end{equation}\] and, for the data vector \(y=\left(y_1, \ldots, y_n\right)^{T}\), the design-matrix form \[\begin{equation} y = X\beta+\varepsilon \mbox{ with } X:=1_{n} \in \mathbb{R}^{n \times 1}, \beta:=\mu \in \mathbb{R}, \varepsilon \sim N\left(0_{n}, \sigma^{2} I_{n}\right) \mbox{ and } \sigma^{2}>0. \end{equation}\]
The one-sample T-test model is evidently identical to the model of independent and identically normally distributed random variables (cf. Chapter 36). The equivalence of the structural, data-distribution, and design-matrix forms of the one-sample T-test model was already discussed in detail in Chapter 36. Accordingly, simulating data based on the one-sample T-test model has the same form as simulating independent and identically normally distributed random variables. The R code below demonstrates this.
# model formulation
library(MASS) # multivariate normal distribution
n = 12 # number of data points
p = 1 # number of beta parameters
X = matrix(rep(1,n), nrow = n) # design matrix
I_n = diag(n) # n x n identity matrix
beta = 5 # true, but unknown, beta parameter
sigsqr = 14 # true, but unknown, variance parameter
# data realization
y = mvrnorm(1, X %*% beta, sigsqr*I_n) # one realization of an n-dimensional random vectorModel estimation
Because the form of the one-sample T-test model is identical to the scenario of independent and identically normally distributed random variables, this also applies to the corresponding beta- and variance-parameter estimators. We thus obtain the following theorem, which was already proved in Chapter 37.
Theorem 40.1 (Parameter estimators in the one-sample T-test model) Let the design-matrix form of the one-sample T-test model be given. Then, for the beta-parameter estimator, \[\begin{equation} \hat{\beta}=\frac{1}{n} \sum_{i=1}^{n} y_i=: \bar{y} \end{equation}\] and for the variance-parameter estimator, \[\begin{equation} \hat{\sigma}^{2}=\frac{1}{n-1} \sum_{i=1}^{n}\left(y_i-\bar{y}\right)^{2}=: s_{y}^{2}. \end{equation}\] Here, \(\bar{y}\) and \(s_{y}^{2}\) again denote the sample mean and sample variance of the random variables \(y_{1}, \ldots, y_{n}\).
40.2 Model evaluation
Based on Definition 38.1, we now formulate the T-test statistic for the one-sample T-test scenario and state its frequentist distribution.
Theorem 40.2 (T-test statistic of the one-sample T-test) Let the design-matrix form of the one-sample T-test model be given. Then, for the T-test statistic with \[\begin{equation} c:=1 \mbox{ and } c^{T} \beta_{0}=: \mu_{0}, \end{equation}\] we obtain \[\begin{equation} T = \sqrt{n}\left(\frac{\bar{y}-\mu_{0}}{s_{y}}\right), \end{equation}\] and \[\begin{equation} T \sim t(\delta, n-1) \mbox{ with } \delta=\sqrt{n}\left(\frac{\mu-\mu_{0}}{\sigma}\right). \end{equation}\]
Proof. By Theorem 38.1, \[\begin{equation} T =\frac{c^{T} \hat{\beta}-c^{T} \beta_{0}}{\sqrt{\hat{\sigma}^{2} c^{T}\left(X^{T} X\right)^{-1} c}} =\frac{1^{T} \bar{y}-1^{T} \mu_{0}}{\sqrt{s_{y}^{2} 1^{T}\left(1_{n}^{T} 1_{n}\right)^{-1} 1}} =\sqrt{n}\left(\frac{\bar{y}-\mu_{0}}{s_{y}}\right). \end{equation}\] Furthermore, by the same theorem, \[\begin{equation} \delta =\frac{c^{T} \beta-c^{T} \beta_{0}}{\sqrt{\sigma^{2} c^{T}\left(X^{T} X\right)^{-1} c}} =\frac{1^{T} \mu-1^{T} \mu_{0}}{\sqrt{\sigma^{2} 1^{T}\left(1_{n}^{T} 1_{n}\right)^{-1} 1}}=\sqrt{n}\left(\frac{\mu-\mu_{0}}{\sigma}\right). \end{equation}\]
The forms of the T-test statistic and its distribution in the one-sample T-test special case of the GLM are therefore naturally identical to the corresponding forms in the GLM-free context. The corresponding theory for confidence intervals and for controlling the size of one-sample T-tests, as well as the use of the power function for evaluating test power, therefore follows analogously.
Application example
The following R code demonstrates the evaluation of a 95% confidence interval for the expectation parameter \(\mu\) and the implementation of a two-sided one-sample T-test with simple null hypothesis \(\Theta_{0}:=\{0\}\) and significance level \(\alpha_{0}:=0.05\) for the application example outlined above.
# data analysis
D = read.csv("./_data/507-t-tests.csv") # load dataset
y = D$dBDI[D$COND == "F2F"] # pre-post difference scores
n = length(y) # number of data points
p = 1 # number of beta parameters
c = 1 # contrast-weight vector
mu_0 = 0 # null-hypothesis parameter
delta = 0.95 # confidence level
alpha_0 = 0.05 # significance level
X = matrix(rep(1,n), nrow = n) # one-sample T-test design matrix
beta_hat = solve(t(X)%*%X)%*%t(X)%*%y # beta-parameter estimator
eps_hat = y - X %*% beta_hat # residual vector
sigsqr_hat = (t(eps_hat) %*% eps_hat)/(n-p) # variance-parameter estimator
t_delta = qt((1+delta)/2,n-1) # \Psi^{-1}((1+\delta)/2, n-1)
lambda = diag(solve(t(X) %*% X)) # \lambda_j values
kappa_u = beta_hat - sqrt(sigsqr_hat*lambda)*t_delta # lower confidence-interval bound
kappa_o = beta_hat + sqrt(sigsqr_hat*lambda)*t_delta # upper confidence-interval bound
t_num = t(c) %*% beta_hat - mu_0 # numerator of the one-sample T-test statistic
t_den = sqrt(sigsqr_hat %*% t(c) %*% solve(t(X) %*% X) %*% c) # denominator of the one-sample T-test statistic
t = t_num/t_den # value of the one-sample T-test statistic
pval = 2*(1 - pt(abs(t), n-1)) # p-value for a two-sided one-sample T-test
k_alpha_0 = qt(1-alpha_0/2, n-1) # critical value
if(abs(t) > k_alpha_0){phi = 1} else {phi = 0} # one-sample T-testBeta-parameter estimator : 3.5
95% confidence interval : 2.27 4.73
Variance-parameter estimator : 3.73
alpha_0 : 0.05
Critical value : 2.2
One-sample T-test statistic : 6.28
phi : 1
p-value : 0In this case, the null hypothesis is rejected at a critical value of \(k_{0.05}=2.20\) and a value of the T-statistic of \(T=6.28\). The 95% confidence interval for the true, but unknown, expectation parameter is \([2.27, 4.73]\) and therefore does not cover the null-hypothesis parameter value \(\mu_0=0\).
40.3 Two-sample T-tests
Application scenario
The application scenario of a two-sample T-test for independent samples is characterized, as is well known, by considering a total of \(n\) univariate data points from two samples (groups) of randomized experimental units. It is assumed in particular that the \(n_{1}\) univariate data points of the first group are realizations of \(n_{1}\) independent and identically normally distributed random variables with expectation parameter \(\mu_{1}\) and variance parameter \(\sigma^{2}\), while the \(n_{2}\) univariate data points of the second group are assumed to be realizations of \(n_{2}\) independent and identically normally distributed random variables with expectation parameter \(\mu_{2}\) and variance parameter \(\sigma^{2}\). Thus, in particular, the true, but unknown, expectation parameters of both groups of random variables may differ, whereas the variance parameters of both groups are assumed to be identical. Finally, one assumes an interest in the inferential comparison of the unknown expectation parameters \(\mu_{1}\) and \(\mu_{2}\), for example their equality \(\mu_{1}=\mu_{2}\) or inequality \(\mu_{1} \neq \mu_{2}\).
Application example
For a concrete application example, we consider the analysis of pre-post intervention BDI difference scores from two groups of 12 patients each in different therapy settings, as shown in Table 40.3. The first column of the table (COND) lists the patient-specific therapy setting (F2F: face-to-face, ONL: online). The second column (dBDI) lists the corresponding patient-specific pre-post intervention BDI difference scores. Positive values again correspond to a decrease in depressive symptoms, and negative values to an increase in depressive symptoms.
| COND | dBDI |
|---|---|
| F2F | 1 |
| F2F | 6 |
| F2F | 4 |
| F2F | 2 |
| F2F | 1 |
| F2F | 5 |
| F2F | 6 |
| F2F | 3 |
| F2F | 2 |
| F2F | 2 |
| F2F | 4 |
| F2F | 6 |
| ONL | 9 |
| ONL | 2 |
| ONL | 10 |
| ONL | -1 |
| ONL | 4 |
| ONL | 11 |
| ONL | 10 |
| ONL | 15 |
| ONL | 4 |
| ONL | 5 |
| ONL | 10 |
| ONL | 10 |
When applying a two-sample T-test to the dBDI data, we assume that the 12 data points of the F2F therapy group are realizations of \(n_{1}=12\) independent and identically normally distributed random variables \(y_{1j} \sim N\left(\mu_{1}, \sigma^{2}\right)\) with \(j=1, \ldots, n_{1}\) and that the 12 data points of the ONL therapy group are realizations of \(n_{2}=12\) independent and identically normally distributed random variables \(y_{2j} \sim N\left(\mu_{2}, \sigma^{2}\right)\) with \(j=1, \ldots, n_{2}\).
Independently of the inferential procedure described below, we again first consider the descriptive statistics of the therapy-setting-specific dBDI values. These are listed in Table 40.4.
| n | Max | Min | Median | Mean | Var | Std | |
|---|---|---|---|---|---|---|---|
| F2F | 12 | 6 | 1 | 3.5 | 3.50 | 3.73 | 1.93 |
| ONL | 12 | 15 | -1 | 9.5 | 7.42 | 20.81 | 4.56 |
Model formulation
Using the index \(i\) for the groups and the index \(j\) for the experimental units in each group, we define the two-sample T-test model as follows.
Definition 40.2 (Two-sample T-test model) For \(i=1,2\) and \(j=1,\ldots,n_{i}\), let \(y_{ij}\) be random variables that model the \(n=n_{1}+n_{2}\) data points of a two-sample T-test scenario. Then the two-sample T-test model has the structural form \[\begin{equation} y_{ij} = \mu_{i}+\varepsilon_{ij} \mbox{ with } \varepsilon_{ij} \sim N\left(0, \sigma^{2}\right) \mbox{ i.i.d. with } \mu_{i} \in \mathbb{R} \mbox{ and } \sigma^{2}>0, \end{equation}\] the data-distribution form \[\begin{equation} y_{ij} \sim N\left(\mu_{i}, \sigma^{2}\right) \mbox{ i.i.d. with } \mu_{i} \in \mathbb{R} \mbox{ and } \sigma^{2}>0, \end{equation}\] and, for the \(n\)-dimensional data vector defined as \[\begin{equation} y := \left(y_{11}, \cdots, y_{1n_{1}}, y_{21}, \cdots, y_{2n_{2}}\right)^{T}, \end{equation}\] the design-matrix form \[\begin{equation} y = X\beta+\varepsilon \end{equation}\] with \[\begin{equation} X:= \begin{pmatrix} 1_{n_{1}} & 0_{n_{1}} \\ 0_{n_{2}} & 1_{n_{2}} \end{pmatrix} \in \mathbb{R}^{n \times 2}, \beta:= \begin{pmatrix} \mu_{1} \\ \mu_{2} \end{pmatrix} \in \mathbb{R}^{2}, \varepsilon \sim N\left(0_{n}, \sigma^{2} I_{n}\right), \sigma^{2}>0 . \end{equation}\]
The definition of the two-sample T-test model in design-matrix form chosen here is not the only possible one, but it is the one under which the equivalence to the two-sample T-test model in the GLM-free context is most clearly visible. In Chapter 41, we will encounter an alternative parameterization of the two-sample T-test model as well. As in the one-sample T-test scenario, the equivalence of the model forms formulated in Definition 40.2 follows from the results in Chapter 36. Apart from the definition of the design matrix and beta-parameter vector, simulating data based on the two-sample T-test model is identical to the simulations of GLM special cases already known, as the following R code demonstrates.
# model formulation
library(MASS) # multivariate normal distribution
n_1 = 12 # number of data points in group 1
n_2 = 12 # number of data points in group 2
n = n_1 + n_2 # total number of data points
p = 2 # number of beta parameters
X = matrix(c(rep(1,n_1), rep(0,n_1), # design matrix
rep(0,n_2), rep(1,n_2)),
nrow = n)
I_n = diag(n) # n x n identity matrix
beta = matrix(c(1,2), nrow = p) # true, but unknown, beta parameter
sigsqr = 10 # true, but unknown, variance parameter
# data realization
y = mvrnorm(1, X %*% beta, sigsqr*I_n) # one realization of an n-dimensional random vectorModel estimation
The two beta-parameter components of the two-sample T-test model in design-matrix form are, unsurprisingly, estimated by the corresponding group sample means. For the variance-parameter estimator, the so-called pooled sample variance results. These are the two core statements of the following theorem.
Theorem 40.3 (Parameter estimation in the two-sample T-test model) Let the design-matrix form of the two-sample T-test model be given. Then, for the beta-parameter estimator, \[\begin{equation} \hat{\beta} =\begin{pmatrix} \frac{1}{n_{1}} \sum_{j=1}^{n_{1}} y_{1j} \\ \frac{1}{n_{2}} \sum_{j=1}^{n_{2}} y_{2j} \end{pmatrix}=:\begin{pmatrix} \bar{y}_{1} \\ \bar{y}_{2} \end{pmatrix} \end{equation}\] and, for the variance-parameter estimator, \[\begin{equation} \hat{\sigma}^{2} = \frac{\sum_{j=1}^{n_{1}}\left(y_{1j}-\bar{y}_{1}\right)^{2}+\sum_{j=1}^{n_{2}}\left(y_{2j}-\bar{y}_{2}\right)^{2}}{n_{1}+n_{2}-2} =: s_{12}^{2}. \end{equation}\]
Proof. For \(i=1,2\), let \(y_i:=\left(y_{i1}, \ldots, y_{i n_{i}}\right)^{T}\). Then, for the beta-parameter estimator, \[\begin{align} \begin{split} \hat{\beta} & = \left(X^{T} X\right)^{-1} X^{T} y \\ & = \left(\begin{pmatrix} 1_{n_{1}} & 0_{n_{2}} \\ 0_{n_{1}} & 1_{n_{2}} \end{pmatrix} \begin{pmatrix} 1_{n_{1}} & 0_{n_{1}} \\ 0_{n_{2}} & 1_{n_{2}} \end{pmatrix}\right)^{-1} \begin{pmatrix} 1_{n_{1}} & 0_{n_{2}} \\ 0_{n_{1}} & 1_{n_{2}} \end{pmatrix} \begin{pmatrix} y_{1} \\ y_{2} \end{pmatrix} \\ & = \begin{pmatrix} n_{1} & 0 \\ 0 & n_{2} \end{pmatrix}^{-1} \begin{pmatrix} \sum_{j=1}^{n_{1}} y_{1j} \\ \sum_{j=1}^{n_{2}} y_{2j} \end{pmatrix} \\ & = \begin{pmatrix} n_{1}^{-1} & 0 \\ 0 & n_{2}^{-1} \end{pmatrix} \begin{pmatrix} \sum_{j=1}^{n_{1}} y_{1j} \\ \sum_{j=1}^{n_{2}} y_{2j} \end{pmatrix} \\ & = \begin{pmatrix} \frac{1}{n_{1}} \sum_{j=1}^{n_{1}} y_{1j} \\ \frac{1}{n_{2}} \sum_{j=1}^{n_{2}} y_{2j} \end{pmatrix} \\ & =: \begin{pmatrix} \bar{y}_{1} \\ \bar{y}_{2} \end{pmatrix}. \end{split} \end{align}\] Similarly, for the variance-parameter estimator with \(n=n_{1}+n_{2}\) and \(p=2\), \[\begin{align} \begin{split} \hat{\sigma}^{2} & =\frac{(y - X \hat{\beta})^{T}(y - X \hat{\beta})}{n-p} \\ & =\frac{1}{n_{1}+n_{2}-2} \left( \begin{pmatrix} y_{1} \\ y_{2} \end{pmatrix}- \begin{pmatrix} 1_{n_{1}} & 0_{n_{1}} \\ 0_{n_{2}} & 1_{n_{2}} \end{pmatrix} \begin{pmatrix} \bar{y}_{1} \\ \bar{y}_{2} \end{pmatrix}\right)^{T} \left(\begin{pmatrix} y_{1} \\ y_{2} \end{pmatrix} -\begin{pmatrix} 1_{n_{1}} & 0_{n_{1}} \\ 0_{n_{2}} & 1_{n_{2}} \end{pmatrix}\begin{pmatrix} \bar{y}_{1} \\ \bar{y}_{2} \end{pmatrix}\right) \\ & = \frac{1}{n_{1}+n_{2}-2} \begin{pmatrix} y_{11}-\bar{y}_{1} \\ \vdots \\ y_{1 n_{1}}-\bar{y}_{1} \\ y_{21}-\bar{y}_{2} \\ \vdots \\ y_{2 n_{2}}-\bar{y}_{2} \end{pmatrix}^{T}\begin{pmatrix} y_{11}-\bar{y}_{1} \\ \vdots \\ y_{1 n_{1}}-\bar{y}_{1} \\ y_{21}-\bar{y}_{2} \\ \vdots \\ y_{2 n_{2}}-\bar{y}_{2} \end{pmatrix} \\ & = \frac{\sum_{j=1}^{n_{1}}\left(y_{1j}-\bar{y}_{1}\right)^{2}+\sum_{j=1}^{n_{2}}\left(y_{2j}-\bar{y}_{2}\right)^{2}}{n_{1}+n_{2}-2} \\ & =: s_{12}^{2}. \end{split} \end{align}\]
Note that the sample variance \(s_{y}^{2}\) of the components of \(y\) generally differs from the pooled sample variance \(s_{12}^{2}\). This is due not least to the fact that the sample variance is based on the overall sample mean \(\bar{y}\), whereas the pooled sample variance is based on the group-specific sample means \(\bar{y}_{1}\) and \(\bar{y}_{2}\). We do not want to deepen the concept of pooled sample variance further here.
Model evaluation
Based on Theorem 38.1, we now formulate the T-test statistic for the two-sample T-test model defined in design-matrix form in Definition 40.2 and state its frequentist distribution.
Theorem 40.4 (T-test statistic of the two-sample T-test) Let the design-matrix form of the two-sample T-test be given. Then, for the T-test statistic with \[\begin{equation} c:=(1,-1)^{T} \mbox{ and } c^{T} \beta_{0}=: \mu_{0}, \end{equation}\] we obtain \[\begin{equation} T =\sqrt{\frac{n_{1} n_{2}}{n_{1}+n_{2}}} \left(\frac{\bar{y}_{1}-\bar{y}_{2}-\mu_{0}}{s_{12}}\right), \end{equation}\] and \[\begin{equation} T \sim t\left(\delta, n_{1}+n_{2}-2\right) \mbox{ with } \delta=\sqrt{\frac{n_{1} n_{2}}{n_{1}+n_{2}}}\left(\frac{\mu_{1}-\mu_{2}-\mu_{0}}{\sigma}\right). \end{equation}\]
Proof. By Theorem 38.1, for the numerators of \(T\) and \(\delta\), \[\begin{equation} c^{T} \hat{\beta}-c^{T} \beta_{0} =\begin{pmatrix} 1 & -1 \end{pmatrix} \begin{pmatrix} \bar{y}_{1} \\ \bar{y}_{2} \end{pmatrix}-\mu_{0} =\bar{y}_{1}-\bar{y}_{2}-\mu_{0} \end{equation}\] and \[\begin{equation} c^{T} \beta-c^{T} \beta_{0} =\begin{pmatrix} 1 & -1 \end{pmatrix} \begin{pmatrix} \mu_{1} \\ \mu_{2} \end{pmatrix}-\mu_{0} =\mu_{1}-\mu_{2}-\mu_{0}, \end{equation}\] respectively. Furthermore, for the denominators of \(T\) and \(\delta\), \[\begin{equation} c^{T}\left(X^{T}X\right)^{-1}c = \begin{pmatrix} 1 & -1 \end{pmatrix} \begin{pmatrix} n_{1}^{-1} & 0 \\ 0 & n_{2}^{-1} \end{pmatrix} \begin{pmatrix} 1 \\ -1 \end{pmatrix} = \begin{pmatrix} n_{1}^{-1} & -n_{2}^{-1} \end{pmatrix} \begin{pmatrix} 1 \\ -1 \end{pmatrix} =\frac{1}{n_{1}}+\frac{1}{n_{2}}. \end{equation}\] In addition, \[\begin{equation} \left(\frac{1}{n_{1}}+\frac{1}{n_{2}}\right)^{-\frac{1}{2}}=\left(\frac{n_{2}}{n_{1} n_{2}}+\frac{n_{1}}{n_{1} n_{2}}\right)^{-\frac{1}{2}}=\left(\frac{n_{1}+n_{2}}{n_{1} n_{2}}\right)^{-\frac{1}{2}}=\left(\frac{n_{1} n_{2}}{n_{1}+n_{2}}\right)^{\frac{1}{2}}. \end{equation}\] Taken together, it follows directly that \[\begin{equation} T = \sqrt{\frac{n_{1} n_{2}}{n_{1}+n_{2}}} \left(\frac{\bar{y}_{1}-\bar{y}_{2}-\mu_{0}}{s_{12}}\right) \mbox{ and } \delta=\sqrt{\frac{n_{1} n_{2}}{n_{1}+n_{2}}}\left(\frac{\mu_{1}-\mu_{2}-\mu_{0}}{\sigma}\right). \end{equation}\]
The forms of the T-test statistic and its distribution in the two-sample T-test model in design-matrix form are therefore again naturally identical to the corresponding forms in the GLM-free context. The corresponding theory for controlling the size of two-sample T-tests and the use of the power function for evaluating test power therefore follows analogously.
Application example
The following R code demonstrates the evaluation of 95% confidence intervals for the expectation parameters \(\mu_{1}\) and \(\mu_{2}\) and the implementation of a two-sided two-sample T-test with null hypothesis \[\begin{equation} \Theta_{0} := \left\{ \begin{pmatrix} \mu_{1} \\ \mu_{2} \end{pmatrix} \in \mathbb{R}^{2} \mid \mu_{1}=\mu_{2} \right\} \end{equation}\] and significance level \(\alpha_{0}:=0.05\) for the application example outlined above.
# data loading
D = read.csv("./_data/507-t-tests.csv") # dataframe
y_1 = D$dBDI[D$COND == "F2F"] # BDI difference scores in the F2F group
y_2 = D$dBDI[D$COND == "ONL"] # BDI difference scores in the ONL group
# model formulation
n_1 = length(y_1) # number of data points in group 1 (F2F)
n_2 = length(y_2) # number of data points in group 2 (ONL)
n = n_1 + n_2 # total number of data points
y = matrix(c(y_1, y_2), nrow = n) # data vector
p = 2 # number of beta parameters
X = matrix(c(rep(1,n_1), rep(0,n_2), # two-sample T-test design matrix
rep(0,n_1), rep(1,n_2)),
nrow = n)
# parameter estimation
beta_hat = solve(t(X) %*% X) %*% t(X) %*% y # beta-parameter estimator
eps_hat = y - X %*% beta_hat # residual vector
sigsqr_hat = (t(eps_hat) %*% eps_hat) /(n-p) # variance-parameter estimator
# confidence interval
delta = 0.95 # confidence level
t_delta = qt((1+delta)/2,n-p) # \Psi^{-1}((1+\delta)/2,n-p)
lambda = diag(solve(t(X) %*% X)) # \lambda_j values
kappa = matrix(rep(NaN,p*2), nrow = p) # \beta_j confidence-interval array
for(j in 1:p){ # iteration over \beta_j
kappa[j,1] = beta_hat[j]-sqrt(sigsqr_hat*lambda[j])*t_delta # lower CI bound
kappa[j,2] = beta_hat[j]+sqrt(sigsqr_hat*lambda[j])*t_delta} # upper CI bound
# hypothesis test
c = matrix(c(1,-1), nrow = 2) # contrast-weight vector
mu_0 = 0 # null hypothesis H_0
alpha_0 = 0.05 # significance level
k_alpha_0 = qt(1 - (alpha_0/2), n-p) # critical value
t_num = t(c) %*% beta_hat - mu_0 # T-test statistic numerator
t_den = sqrt(sigsqr_hat*t(c) %*% solve(t(X) %*% X)%*%c) # T-test statistic denominator
t = t_num/t_den # T-test statistic
if(abs(t) >= k_alpha_0){phi = 1} else {phi = 0} # test 1_{|T(X) >= k_alpha_0|}
pval = 2*(1-pt(abs(t), n_1+n_2-2)) # p-valueBeta-parameter estimator : 3.5 7.42
95% confidence intervals : 1.4 5.32 5.6 9.51
Variance-parameter estimator : 12.27
alpha_0 : 0.05
Critical value : 2.07
Two-sample T-test statistic : -2.74
phi : 1
p-value : 0.01In this case, the null hypothesis is rejected at a critical value of \(k_{0.05}=2.07\) and a value of the T-statistic of \(T=-2.74\). Inferentially, there is therefore evidence that the true, but unknown, expectation parameter in the F2F therapy setting differs from the true, but unknown, expectation parameter in the ONL therapy setting. The 95% confidence intervals for the true, but unknown, expectation parameters \(\mu_{1}\) and \(\mu_{2}\) are \([1.40,5.60]\) and \([5.32,9.51]\), respectively.
40.4 Bibliographic remarks
Although the frequentist literature of the first half of the twentieth century is permeated by the equivalence of regression and analysis-of-variance linear models, it is difficult to name a definite source that would have priority with respect to describing T-tests as special cases of the GLM. We therefore refer here more generally to Fisher (1925) and Fisher (1935).
Fisher, R. A. (1925). Theory of Statistical Estimation. Mathematical Proceedings of the Cambridge Philosophical Society, 22(5), 700–725. https://doi.org/10.1017/S0305004100009580
Fisher, R. A. (1935). The design of experiments (1. ed). Hafner Press.