Print this page and work through it on paper. Revealed answers are never printed, so you can check one question and still print a clean sheet.

1. 1.Computation · 9 min

   Three brands of battery are tested for lifetime, in hours. The observations are

   Brand A: $2,4,6$. Brand B: $5,7,7,9,7$. Brand C: $8,10,12,10$.

   Use $F_{0.05,2,9}=4.256$, and keep every quantity as an exact fraction.

   1. (a)Compute the three group means, the grand mean, $SSB$ and $SSW$. Explain why the grand mean is not $(4+7+10)/3$.
   2. (b)Write out the ANOVA table — sums of squares, degrees of freedom, mean squares — and the $F$ statistic. Test $H_0:{\mu }_A={\mu }_B={\mu }_C$ at $\alpha =0.05$, and state exactly what the conclusion does and does not say.
   3. (c)Compute $SST$ directly from the twelve observations and confirm the decomposition of the total sum of squares, degrees of freedom included. Then compute ${\eta }^2$ and say what it measures that the $F$-test does not.
   Show the solution to question 1

   (a) Group means and the grand mean. ${\bar{x}}_A=12/3=4$, ${\bar{x}}_B=35/5=7$, ${\bar{x}}_C=40/4=10$. The grand mean is the mean of all $N=3+5+4=12$ observations, which is the group means weighted by group size:

   $$\bar{x}=\frac{12+35+40}{12}=\frac{87}{12}=\frac{29}{4}.$$

   The unweighted average $(4+7+10)/3=7$ gives Brand A's three observations the same weight as Brand B's five. The grand mean has to reproduce the total, and $7\times 12=84\ne 87$.

   Between. The group means sit at $4-\frac{29}{4}=-\frac{13}{4}$, $7-\frac{29}{4}=-\frac{1}{4}$ and $10-\frac{29}{4}=\frac{11}{4}$ from the grand mean, so

   $$SSB=\sum _i{n}_i({\bar{x}}_i-\bar{x}{)}^2=3\cdot \frac{169}{16}+5\cdot \frac{1}{16}+4\cdot \frac{121}{16}=\frac{507+5+484}{16}=\frac{996}{16}=\frac{249}{4}.$$

   Within. Deviations from each group's own mean. Brand A: $-2,0,2$, squares summing to $8$. Brand B: $-2,0,0,2,0$, sum $8$. Brand C: $-2,0,2,0$, sum $8$. So $SSW=24$.

   (b) The table. Degrees of freedom are $k-1=2$ between and $N-k=12-3=9$ within. Then $MSB=\frac{249}{4}\cdot \frac{1}{2}=\frac{249}{8}$ and $MSW=\frac{24}{9}=\frac{8}{3}$, and

   $$F=\frac{MSB}{MSW}=\frac{249}{8}\cdot \frac{3}{8}=\frac{747}{64}\approx \mathrm{11.67.}$$

   In table form: Between — $SS=249/4$, df $2$, $MS=249/8$; Within — $SS=24$, df $9$, $MS=8/3$; Total — $SS=345/4$, df $11$; $F=747/64$.

   Since $11.67>4.256$, reject $H_0$ at the $5\%$ level: the three brands do not all have the same mean lifetime. That is all it says. The alternative is “not all equal”, so the test does not identify which brands differ, and it does not say that all three differ from each other — a post-hoc procedure is needed for that.

   (c) The identity as an arithmetic check. First, ${\sum }_{i,j}{x}_{ij}^2=(4+16+36)+(25+49+49+81+49)+(64+100+144+100)=56+253+408=717$. Expanding the square, $\sum (x_{ij}-\bar{x}{)}^2=\sum x_{ij}^2-2\bar{x}\sum x_{ij}+N{\bar{x}}^2=\sum x_{ij}^2-N{\bar{x}}^2$, so

   $$SST=717-12\cdot \frac{841}{16}=717-\frac{2523}{4}=\frac{2868-2523}{4}=\frac{345}{4}.$$

   And $SSB+SSW=\frac{249}{4}+\frac{96}{4}=\frac{345}{4}$ ✓, by the decomposition of the total sum of squares. The degrees of freedom agree too: $2+9=11=N-1$ ✓.

   Effect size:

   $${\eta }^2=\frac{SSB}{SST}=\frac{249/4}{345/4}=\frac{249}{345}=\frac{83}{115}\approx \mathrm{0.72.}$$

   About $72\%$ of the variation in lifetime is attributable to brand. That is a statement about how much brand matters. The $F$-test answers only whether any difference exists, and with enough data it rejects for differences of no practical size; ${\eta }^2$ is what separates “detectable” from “large”.

2. 2.Proof · 6 min

   Let group $i$ have $n_i$ observations $x_{ij}$, mean ${\bar{x}}_i$ and sample variance $s_i^2=\frac{1}{n_i-1}{\sum }_j(x_{ij}-{\bar{x}}_i{)}^2$, and let $N={\sum }_i{n}_i$ with $k$ groups. Prove each statement directly from these definitions.

   1. (a)The grand mean is $\bar{x}=\frac{1}{N}{\sum }_i{n}_i{\bar{x}}_i$, and with equal group sizes it is the plain average of the group means.
   2. (b)$MSW=\frac{{\sum }_i(n_i-1)s_i^2}{{\sum }_i(n_i-1)}$ — a weighted average of the group sample variances with weights $n_i-1$ — and with equal group sizes it is the plain average of the $s_i^2$.
   3. (c)If every group has the same sample variance $s^2$, then $MSW=s^2$ whatever the group sizes. Then say in one sentence what (b) predicts when the group sizes are unequal and the variances differ.
   Show the solution to question 2

   (a) The grand mean. By definition $\bar{x}$ is the total divided by $N$. The total is a sum over groups of each group's total, and a group's total is ${\sum }_j{x}_{ij}=n_i{\bar{x}}_i$ by the definition of ${\bar{x}}_i$. So

   $$\bar{x}=\frac{1}{N}\sum _i\sum _j{x}_{ij}=\frac{1}{N}\sum _i{n}_i{\bar{x}}_i.$$

   If every $n_i=n$ then $N=kn$ and $\bar{x}=\frac{1}{kn}{\sum }_{i}n{\bar{x}}_i=\frac{1}{k}{\sum }_i{\bar{x}}_i$, the plain average. $■$

   (b) Within mean square. By the definition of $s_i^2$, group $i$'s within sum of squares is ${\sum }_j(x_{ij}-{\bar{x}}_i{)}^2=(n_i-1)s_i^2$. Summing over groups,

   $$SSW=\sum _i(n_i-1)s_i^2,\text{and}N-k=\sum _i{n}_i-\sum _{i}1=\sum _i(n_i-1).$$

   Hence $MSW=SSW/(N-k)={\sum }_i(n_i-1)s_i^2/{\sum }_i(n_i-1)$, which is a weighted average of the $s_i^2$ because the weights $n_i-1$ are positive and the denominator is their sum. With all $n_i=n$ the weights are all $n-1$, so

   $$MSW=\frac{(n-1){\sum }_i{s}_i^2}{k(n-1)}=\frac{1}{k}\sum _i{s}_i^2.■$$

   (c) Equal variances. If $s_i^2=s^2$ for every $i$, then by (b)

   $$MSW=\frac{s^2{\sum }_i(n_i-1)}{{\sum }_i(n_i-1)}=s^2,$$

   independently of the $n_i$. $■$

   When the sizes and the variances are both unequal, (b) says $MSW$ is pulled toward the variances of the large groups, since they carry most of the weight — so it understates the noise in the small groups if those are the more variable ones, and overstates it if they are the less variable ones.

3. 3.Computation · 7 min

   Three designs each compare $k=3$ groups. The group sizes and sample standard deviations are

   (i) $n=(10,10,10)$, $s=(2,6,6)$; (ii) $n=(40,5,5)$, $s=(2,6,6)$; (iii) $n=(40,5,5)$, $s=(6,2,2)$.

   1. (a)Compute $MSW$ for each design as an exact fraction, and compare it with the variance of the two smaller groups (in (i), of groups 2 and 3).
   2. (b)For each design say whether the ordinary $F$-test is approximately valid, anti-conservative (rejects too often) or conservative, and explain the mechanism in terms of which groups carry the weight in $MSW$.
   3. (c)What single analysis would you recommend for (ii) and (iii)? Explain why a preliminary test of equal variances is not the right way to decide.
   Show the solution to question 3

   (a) The three $MSW$ values. $MSW$ is the weighted average ${\sum }_i(n_i-1)s_i^2/(N-k)$, with weights $n_i-1$.

   (i) Weights $9,9,9$ and variances $4,36,36$:

   $$MSW=\frac{9\cdot 4+9\cdot 36+9\cdot 36}{27}=\frac{684}{27}=\frac{76}{3}\approx 25.3,$$

   the plain average of $4$, $36$ and $36$. It sits between the group variances, closer to the $36$ shared by two of the three groups.

   (ii) Weights $39,4,4$ and variances $4,36,36$:

   $$MSW=\frac{39\cdot 4+4\cdot 36+4\cdot 36}{47}=\frac{156+144+144}{47}=\frac{444}{47}\approx \mathrm{9.4.}$$

   The two small groups have variance $36$, nearly four times $MSW$.

   (iii) Weights $39,4,4$ and variances $36,4,4$:

   $$MSW=\frac{39\cdot 36+4\cdot 4+4\cdot 4}{47}=\frac{1404+16+16}{47}=\frac{1436}{47}\approx \mathrm{30.6.}$$

   The two small groups have variance $4$; $MSW$ overstates their noise by a factor of about $7\frac{1}{2}$.

   (b) Which way each design fails. In every design the variance ratio is $36/4=9$, beyond the factor-of-four rule of thumb, so the difference between the designs is entirely the group sizes.

   (i) Approximately valid. With equal group sizes the weights are equal, $MSW$ is a fair average, and the $F$-test is robust to unequal variances. The true level stays close to $5\%$.

   (ii) Anti-conservative. This is the dangerous pairing: small groups with the large variances. The $39$ degrees of freedom of the tight group dominate $MSW$, so any comparison that involves group 2 or 3 is divided by a denominator too small by almost a factor of four. $F$ comes out too large and the test rejects far more often than $5\%$ when $H_0$ is true.

   (iii) Conservative. The reverse pairing. The large group is the variable one and pulls $MSW$ up, so differences involving the small, tight groups are divided by a denominator that is far too large. The level is below nominal and power is lost; rejections that do occur are trustworthy, but real differences go undetected.

   (c) The recommendation. Use Welch's ANOVA for both (ii) and (iii), unconditionally. It estimates each group's variance separately and adjusts the degrees of freedom, exactly as Welch's $t$-test does for two groups, and it costs almost nothing when the variances turn out to be equal.

   Testing for equal variances first and choosing the main test on the result is a two-stage procedure whose overall error rate nobody computes. It is also weak precisely where it is needed: with $n=5$ in two of the groups, Levene's test has little power even against a nine-fold variance ratio, so a non-rejection would say almost nothing. The group sizes are known before any data are collected, and unequal sizes are reason enough.

4. 4.Computation · 8 min

   Four diets are compared with $n=6$ animals each. The mean weight gains are $30,33,37,40$, and the one-way ANOVA gives $MSW=24$ on $20$ degrees of freedom with a significant $F$. Use $q_{0.05,4,20}=3.958$, $t_{0.025,20}=2.086$, and — for six comparisons at familywise $0.05$ — the Bonferroni critical value $t_{0.05/12,20}=2.927$.

   1. (a)Compute Tukey's HSD and list every pair declared different at familywise $\alpha =0.05$. Which comparisons come closest to the threshold without crossing it, and what may be said about those pairs?
   2. (b)Compute the Bonferroni margin for a pairwise difference, and explain — without consulting any table — why it had to come out larger than the HSD.
   3. (c)Compute the margin an uncorrected pairwise $t$-test would use. Which additional pairs would it declare different, and what is the approximate familywise error rate of that procedure if all four diets were in fact identical?
   Show the solution to question 4

   (a) Tukey. With equal group sizes,

   $$\text{HSD}=q_{0.05,4,20}\sqrt{\frac{MSW}{n}}=3.958\sqrt{\frac{24}{6}}=3.958\times 2=\mathrm{7.916.}$$

   The six pairwise differences are $|30-33|=3$, $|30-37|=7$, $|30-40|=10$, $|33-37|=4$, $|33-40|=7$ and $|37-40|=3$. Only $10$ exceeds $7.916$: diets 1 and 4 differ. No other pair can be separated.

   The closest calls are $(1,3)$ and $(2,4)$, both at $7$, under the threshold by less than one unit. What may be said about them is only that the data do not separate them at familywise $5\%$ — not that those diets are equal. Failure to reject is not evidence of equality, and with $n=6$ the procedure has limited power.

   (b) Bonferroni. Six comparisons, each two-sided at $0.05/6$, so each tail gets $0.05/12$. The margin is

   $$t_{0.05/12,20}\sqrt{MSW\left(\frac{1}{n}+\frac{1}{n}\right)}=2.927\sqrt{24\cdot \frac{2}{6}}=2.927\sqrt{8}=2.927\times 2\sqrt{2}\approx 2.927\times 2.828\approx \mathrm{8.28.}$$

   That exceeds $7.916$, and it had to. Bonferroni bounds the familywise rate by the union bound, which adds up the six comparisonwise rates as if the comparisons were unrelated; but they share the same four group means and the same $MSW$, so the bound is slack. Tukey's HSD instead uses the exact distribution of the most extreme comparison — the studentised range — and achieves $\alpha$ rather than staying under it. A threshold that only bounds the rate must be at least as strict as one that attains it. Here the two procedures reach the same conclusions ($10>8.28$, nothing else does), but Bonferroni has less room to spare.

   (c) Uncorrected. The margin for a single two-sided $t$-test at $5\%$ is

   $$t_{0.025,20}\sqrt{8}=2.086\times 2\sqrt{2}\approx 2.086\times 2.828\approx \mathrm{5.90.}$$

   Now $7$, $10$ and $7$ all exceed it: $(1,3)$ and $(2,4)$ would be declared different in addition to $(1,4)$ — two extra claims, bought by abandoning the familywise guarantee.

   Treating the six comparisons as independent, the probability of at least one false rejection when all four diets are identical is about

   $$1-(0.95{)}^6=1-0.7351\approx 0.265,$$

   using $0.95^2=0.9025$, $0.95^3=0.8574$ and $0.95^6=0.8574^2\approx 0.7351$. Roughly one experiment in four would report a spurious difference somewhere, which is the error rate the $F$-test was built to avoid.

5. 5.Critique · 9 min

   Reaction times are compared across three conditions. Condition 1 has $36$ observations, obtained by testing $12$ volunteers three times each; conditions 2 and 3 have $6$ observations each, from six further volunteers apiece. The sample standard deviations are $3$, $12$ and $12$. The analyst's report reads:

   Step 1. Levene's test of equal variances gives $p=0.09$, so the variances do not differ significantly and the equal-variance assumption holds.

   Step 2. In any case the $F$-test is robust to unequal variances, so pooling the three groups into $MSW$ is safe.

   Step 3. A histogram of all $48$ reaction times is strongly right-skewed. Normality therefore fails, and since normality is the most important of the three assumptions, the $F$-test cannot be trusted until the data are transformed.

   Step 4. Independence was verified by plotting the residuals $x_{ij}-{\bar{x}}_i$ against condition and seeing no pattern.

   Every one of the four steps contains an error.

   1. (a)For each step, state precisely what is wrong and name the principle it violates.
   2. (b)Compute $MSW$ for this design. For a comparison between conditions 2 and 3, compare the pooled standard error $\sqrt{MSW(1/6+1/6)}$ with the standard error built from those two groups' own variance, and say what the discrepancy does to any $t$ or $F$ involving them.
   3. (c)Write the two or three sentences the report should have contained instead.
   Show the solution to question 5

   (a) The four errors.

   Step 1 mistakes non-rejection for confirmation, and uses a gatekeeper. A $p$-value of $0.09$ does not establish equal variances; with only $6$ observations in two of the three groups, Levene's test has little power, and the observed variance ratio $144/9=16$ is four times the informal factor-of-four limit. Worse, deciding the main analysis on the outcome of a preliminary variance test makes the whole thing a two-stage procedure whose error rate is not the nominal one. Even a convincing Levene verdict would not license pooling here.

   Step 2 drops the condition on robustness. The $F$-test is robust to unequal variances when the group sizes are equal. These are $36,6,6$, and the pairing is the dangerous one — the small groups carry the large variances — which inflates $F$ and makes the test reject far too often.

   Step 3 checks the wrong quantity and ranks the assumptions backwards. The assumption is that the errors are normal, which is a statement about the residuals $x_{ij}-{\bar{x}}_i$, not about the pooled raw data; pooled data from groups with different means is expected to look irregular. And normality is the least important of the three assumptions, because the $F$ statistic is built from group means that the central limit theorem is already normalising — the most important is independence. (A log transformation may well be sensible for reaction times, but not for the reason given, and not as the first priority.)

   Step 4 treats independence as something the data can certify. Independence is a property of how the data were collected and cannot be checked from the numbers; a residual plot with no pattern is fully consistent with strong dependence. And here the design tells us it fails: condition 1's $36$ observations are $12$ volunteers measured three times each. Repeated measurements on one person are correlated, so those $36$ numbers carry something nearer $12$ independent pieces of information, and every degree-of-freedom count in the analysis is wrong. This needs a different model — one with volunteer as a random effect — not a correction.

   (b) How far off the pooled denominator is. With $N-k=48-3=45$,

   $$MSW=\frac{35\cdot 9+5\cdot 144+5\cdot 144}{45}=\frac{315+720+720}{45}=\frac{1755}{45}=39.$$

   For conditions 2 versus 3 the pooled standard error is $\sqrt{39(1/6+1/6)}=\sqrt{13}\approx 3.6$. Using those two groups' own variance, $144$, it is $\sqrt{144(1/6+1/6)}=\sqrt{48}=4\sqrt{3}\approx 6.9$. The pooled figure is too small by a factor of $\sqrt{48/13}\approx 1.9$, so any $t$ statistic between conditions 2 and 3 is inflated by about $1.9$ and any $F$, being a squared quantity, by about $48/13\approx 3.7$. A difference that is genuinely two standard errors out is reported as nearly four.

   Note this calculation still takes condition 1's $35$ within-group degrees of freedom at face value, which Step 4 shows is itself unjustified.

   (c) What the report should say. “The group sizes are very unequal and the variance ratio is $16$, with the small groups the more variable, so the pooled $F$-test is not valid; Welch's ANOVA is used instead. Condition 1 consists of three repeated measurements on each of $12$ volunteers, so the analysis models volunteer as a random effect rather than treating the $36$ values as independent. Normality was assessed on the residuals, not the raw times; it is the least critical assumption, and a log scale was considered on substantive grounds.”

6. 6.Proof · 9 min

   Work in the one-way model, under which the group means ${\bar{X}}_i$ are independent with ${\bar{X}}_i\sim N({\mu }_i,{\sigma }^2/n_i)$. A contrast is $L={\sum }_i{c}_i{\mu }_i$ with ${\sum }_i{c}_i=0$, estimated by $\hat{L}={\sum }_i{c}_i{\bar{X}}_i$.

   1. (a)Prove that $E(\hat{L})=L$ and $\mathrm{Var}(\hat{L})={\sigma }^2{\sum }_i{c}_i^2/n_i$, stating exactly where independence is used. Deduce the standard error $\sqrt{MSW{\sum }_i{c}_i^2/n_i}$, and check that for the pairwise contrast $(1,-1,0,\dots ,0)$ it reduces to the pooled two-sample standard error.
   2. (b)Prove that ${\sum }_i{c}_i{\mu }_i$ is unchanged when the same constant $\delta$ is added to every ${\mu }_i$ if and only if ${\sum }_i{c}_i=0$, and explain in one sentence why this property is what makes a contrast a comparison rather than a level.
   3. (c)Three fertilisers give plot-yield means $8$, $11$ and $20$ with $n=4$ each and $MSW=6$ on $9$ degrees of freedom. Test whether fertiliser 3 differs from the average of the other two: first as a planned contrast using $t_{0.025,9}=2.262$, then as an unplanned one using Scheffé's critical value with $F_{0.05,2,9}=4.256$. Say when each critical value is the legitimate one.
   Show the solution to question 6

   (a) Mean and variance of $\hat{L}$. Expectation is linear, so $E(\hat{L})={\sum }_i{c}_{i}E({\bar{X}}_i)={\sum }_i{c}_i{\mu }_i=L$; no independence is needed for this.

   For the variance, expand the linear combination:

   $$\mathrm{Var}\left(\sum _i{c}_i{\bar{X}}_i\right)=\sum _i{c}_i^2\mathrm{Var}({\bar{X}}_i)+2\sum _{i<i^{\prime }}{c}_i{c}_{i^{\prime }}\mathrm{Cov}({\bar{X}}_i,{\bar{X}}_{i^{\prime }}).$$

   The covariances vanish because different groups consist of disjoint sets of independent observations, so the group means are independent — this is the one place independence enters. What remains is ${\sum }_i{c}_i^2{\sigma }^2/n_i$. $■$

   By the theorem on expected mean squares, $MSW$ is unbiased for ${\sigma }^2$ whether or not $H_0$ holds, so replacing ${\sigma }^2$ by $MSW$ gives the standard error $\sqrt{MSW{\sum }_i{c}_i^2/n_i}$. For $c=(1,-1,0,\dots ,0)$ the sum is $1/n_1+1/n_2$ and the standard error is $\sqrt{MSW(1/n_1+1/n_2)}=S_p\sqrt{1/n_1+1/n_2}$, since $MSW$ is precisely the pooled variance — the two-sample formula recovered. ✓

   (b) Invariance under a common shift. Adding $\delta$ to every mean,

   $$\sum _i{c}_i({\mu }_i+\delta )=\sum _i{c}_i{\mu }_i+\delta \sum _i{c}_i.$$

   If ${\sum }_i{c}_i=0$ the second term vanishes for every $\delta$. Conversely, if the value is unchanged for some $\delta \ne 0$, then $\delta {\sum }_i{c}_i=0$ forces ${\sum }_i{c}_i=0$. $■$

   A comparison is a statement about differences between means, and differences are exactly the quantities that survive a change of origin; a combination with $\sum c_i\ne 0$ changes when every group is shifted equally, so it measures a level, which depends on where zero is.

   (c) The fertiliser contrast. “Fertiliser 3 versus the average of 1 and 2” is $c=(-\frac{1}{2},-\frac{1}{2},1)$, which sums to zero ✓. The estimate is

   $$\hat{L}=-\frac{1}{2}(8)-\frac{1}{2}(11)+20=-\frac{19}{2}+20=\frac{21}{2}.$$

   For the standard error, ${\sum }_i{c}_i^2/n_i=(\frac{1}{4}+\frac{1}{4}+1)/4=\frac{3}{2}\cdot \frac{1}{4}=\frac{3}{8}$, so

   $$SE=\sqrt{6\cdot \frac{3}{8}}=\sqrt{\frac{9}{4}}=\frac{3}{2},t=\frac{21/2}{3/2}=7.$$

   Planned: against $t_{0.025,9}=2.262$, $7$ is far into the rejection region. Unplanned: Scheffé's critical value is $\sqrt{(k-1)F_{0.05,2,9}}=\sqrt{2\times 4.256}=\sqrt{8.512}\approx 2.92$ (since $2.92^2=8.53$), and $7>2.92$ as well.

   The $t$ critical value is legitimate only if the contrast was fixed before the data were seen. If it was chosen after noticing that fertiliser 3 stood out, it is one of infinitely many contrasts that could have been picked, and only Scheffé's value — which bounds the maximum over all of them simultaneously — controls the error rate. Here the conclusion holds under either standard, so it does not matter which story is true; it often does.

7. 7.Computation · 10 min

   A two-way design has factor $A$ (two coatings) and factor $B$ (three curing temperatures), with $n=5$ specimens per cell. The cell means, listed at $B_1,B_2,B_3$, are

   $A_1$: $6,9,12$. $A_2$: $10,11,12$.

   The within-cell sum of squares is $SSE=96$. Use $F_{0.05,1,24}=4.260$ and $F_{0.05,2,24}=3.403$.

   1. (a)Compute the row means ${\bar{x}}_{i\cdot }$, the column means ${\bar{x}}_{\cdot j}$ and the grand mean, and hence $SSA$ and $SSB$.
   2. (b)For each cell compute the departure from additivity ${\bar{x}}_{ij}-{\bar{x}}_{i\cdot }-{\bar{x}}_{\cdot j}+\bar{x}$. Check that the departures sum to zero along every row and every column, and compute $SSAB$.
   3. (c)Build the full ANOVA table, check that the degrees of freedom sum to $abn-1$, test the three effects at $\alpha =0.05$, and decide whether the main effects may be interpreted. If they may, state each in words.
   Show the solution to question 7

   (a) Marginal means and the main-effect sums of squares. Row means: ${\bar{x}}_{1\cdot }=(6+9+12)/3=9$ and ${\bar{x}}_{2\cdot }=(10+11+12)/3=11$. Column means: ${\bar{x}}_{\cdot 1}=(6+10)/2=8$, ${\bar{x}}_{\cdot 2}=10$, ${\bar{x}}_{\cdot 3}=12$. Grand mean $\bar{x}=(9+11)/2=10$, which also equals $(8+10+12)/3$ ✓.

   Each row mean averages $bn=15$ observations and each column mean $an=10$, so

   $$SSA=bn\sum _i({\bar{x}}_{i\cdot }-\bar{x}{)}^2=15[(-1{)}^2+1^2]=30,SSB=an\sum _j({\bar{x}}_{\cdot j}-\bar{x}{)}^2=10[(-2{)}^2+0+2^2]=80.$$

   (b) Departures from additivity. Additivity predicts cell $(i,j)$ to be ${\bar{x}}_{i\cdot }+{\bar{x}}_{\cdot j}-\bar{x}$. Row 1 predictions: $9+8-10=7$, $9+10-10=9$, $9+12-10=11$; the actual means are $6,9,12$, so the departures are $-1,0,+1$. Row 2 predictions: $11+8-10=9$, $11$, $13$; actual $10,11,12$, departures $+1,0,-1$.

   Row sums: $-1+0+1=0$ and $1+0-1=0$ ✓. Column sums: $-1+1$, $0+0$, $1-1$, all $0$ ✓. This is forced by the sum-to-zero constraints on the interaction effects, and it is why the interaction has only $(a-1)(b-1)=2$ degrees of freedom — two of the six departures determine the rest.

   Each departure is shared by $n=5$ observations, so

   $$SSAB=n\sum _{i,j}(\text{departure}{)}^2=5[1+0+1+1+0+1]=20.$$

   (c) The table and the tests. Degrees of freedom: $A$ has $a-1=1$; $B$ has $b-1=2$; $AB$ has $(a-1)(b-1)=2$; error has $ab(n-1)=6\times 4=24$. Sum $1+2+2+24=29=abn-1=30-1$ ✓.

   Mean squares: $MSA=30$, $MSB=40$, $MSAB=10$, $MSE=96/24=4$. Every effect is tested over $MSE$:

   $$F_A=\frac{30}{4}=7.5,F_B=\frac{40}{4}=10,F_{AB}=\frac{10}{4}=\mathrm{2.5.}$$

   Against the critical values: $F_A=7.5>4.260$, significant; $F_B=10>3.403$, significant; $F_{AB}=2.5<3.403$, not significant. By the two-way decomposition the total is $SST=30+80+20+96=226$ on $29$ degrees of freedom.

   The interaction is not significant, so the main effects may be interpreted as averages over the other factor. Coating $A_2$ produces a response $2$ units higher than $A_1$ on average ($11$ against $9$). Each step up in curing temperature raises the response by $2$ units on average ($8,10,12$).

   One caveat belongs in the report. A non-significant interaction is not a proof of additivity: in the sample, temperature raised $A_1$ by $6$ across the range but $A_2$ by only $2$, and with $MSE=4$ that difference is simply not distinguishable from noise at $F=2.5$. The main effects are the right summary, but “no interaction” should be read as “none detected”.

8. 8.Sketch · 9 min

   Three $2\times 2$ designs have factor $A$ at levels $A_1,A_2$ and factor $B$ at levels $B_1,B_2$, with $n=3$ observations per cell. The cell means, listed at $B_1,B_2$, are

   (i) $A_1$: $4,8$; $A_2$: $6,10$. (ii) $A_1$: $4,8$; $A_2$: $8,4$. (iii) $A_1$: $4,4$; $A_2$: $4,10$.

   1. (a)Draw the interaction plot for each design — level of $B$ on the horizontal axis, one line per level of $A$ — and describe the geometry of the two lines.
   2. (b)Compute $SSA$, $SSB$ and $SSAB$ for each design, and confirm in each case that they add up to $n{\sum }_{i,j}({\bar{x}}_{ij}-\bar{x}{)}^2$, the total between-cell sum of squares.
   3. (c)For each design, say which sums of squares the picture predicted would vanish, and why. In design (ii), explain what the statement “$B$ has no effect” gets wrong and give the simple effects of $B$ instead.
   Show the solution to question 8

   (a) The three plots. Each plot has $B_1$ and $B_2$ as two positions on the horizontal axis, cell mean on the vertical axis, and two line segments: the $A_1$ segment joins $(B_1,{\bar{x}}_{11})$ to $(B_2,{\bar{x}}_{12})$ and the $A_2$ segment joins $(B_1,{\bar{x}}_{21})$ to $(B_2,{\bar{x}}_{22})$. Label the segments and use a common vertical scale from $0$ to $12$ so the three pictures can be compared.

   (i) $A_1$ runs from $4$ up to $8$; $A_2$ runs from $6$ up to $10$. The two segments are parallel, both rising by $4$, with $A_2$ a constant $2$ above $A_1$. Draw the vertical gap at both ends and mark it $2$ each time — that equal gap is additivity.

   (ii) $A_1$ runs from $4$ up to $8$; $A_2$ runs from $8$ down to $4$. The segments form an X, crossing at the midpoint, height $6$. One slope is $+4$ and the other $-4$; mark both.

   (iii) $A_1$ is flat at $4$; $A_2$ rises from $4$ to $10$. The segments start together at $B_1$ and diverge, the gap growing from $0$ to $6$. Mark the shared starting point.

   (b) The sums of squares. Throughout, $a=b=2$, $n=3$, so each row and column mean averages $6$ observations and $SSA=6{\sum }_i({\bar{x}}_{i\cdot }-\bar{x}{)}^2$, $SSB=6{\sum }_j({\bar{x}}_{\cdot j}-\bar{x}{)}^2$, and $SSAB=3{\sum }_{i,j}(\text{departure}{)}^2$ with departure ${\bar{x}}_{ij}-{\bar{x}}_{i\cdot }-{\bar{x}}_{\cdot j}+\bar{x}$.

   (i) Row means $6,8$; column means $5,9$; grand mean $7$. $SSA=6[1+1]=12$; $SSB=6[4+4]=48$. Predicted cells: $6+5-7=4$, $6+9-7=8$, $8+5-7=6$, $8+9-7=10$ — every cell is exactly predicted, so $SSAB=0$. Between-cell total: $3[(4-7{)}^2+(8-7{)}^2+(6-7{)}^2+(10-7{)}^2]=3[9+1+1+9]=60=12+48+0$ ✓.

   (ii) Row means $6,6$; column means $6,6$; grand mean $6$. $SSA=0$ and $SSB=0$. Every cell is predicted to be $6$, so the departures are $-2,+2,+2,-2$ and $SSAB=3[4+4+4+4]=48$. Between-cell total: $3[4+4+4+4]=48=0+0+48$ ✓.

   (iii) Row means $4,7$; column means $4,7$; grand mean $\frac{11}{2}$. $SSA=6[\frac{9}{4}+\frac{9}{4}]=27$ and likewise $SSB=27$. Predicted cells: $4+4-\frac{11}{2}=\frac{5}{2}$, $4+7-\frac{11}{2}=\frac{11}{2}$, $7+4-\frac{11}{2}=\frac{11}{2}$, $7+7-\frac{11}{2}=\frac{17}{2}$; actual $4,4,4,10$; departures $+\frac{3}{2},-\frac{3}{2},-\frac{3}{2},+\frac{3}{2}$. So $SSAB=3\cdot 4\cdot \frac{9}{4}=27$. Between-cell total: $3[\frac{9}{4}+\frac{9}{4}+\frac{9}{4}+\frac{81}{4}]=3\cdot 27=81=27+27+27$ ✓.

   (c) Reading the pictures. (i) Parallel lines are the picture of additivity, so $SSAB$ had to vanish: the interaction is exactly the departure from parallel, and there is none. Both main effects are visible — the lines rise (a $B$ effect) and are separated (an $A$ effect).

   (ii) The X is symmetric: the $A_1$ and $A_2$ segments have the same average height ($6$), so the row means agree and $SSA=0$; and the heights at $B_1$ and $B_2$ average to the same $6$, so the column means agree and $SSB=0$. Everything is interaction. Nothing vanishes in (iii): the lines are not parallel, their average heights differ, and the average heights at the two $B$ levels differ.

   In (ii), “$B$ has no effect” reads $SSB=0$ as if $B$ were inert. But $SSB$ measures the effect of $B$ averaged over the levels of $A$, and here it is being averaged over a sign change. The simple effects are: within $A_1$, moving from $B_1$ to $B_2$ raises the response by $8-4=+4$; within $A_2$, it lowers it by $4-8=-4$. Factor $B$ moves the response by $4$ units at every level of $A$ — in opposite directions. When the interaction is present the correct report is these simple effects or the cell means themselves, never the main effect.

9. 9.Computation · 9 min

   A testing authority draws $k=12$ laboratories at random from the national register and sends each of them $n=6$ portions of the same reference material. The one-way ANOVA of the measured concentrations gives $MSB=75$ and $MSW=15$. Use $F_{0.05,11,60}=1.952$.

   1. (a)Explain why this is a random-effects situation, write down the model, and state the null hypothesis in terms of its parameters.
   2. (b)Estimate the between-laboratory variance ${\sigma }_A^2$, the within-laboratory variance ${\sigma }^2$, the variance of a single measurement, and the intraclass correlation. Interpret the last two in a sentence each.
   3. (c)Compute $F$ and test at $\alpha =0.05$. Say what a rejection means here, and how that differs from what the same numbers would mean if these twelve laboratories were the only ones of interest.
   4. (d)A repeat exercise with twelve new laboratories gives $MSB=12$ and $MSW=15$. Estimate ${\sigma }_A^2$, explain the result, and say what it does and does not indicate.
   Show the solution to question 9

   (a) Why random. The twelve laboratories are of no interest in themselves; they were drawn from a register so that the conclusion can be about the population of laboratories, and the question is how much laboratories vary. That is the random-effects model,

   $$X_{ij}=\mu +A_i+{\epsilon }_{ij},A_i\stackrel{\text{iid}}{\sim }N(0,{\sigma }_A^2),{\epsilon }_{ij}\stackrel{\text{iid}}{\sim }N(0,{\sigma }^2),$$

   with the $A_i$ independent of the ${\epsilon }_{ij}$. The null hypothesis is $H_0:{\sigma }_A^2=0$ — laboratories do not vary at all — rather than a statement about twelve particular means.

   (b) The variance components. By the expected mean squares under random effects, $E(MSW)={\sigma }^2$ and $E(MSB)={\sigma }^2+n{\sigma }_A^2$. Equating mean squares to their expectations,

   $${\hat{\sigma }}^2=MSW=15,{\hat{\sigma }}_A^2=\frac{MSB-MSW}{n}=\frac{75-15}{6}=10.$$

   A single measurement has variance $\mathrm{Var}(X_{ij})=\mathrm{Var}(A_i)+\mathrm{Var}({\epsilon }_{ij})={\sigma }_A^2+{\sigma }^2$ by independence, estimated by $10+15=25$: a measurement sent to a randomly chosen laboratory has standard deviation about $5$ concentration units, of which laboratory choice and within-laboratory noise are both substantial parts. The intraclass correlation is

   $$\frac{{\hat{\sigma }}_A^2}{{\hat{\sigma }}_A^2+{\hat{\sigma }}^2}=\frac{10}{25}=0.4,$$

   so $40\%$ of the variance of a single measurement is between laboratories, and two portions measured by the same laboratory have correlation $0.4$.

   (c) The test and what it means. $F=MSB/MSW=75/15=5$ on $k-1=11$ and $k(n-1)=60$ degrees of freedom. Since $5>1.952$, reject $H_0$.

   The rejection says ${\sigma }_A^2>0$: laboratories in the population genuinely differ in what they report for the same material. It is a statement about all laboratories on the register, and the natural follow-up is the estimate ${\hat{\sigma }}_A^2=10$ and the intraclass correlation. Had the twelve been fixed — the only twelve the authority uses — the same $F=5$ would say that these particular laboratories do not all have the same mean, and the follow-up would be a post-hoc procedure asking which ones differ. The arithmetic is identical; the estimand is not.

   (d) A negative estimate. Now ${\hat{\sigma }}_A^2=(12-15)/6=-\frac{1}{2}$. A variance cannot be negative, but the estimate is not a computational error: the method of moments returns a negative value whenever $MSB<MSW$, and under $H_0$ that happens roughly half the time, since both mean squares then estimate the same ${\sigma }^2$. The usual remedy is to truncate at zero and report ${\hat{\sigma }}_A^2=0$, with the caveat that truncation biases the estimator upward.

   Here $F=12/15=0.8<1.952$, so $H_0$ is not rejected. What this indicates: the data provide no evidence of between-laboratory variance. What it does not indicate: that laboratories are identical. A small $F$ is not evidence for the null beyond the non-rejection, and with $n=6$ per laboratory the estimate of ${\sigma }_A^2$ is imprecise.

10. 10.Proof · 10 min

    Work in the balanced random-effects model $X_{ij}=\mu +A_i+{\epsilon }_{ij}$ for $i=1,\dots ,k$ and $j=1,\dots ,n$, with $A_i\stackrel{\text{iid}}{\sim }N(0,{\sigma }_A^2)$, ${\epsilon }_{ij}\stackrel{\text{iid}}{\sim }N(0,{\sigma }^2)$, and every $A_i$ independent of every ${\epsilon }_{ij}$. You may quote $E(MSW)={\sigma }^2$ and $E(MSB)={\sigma }^2+n{\sigma }_A^2$.

    1. (a)Prove that two observations in the same group have covariance ${\sigma }_A^2$ and correlation $\rho ={\sigma }_A^2/({\sigma }_A^2+{\sigma }^2)$, and that observations in different groups are independent. Deduce that $\rho \ge 0$ always, and identify the case $\rho =0$.
    2. (b)Prove that the grand mean satisfies $\mathrm{Var}(\bar{X})=\frac{{\sigma }_A^2}{k}+\frac{{\sigma }^2}{kn}$. Find its limit as $n\to \infty$ with $k$ fixed, and explain what this says about choosing between more groups and more observations per group when the aim is to estimate $\mu$.
    3. (c)Prove that ${\hat{\sigma }}_A^2=(MSB-MSW)/n$ is unbiased for ${\sigma }_A^2$, and then that the truncated estimator $\max ({\hat{\sigma }}_A^2,0)$ has expectation strictly greater than ${\sigma }_A^2$ whenever $P(MSB<MSW)>0$. Why is that probability certainly positive under $H_0$?
    Show the solution to question 10

    (a) The covariance structure. Fix a group $i$ and two distinct observations $j\ne l$ in it. The constant $\mu$ contributes nothing to any covariance, so by bilinearity

    $$\mathrm{Cov}(X_{ij},X_{il})=\mathrm{Cov}(A_i+{\epsilon }_{ij},A_i+{\epsilon }_{il})=\mathrm{Var}(A_i)+\mathrm{Cov}(A_i,{\epsilon }_{il})+\mathrm{Cov}({\epsilon }_{ij},A_i)+\mathrm{Cov}({\epsilon }_{ij},{\epsilon }_{il}).$$

    The last three terms are zero: the $A$'s are independent of the $\epsilon$'s, and distinct $\epsilon$'s are independent. So the covariance is ${\sigma }_A^2$ — the shared group effect is the only thing two group-mates have in common. For the variance, the same expansion with $j=l$ gives $\mathrm{Var}(X_{ij})={\sigma }_A^2+{\sigma }^2$, and therefore

    $$\rho =\frac{\mathrm{Cov}(X_{ij},X_{il})}{\sqrt{\mathrm{Var}(X_{ij})\mathrm{Var}(X_{il})}}=\frac{{\sigma }_A^2}{{\sigma }_A^2+{\sigma }^2}.$$

    For observations in different groups $i\ne i^{\prime }$, every one of the four terms in the expansion of $\mathrm{Cov}(A_i+{\epsilon }_{ij},A_{i^{\prime }}+{\epsilon }_{i^{\prime }l})$ is zero: distinct $A$'s are independent, $A$'s and $\epsilon$'s are independent, and distinct $\epsilon$'s are independent. So the two observations are uncorrelated. They are moreover independent, because the whole vector $(X_{ij})$ is a linear function of the independent normal variables $A_1,\dots ,A_k,{\epsilon }_{11},\dots ,{\epsilon }_{kn}$ and is therefore jointly normal, and for jointly normal variables zero correlation is independence. $■$

    Since ${\sigma }_A^2\ge 0$ is a variance and the denominator is positive, $\rho \ge 0$: this model can never make group-mates negatively correlated. Equality $\rho =0$ holds exactly when ${\sigma }_A^2=0$, which is $H_0$ — under the null the group structure carries no information and the $kn$ observations are simply i.i.d. $N(\mu ,{\sigma }^2)$.

    (b) Variance of the grand mean. Averaging the model over all $kn$ observations,

    $$\bar{X}=\mu +\frac{1}{k}\sum _{i=1}^k{A}_i+\frac{1}{kn}\sum _{i,j}{\epsilon }_{ij}.$$

    The two random sums are independent (one is built from the $A$'s, the other from the $\epsilon$'s), so their variances add:

    $$\mathrm{Var}\left(\frac{1}{k}\sum _i{A}_i\right)=\frac{k{\sigma }_A^2}{k^2}=\frac{{\sigma }_A^2}{k},\mathrm{Var}\left(\frac{1}{kn}\sum _{i,j}{\epsilon }_{ij}\right)=\frac{kn{\sigma }^2}{(kn{)}^2}=\frac{{\sigma }^2}{kn},$$

    using independence within each sum. Hence $\mathrm{Var}(\bar{X})={\sigma }_A^2/k+{\sigma }^2/(kn)$. $■$ (Equivalently: $\bar{X}$ is the average of the $k$ i.i.d. group means, each with variance ${\sigma }_A^2+{\sigma }^2/n$ by the theorem on expected mean squares under random effects, and dividing by $k$ gives the same thing.)

    As $n\to \infty$ with $k$ fixed the second term vanishes and $\mathrm{Var}(\bar{X})\to {\sigma }_A^2/k$, which is not zero unless ${\sigma }_A^2=0$. More observations per group eliminate the measurement noise but leave the $k$ draws of the group effect exactly as they were; the grand mean can never estimate $\mu$ more precisely than an average of $k$ observations from $N(\mu ,{\sigma }_A^2)$ does. If the aim is $\mu$, precision comes from sampling more groups. With a fixed total of $kn$ observations, the design that spreads them over many groups beats the one that concentrates them in a few — for instance $k=100$, $n=2$ gives ${\sigma }_A^2/100+{\sigma }^2/200$, while $k=2$, $n=100$ gives ${\sigma }_A^2/2+{\sigma }^2/200$.

    (c) Unbiasedness, and the cost of truncation. By the quoted expectations and linearity,

    $$E({\hat{\sigma }}_A^2)=\frac{E(MSB)-E(MSW)}{n}=\frac{{\sigma }^2+n{\sigma }_A^2-{\sigma }^2}{n}={\sigma }_A^2.■$$

    Write $T={\hat{\sigma }}_A^2$ and $T^{+}=\max (T,0)$. Then $T^{+}-T=\max (-T,0)\ge 0$ everywhere, and $T^{+}-T=-T>0$ exactly on the event $\{T<0\}=\{MSB<MSW\}$. A non-negative random variable that is strictly positive on an event of positive probability has strictly positive expectation, so if $P(MSB<MSW)>0$ then

    $$E(T^{+})-E(T)=E[(T^{+}-T)]>0,\text{i.e.}E[\max ({\hat{\sigma }}_A^2,0)]>{\sigma }_A^2.■$$

    Under $H_0$ the ratio $MSB/MSW$ has the $F_{k-1,k(n-1)}$ distribution, which has positive density on all of $(0,\infty )$, so $P(MSB/MSW<1)$ is strictly positive — in fact close to one half, since both mean squares are estimating the same ${\sigma }^2$ and their ratio falls on either side of $1$ with comparable probability. The consequence is uncomfortable: when the true between-group variance is exactly zero, the truncated estimator has strictly positive expectation. This is the awkwardness of the moment estimator that pushes practice toward likelihood-based estimates of variance components.

11. 11.Critique · 10 min

    Three training programmes are compared, $n=10$ trainees each, with score means ${\bar{x}}_1=12$, ${\bar{x}}_2=10$, ${\bar{x}}_3=14$ and $MSW=10$ on $27$ degrees of freedom. An analyst fits the indicator model with programme 1 as reference and reports:

    Step 1. ${\hat{\beta }}_2=-2$ and ${\hat{\beta }}_3=2$, each with standard error $\sqrt{2}$, so $t=\mp 1.41$. Against $t_{0.025,27}=2.052$ neither is significant.

    Step 2. The ANOVA $F$-test is just these two tests combined, so it cannot be significant either. The three programmes are equivalent.

    Step 3. Refitting with programme 2 as reference gives ${\hat{\beta }}_3=4$ with $t=2.83$, which is significant. Since the two fits contradict each other, the coding is unreliable, and the first fit is the one reported.

    Use $F_{0.05,2,27}=3.354$ and $q_{0.05,3,27}=3.51$.

    1. (a)Verify the standard error and the $t$ values in Step 1. Then identify the error in each of Steps 2 and 3, naming the principle each violates.
    2. (b)Carry out the model-comparison $F$-test from the numbers given and state the correct conclusion.
    3. (c)Explain why $F$ rejects while neither coefficient in Step 1 is significant: which comparison does $F$ see that the reference coding hides? Then decide, using Tukey's HSD, whether that comparison survives a familywise correction.
    Show the solution to question 11

    (a) Step 1 is correct; Steps 2 and 3 are not. Under indicator coding ${\hat{\beta }}_i={\bar{x}}_i-{\bar{x}}_1$, a difference of two independent group means, so its variance is ${\sigma }^2(1/n_1+1/n_i)$ and, estimating ${\sigma }^2$ by $MSW$,

    $$SE({\hat{\beta }}_i)=\sqrt{10\left(\frac{1}{10}+\frac{1}{10}\right)}=\sqrt{2}.t=\frac{\mp 2}{\sqrt{2}}=\mp \sqrt{2}\approx \mp \mathrm{1.41.}✓$$

    Step 2 misdescribes the $F$-test and then misreads a non-rejection. The $F$-test is a model comparison: it tests the single joint hypothesis ${\beta }_2={\beta }_3=0$ by comparing $SSE$ under the full model with $SSE$ under the intercept-only model. It is not a combination of the two coefficient tests, which are two uncorrected comparisons against the reference — and neither of them tests whether programmes 2 and 3 differ from each other. A joint hypothesis can be rejected while every marginal one survives. Separately, “not significant” is not “equivalent”: failing to reject is not evidence of equality.

    Step 3 mistakes a reparameterisation for a contradiction. The coefficients are comparisons against whichever reference was chosen, so of course they change with the reference; what does not change is the set of fitted values, hence $SSE$, hence $F$. The two fits agree perfectly: under reference 2, ${\hat{\beta }}_3={\bar{x}}_3-{\bar{x}}_2=14-10=4$, which is exactly ${\hat{\beta }}_3-{\hat{\beta }}_2=2-(-2)=4$ from the first fit. Choosing which fit to report according to which looks significant is a selection, and it is exactly the uncorrected reference-level comparison the $F$-test exists to replace.

    (b) The $F$-test. The grand mean is $(12+10+14)/3=12$, so

    $$SSB=10[(12-12{)}^2+(10-12{)}^2+(14-12{)}^2]=10[0+4+4]=80,SSW=27\times 10=270.$$

    By the decomposition of the total sum of squares, $SST=350$. The full model fits each group its own mean, so $SS{E}_{\text{full}}=SSW=270$, while the intercept-only model has $SSE=SST=350$. The model-comparison statistic is

    $$F=\frac{(350-270)/2}{270/27}=\frac{40}{10}=4.0>\mathrm{3.354.}$$

    Reject $H_0$ at $5\%$: the three programmes do not all have the same mean score. Step 2's conclusion is the opposite of the truth.

    (c) What $F$ sees. Programme 1 happens to sit in the middle of the three means. Each coefficient in the reference-1 coding measures a gap of $2$, which is $\sqrt{2}\approx 1.41$ standard errors; but the gap between programmes 2 and 3 is $4$, which is $2\sqrt{2}\approx 2.83$ standard errors, and no coefficient in that coding measures it. The $F$-test does not compare anything to a reference: $SSB$ counts programme 2 and programme 3 as each $2$ away from the grand mean on opposite sides, so the $4$-unit spread between them is fully present in the numerator. That is why $F$ rejects while both coefficients fail — the reference coding was looking at the two smallest of the three differences.

    Whether the 2-versus-3 comparison survives a familywise correction is a Tukey question. With equal group sizes,

    $$\text{HSD}=q_{0.05,3,27}\sqrt{\frac{MSW}{n}}=3.51\sqrt{\frac{10}{10}}=\mathrm{3.51.}$$

    The differences are $|12-10|=2$, $|12-14|=2$ and $|10-14|=4$. Only $4>3.51$: programmes 2 and 3 differ at familywise $5\%$, and neither can be separated from programme 1. So the $t=2.83$ in Step 3 was uncorrected, but the finding it pointed at is genuine. The correct report is: the programmes differ ($F=4.0$, $p<0.05$), and the difference the data can locate is between programmes 2 and 3.

12. 12.Proof · 12 min

    Consider $k$ groups with $n_i$ observations $x_{ij}$, group means ${\bar{x}}_i$, grand mean $\bar{x}$ and $N={\sum }_i{n}_i$. The indicator model with group $1$ as reference is $X={\beta }_0+{\beta }_2{D}_2+\cdots +{\beta }_k{D}_k+\epsilon$. Prove the following from scratch. The only results you may quote are the decomposition $SST=SSB+SSW$ and, in part (d), the proposition that with two groups $F=t^2$.

    1. (a)Show that for any number $m$ and any group $i$, ${\sum }_j(x_{ij}-m{)}^2={\sum }_j(x_{ij}-{\bar{x}}_i{)}^2+n_i({\bar{x}}_i-m{)}^2$. Deduce that the least-squares fit of the indicator model gives every observation in group $i$ the fitted value ${\bar{x}}_i$, and conclude $SS{E}_{\text{full}}=SSW$.
    2. (b)Read off the least-squares coefficients ${\hat{\beta }}_0,{\hat{\beta }}_2,\dots ,{\hat{\beta }}_k$. Then show that if group $r$ is used as the reference instead, the coefficients change but the fitted values, $SS{E}_{\text{full}}$ and the $F$ statistic do not.
    3. (c)Show that the intercept-only model has residual sum of squares $SST$, and hence that the model-comparison statistic $\frac{(SST-SS{E}_{\text{full}})/(k-1)}{SS{E}_{\text{full}}/(N-k)}$ equals $MSB/MSW$.
    4. (d)For $k=2$, show that the $t$ statistic for $H_0:{\beta }_2=0$ is the pooled two-sample $t$ statistic, so that the coefficient test and the $F$-test coincide. Explain in one sentence why no such coincidence holds for $k\ge 3$.
    Show the solution to question 12

    (a) The one-group identity, and the fit. Insert and remove the group mean: $x_{ij}-m=(x_{ij}-{\bar{x}}_i)+({\bar{x}}_i-m)$. Squaring and summing over $j$,

    $$\sum _j(x_{ij}-m{)}^2=\sum _j(x_{ij}-{\bar{x}}_i{)}^2+2({\bar{x}}_i-m)\sum _j(x_{ij}-{\bar{x}}_i)+n_i({\bar{x}}_i-m{)}^2.$$

    The middle sum is ${\sum }_j{x}_{ij}-n_i{\bar{x}}_i=0$ by the definition of ${\bar{x}}_i$, which proves the identity. $■$ Since the second term on the right is non-negative and zero only at $m={\bar{x}}_i$, the identity also says: over all $m$, the sum ${\sum }_j(x_{ij}-m{)}^2$ is uniquely minimised at $m={\bar{x}}_i$.

    Now the model. An observation in group $1$ has every indicator zero, so its fitted value is ${\beta }_0$; an observation in group $i\ge 2$ has $D_i=1$ and the others zero, so its fitted value is ${\beta }_0+{\beta }_i$. Write $m_1={\beta }_0$ and $m_i={\beta }_0+{\beta }_i$ for the group levels. The map $({\beta }_0,{\beta }_2,\dots ,{\beta }_k)\mapsto (m_1,\dots ,m_k)$ is a bijection of ${\mathbb{R}}^k$ — its inverse is ${\beta }_0=m_1$, ${\beta }_i=m_i-m_1$ — so minimising the residual sum of squares over the $\beta$'s is the same as minimising

    $$\sum _i\sum _j(x_{ij}-m_i{)}^2$$

    over all $(m_1,\dots ,m_k)$ freely. The sum splits into $k$ pieces, and piece $i$ involves only $m_i$. By the identity, each piece is minimised uniquely at $m_i={\bar{x}}_i$, so the whole sum is minimised there. The fitted value of every observation in group $i$ is ${\bar{x}}_i$, and the minimum value is

    $$SS{E}_{\text{full}}=\sum _i\sum _j(x_{ij}-{\bar{x}}_i{)}^2=SSW.■$$

    (b) The coefficients, and changing the reference. From the inverse map, ${\hat{\beta }}_0={\hat{m}}_1={\bar{x}}_1$ and ${\hat{\beta }}_i={\hat{m}}_i-{\hat{m}}_1={\bar{x}}_i-{\bar{x}}_1$ for $i=2,\dots ,k$: the intercept is the reference group's mean, and each slope is a difference from it.

    With group $r$ as reference, the fitted value in group $r$ is ${\beta }_0$ and in group $i\ne r$ it is ${\beta }_0+{\beta }_i$; the correspondence ${\beta }_0=m_r$, ${\beta }_i=m_i-m_r$ is again a bijection onto the same set of level vectors $(m_1,\dots ,m_k)$. The minimisation over the levels is therefore the same problem with the same unique solution ${\hat{m}}_i={\bar{x}}_i$. So the fitted values are unchanged, the residuals are unchanged, $SS{E}_{\text{full}}=SSW$ is unchanged, and $F$ — which is built from $SST$, $SS{E}_{\text{full}}$ and the degrees of freedom, none of which mention the coding — is unchanged. What does change is the coefficients: ${\hat{\beta }}_0={\bar{x}}_r$ and ${\hat{\beta }}_i={\bar{x}}_i-{\bar{x}}_r$, a different set of comparisons describing the same fit. $■$

    (c) The reduced model. The intercept-only model gives every observation the fitted value $m$. Apply the identity of (a) with the entire sample treated as one group of size $N$ and mean $\bar{x}$:

    $$\sum _{i,j}(x_{ij}-m{)}^2=\sum _{i,j}(x_{ij}-\bar{x}{)}^2+N(\bar{x}-m{)}^2\ge SST,$$

    with equality at $m=\bar{x}$. So the least-squares intercept is the grand mean and $SS{E}_{\text{reduced}}=SST$. Then, by the quoted decomposition, $SST-SS{E}_{\text{full}}=SST-SSW=SSB$, and

    $$\frac{(SST-SS{E}_{\text{full}})/(k-1)}{SS{E}_{\text{full}}/(N-k)}=\frac{SSB/(k-1)}{SSW/(N-k)}=\frac{MSB}{MSW}.■$$

    The degrees of freedom are the right ones: the full model has $k$ parameters, leaving $N-k$ residual degrees of freedom, and the reduced model has $1$, so the comparison spends $k-1$.

    (d) Two groups. For $k=2$ the one coefficient is ${\hat{\beta }}_2={\bar{X}}_2-{\bar{X}}_1$. Under the model the two group means are independent with variances ${\sigma }^2/n_1$ and ${\sigma }^2/n_2$, so $\mathrm{Var}({\hat{\beta }}_2)={\sigma }^2(1/n_1+1/n_2)$. The regression estimate of ${\sigma }^2$ is the residual mean square $SS{E}_{\text{full}}/(N-2)=SSW/(N-2)$, which by (a) is the pooled variance $S_p^2$. Hence

    $$t=\frac{{\hat{\beta }}_2}{\widehat{SE}({\hat{\beta }}_2)}=\frac{{\bar{X}}_2-{\bar{X}}_1}{S_p\sqrt{1/n_1+1/n_2}},$$

    the pooled two-sample $t$ statistic, and by the proposition that with two groups $F$ is the square of $t$, the coefficient test and the model-comparison $F$-test reject on exactly the same data, with matching critical values $F_{\alpha ,1,\nu }=t_{\alpha /2,\nu }^2$. $■$

    For $k\ge 3$ there are $k-1$ coefficients, each testing one difference from the reference at an uncorrected level, whereas $F$ tests the single joint hypothesis that all $k-1$ vanish — and a joint hypothesis is neither the conjunction of separately tested marginal ones nor sensitive to which group was chosen as the reference, as the previous problem shows in numbers.