Analysis of Variance

(a) Group means and the grand mean. ${\bar{x}}_A=12/3=4$, ${\bar{x}}_B=35/5=7$, ${\bar{x}}_C=40/4=10$. The grand mean is the mean of all $N=3+5+4=12$ observations, which is the group means weighted by group size:

$$\bar{x}=\frac{12+35+40}{12}=\frac{87}{12}=\frac{29}{4}.$$

The unweighted average $(4+7+10)/3=7$ gives Brand A's three observations the same weight as Brand B's five. The grand mean has to reproduce the total, and $7\times 12=84\ne 87$.

Between. The group means sit at $4-\frac{29}{4}=-\frac{13}{4}$, $7-\frac{29}{4}=-\frac{1}{4}$ and $10-\frac{29}{4}=\frac{11}{4}$ from the grand mean, so

$$SSB=\sum _i{n}_i({\bar{x}}_i-\bar{x}{)}^2=3\cdot \frac{169}{16}+5\cdot \frac{1}{16}+4\cdot \frac{121}{16}=\frac{507+5+484}{16}=\frac{996}{16}=\frac{249}{4}.$$

Within. Deviations from each group's own mean. Brand A: $-2,0,2$, squares summing to $8$. Brand B: $-2,0,0,2,0$, sum $8$. Brand C: $-2,0,2,0$, sum $8$. So $SSW=24$.

(b) The table. Degrees of freedom are $k-1=2$ between and $N-k=12-3=9$ within. Then $MSB=\frac{249}{4}\cdot \frac{1}{2}=\frac{249}{8}$ and $MSW=\frac{24}{9}=\frac{8}{3}$, and

$$F=\frac{MSB}{MSW}=\frac{249}{8}\cdot \frac{3}{8}=\frac{747}{64}\approx \mathrm{11.67.}$$

In table form: Between — $SS=249/4$, df $2$, $MS=249/8$; Within — $SS=24$, df $9$, $MS=8/3$; Total — $SS=345/4$, df $11$; $F=747/64$.

Since $11.67>4.256$, reject $H_0$ at the $5\%$ level: the three brands do not all have the same mean lifetime. That is all it says. The alternative is “not all equal”, so the test does not identify which brands differ, and it does not say that all three differ from each other — a post-hoc procedure is needed for that.

(c) The identity as an arithmetic check. First, ${\sum }_{i,j}{x}_{ij}^2=(4+16+36)+(25+49+49+81+49)+(64+100+144+100)=56+253+408=717$. Expanding the square, $\sum (x_{ij}-\bar{x}{)}^2=\sum x_{ij}^2-2\bar{x}\sum x_{ij}+N{\bar{x}}^2=\sum x_{ij}^2-N{\bar{x}}^2$, so

$$SST=717-12\cdot \frac{841}{16}=717-\frac{2523}{4}=\frac{2868-2523}{4}=\frac{345}{4}.$$

And $SSB+SSW=\frac{249}{4}+\frac{96}{4}=\frac{345}{4}$ ✓, by the decomposition of the total sum of squares. The degrees of freedom agree too: $2+9=11=N-1$ ✓.

Effect size:

$${\eta }^2=\frac{SSB}{SST}=\frac{249/4}{345/4}=\frac{249}{345}=\frac{83}{115}\approx \mathrm{0.72.}$$

About $72\%$ of the variation in lifetime is attributable to brand. That is a statement about how much brand matters. The $F$-test answers only whether any difference exists, and with enough data it rejects for differences of no practical size; ${\eta }^2$ is what separates “detectable” from “large”.

(a) The grand mean. By definition $\bar{x}$ is the total divided by $N$. The total is a sum over groups of each group's total, and a group's total is ${\sum }_j{x}_{ij}=n_i{\bar{x}}_i$ by the definition of ${\bar{x}}_i$. So

$$\bar{x}=\frac{1}{N}\sum _i\sum _j{x}_{ij}=\frac{1}{N}\sum _i{n}_i{\bar{x}}_i.$$

If every $n_i=n$ then $N=kn$ and $\bar{x}=\frac{1}{kn}{\sum }_{i}n{\bar{x}}_i=\frac{1}{k}{\sum }_i{\bar{x}}_i$, the plain average. $■$

(b) Within mean square. By the definition of $s_i^2$, group $i$'s within sum of squares is ${\sum }_j(x_{ij}-{\bar{x}}_i{)}^2=(n_i-1)s_i^2$. Summing over groups,

$$SSW=\sum _i(n_i-1)s_i^2,\text{and}N-k=\sum _i{n}_i-\sum _{i}1=\sum _i(n_i-1).$$

Hence $MSW=SSW/(N-k)={\sum }_i(n_i-1)s_i^2/{\sum }_i(n_i-1)$, which is a weighted average of the $s_i^2$ because the weights $n_i-1$ are positive and the denominator is their sum. With all $n_i=n$ the weights are all $n-1$, so

$$MSW=\frac{(n-1){\sum }_i{s}_i^2}{k(n-1)}=\frac{1}{k}\sum _i{s}_i^2.■$$

(c) Equal variances. If $s_i^2=s^2$ for every $i$, then by (b)

$$MSW=\frac{s^2{\sum }_i(n_i-1)}{{\sum }_i(n_i-1)}=s^2,$$

independently of the $n_i$. $■$

When the sizes and the variances are both unequal, (b) says $MSW$ is pulled toward the variances of the large groups, since they carry most of the weight — so it understates the noise in the small groups if those are the more variable ones, and overstates it if they are the less variable ones.

(a) The three $MSW$ values. $MSW$ is the weighted average ${\sum }_i(n_i-1)s_i^2/(N-k)$, with weights $n_i-1$.

(i) Weights $9,9,9$ and variances $4,36,36$:

$$MSW=\frac{9\cdot 4+9\cdot 36+9\cdot 36}{27}=\frac{684}{27}=\frac{76}{3}\approx 25.3,$$

the plain average of $4$, $36$ and $36$. It sits between the group variances, closer to the $36$ shared by two of the three groups.

(ii) Weights $39,4,4$ and variances $4,36,36$:

$$MSW=\frac{39\cdot 4+4\cdot 36+4\cdot 36}{47}=\frac{156+144+144}{47}=\frac{444}{47}\approx \mathrm{9.4.}$$

The two small groups have variance $36$, nearly four times $MSW$.

(iii) Weights $39,4,4$ and variances $36,4,4$:

$$MSW=\frac{39\cdot 36+4\cdot 4+4\cdot 4}{47}=\frac{1404+16+16}{47}=\frac{1436}{47}\approx \mathrm{30.6.}$$

The two small groups have variance $4$; $MSW$ overstates their noise by a factor of about $7\frac{1}{2}$.

(b) Which way each design fails. In every design the variance ratio is $36/4=9$, beyond the factor-of-four rule of thumb, so the difference between the designs is entirely the group sizes.

(i) Approximately valid. With equal group sizes the weights are equal, $MSW$ is a fair average, and the $F$-test is robust to unequal variances. The true level stays close to $5\%$.

(ii) Anti-conservative. This is the dangerous pairing: small groups with the large variances. The $39$ degrees of freedom of the tight group dominate $MSW$, so any comparison that involves group 2 or 3 is divided by a denominator too small by almost a factor of four. $F$ comes out too large and the test rejects far more often than $5\%$ when $H_0$ is true.

(iii) Conservative. The reverse pairing. The large group is the variable one and pulls $MSW$ up, so differences involving the small, tight groups are divided by a denominator that is far too large. The level is below nominal and power is lost; rejections that do occur are trustworthy, but real differences go undetected.

(c) The recommendation. Use Welch's ANOVA for both (ii) and (iii), unconditionally. It estimates each group's variance separately and adjusts the degrees of freedom, exactly as Welch's $t$-test does for two groups, and it costs almost nothing when the variances turn out to be equal.

Testing for equal variances first and choosing the main test on the result is a two-stage procedure whose overall error rate nobody computes. It is also weak precisely where it is needed: with $n=5$ in two of the groups, Levene's test has little power even against a nine-fold variance ratio, so a non-rejection would say almost nothing. The group sizes are known before any data are collected, and unequal sizes are reason enough.

(a) Tukey. With equal group sizes,

$$\text{HSD}=q_{0.05,4,20}\sqrt{\frac{MSW}{n}}=3.958\sqrt{\frac{24}{6}}=3.958\times 2=\mathrm{7.916.}$$

The six pairwise differences are $|30-33|=3$, $|30-37|=7$, $|30-40|=10$, $|33-37|=4$, $|33-40|=7$ and $|37-40|=3$. Only $10$ exceeds $7.916$: diets 1 and 4 differ. No other pair can be separated.

The closest calls are $(1,3)$ and $(2,4)$, both at $7$, under the threshold by less than one unit. What may be said about them is only that the data do not separate them at familywise $5\%$ — not that those diets are equal. Failure to reject is not evidence of equality, and with $n=6$ the procedure has limited power.

(b) Bonferroni. Six comparisons, each two-sided at $0.05/6$, so each tail gets $0.05/12$. The margin is

$$t_{0.05/12,20}\sqrt{MSW\left(\frac{1}{n}+\frac{1}{n}\right)}=2.927\sqrt{24\cdot \frac{2}{6}}=2.927\sqrt{8}=2.927\times 2\sqrt{2}\approx 2.927\times 2.828\approx \mathrm{8.28.}$$

That exceeds $7.916$, and it had to. Bonferroni bounds the familywise rate by the union bound, which adds up the six comparisonwise rates as if the comparisons were unrelated; but they share the same four group means and the same $MSW$, so the bound is slack. Tukey's HSD instead uses the exact distribution of the most extreme comparison — the studentised range — and achieves $\alpha$ rather than staying under it. A threshold that only bounds the rate must be at least as strict as one that attains it. Here the two procedures reach the same conclusions ($10>8.28$, nothing else does), but Bonferroni has less room to spare.

(c) Uncorrected. The margin for a single two-sided $t$-test at $5\%$ is

$$t_{0.025,20}\sqrt{8}=2.086\times 2\sqrt{2}\approx 2.086\times 2.828\approx \mathrm{5.90.}$$

Now $7$, $10$ and $7$ all exceed it: $(1,3)$ and $(2,4)$ would be declared different in addition to $(1,4)$ — two extra claims, bought by abandoning the familywise guarantee.

Treating the six comparisons as independent, the probability of at least one false rejection when all four diets are identical is about

$$1-(0.95{)}^6=1-0.7351\approx 0.265,$$

using $0.95^2=0.9025$, $0.95^3=0.8574$ and $0.95^6=0.8574^2\approx 0.7351$. Roughly one experiment in four would report a spurious difference somewhere, which is the error rate the $F$-test was built to avoid.

(a) The four errors.

Step 1 mistakes non-rejection for confirmation, and uses a gatekeeper. A $p$-value of $0.09$ does not establish equal variances; with only $6$ observations in two of the three groups, Levene's test has little power, and the observed variance ratio $144/9=16$ is four times the informal factor-of-four limit. Worse, deciding the main analysis on the outcome of a preliminary variance test makes the whole thing a two-stage procedure whose error rate is not the nominal one. Even a convincing Levene verdict would not license pooling here.

Step 2 drops the condition on robustness. The $F$-test is robust to unequal variances when the group sizes are equal. These are $36,6,6$, and the pairing is the dangerous one — the small groups carry the large variances — which inflates $F$ and makes the test reject far too often.

Step 3 checks the wrong quantity and ranks the assumptions backwards. The assumption is that the errors are normal, which is a statement about the residuals $x_{ij}-{\bar{x}}_i$, not about the pooled raw data; pooled data from groups with different means is expected to look irregular. And normality is the least important of the three assumptions, because the $F$ statistic is built from group means that the central limit theorem is already normalising — the most important is independence. (A log transformation may well be sensible for reaction times, but not for the reason given, and not as the first priority.)

Step 4 treats independence as something the data can certify. Independence is a property of how the data were collected and cannot be checked from the numbers; a residual plot with no pattern is fully consistent with strong dependence. And here the design tells us it fails: condition 1's $36$ observations are $12$ volunteers measured three times each. Repeated measurements on one person are correlated, so those $36$ numbers carry something nearer $12$ independent pieces of information, and every degree-of-freedom count in the analysis is wrong. This needs a different model — one with volunteer as a random effect — not a correction.

(b) How far off the pooled denominator is. With $N-k=48-3=45$,

$$MSW=\frac{35\cdot 9+5\cdot 144+5\cdot 144}{45}=\frac{315+720+720}{45}=\frac{1755}{45}=39.$$

For conditions 2 versus 3 the pooled standard error is $\sqrt{39(1/6+1/6)}=\sqrt{13}\approx 3.6$. Using those two groups' own variance, $144$, it is $\sqrt{144(1/6+1/6)}=\sqrt{48}=4\sqrt{3}\approx 6.9$. The pooled figure is too small by a factor of $\sqrt{48/13}\approx 1.9$, so any $t$ statistic between conditions 2 and 3 is inflated by about $1.9$ and any $F$, being a squared quantity, by about $48/13\approx 3.7$. A difference that is genuinely two standard errors out is reported as nearly four.

Note this calculation still takes condition 1's $35$ within-group degrees of freedom at face value, which Step 4 shows is itself unjustified.

(c) What the report should say. “The group sizes are very unequal and the variance ratio is $16$, with the small groups the more variable, so the pooled $F$-test is not valid; Welch's ANOVA is used instead. Condition 1 consists of three repeated measurements on each of $12$ volunteers, so the analysis models volunteer as a random effect rather than treating the $36$ values as independent. Normality was assessed on the residuals, not the raw times; it is the least critical assumption, and a log scale was considered on substantive grounds.”

(a) Mean and variance of $\hat{L}$. Expectation is linear, so $E(\hat{L})={\sum }_i{c}_{i}E({\bar{X}}_i)={\sum }_i{c}_i{\mu }_i=L$; no independence is needed for this.

For the variance, expand the linear combination:

$$\mathrm{Var}\left(\sum _i{c}_i{\bar{X}}_i\right)=\sum _i{c}_i^2\mathrm{Var}({\bar{X}}_i)+2\sum _{i<i^{\prime }}{c}_i{c}_{i^{\prime }}\mathrm{Cov}({\bar{X}}_i,{\bar{X}}_{i^{\prime }}).$$

The covariances vanish because different groups consist of disjoint sets of independent observations, so the group means are independent — this is the one place independence enters. What remains is ${\sum }_i{c}_i^2{\sigma }^2/n_i$. $■$

By the theorem on expected mean squares, $MSW$ is unbiased for ${\sigma }^2$ whether or not $H_0$ holds, so replacing ${\sigma }^2$ by $MSW$ gives the standard error $\sqrt{MSW{\sum }_i{c}_i^2/n_i}$. For $c=(1,-1,0,\dots ,0)$ the sum is $1/n_1+1/n_2$ and the standard error is $\sqrt{MSW(1/n_1+1/n_2)}=S_p\sqrt{1/n_1+1/n_2}$, since $MSW$ is precisely the pooled variance — the two-sample formula recovered. ✓

(b) Invariance under a common shift. Adding $\delta$ to every mean,

$$\sum _i{c}_i({\mu }_i+\delta )=\sum _i{c}_i{\mu }_i+\delta \sum _i{c}_i.$$

If ${\sum }_i{c}_i=0$ the second term vanishes for every $\delta$. Conversely, if the value is unchanged for some $\delta \ne 0$, then $\delta {\sum }_i{c}_i=0$ forces ${\sum }_i{c}_i=0$. $■$

A comparison is a statement about differences between means, and differences are exactly the quantities that survive a change of origin; a combination with $\sum c_i\ne 0$ changes when every group is shifted equally, so it measures a level, which depends on where zero is.

(c) The fertiliser contrast. “Fertiliser 3 versus the average of 1 and 2” is $c=(-\frac{1}{2},-\frac{1}{2},1)$, which sums to zero ✓. The estimate is

$$\hat{L}=-\frac{1}{2}(8)-\frac{1}{2}(11)+20=-\frac{19}{2}+20=\frac{21}{2}.$$

For the standard error, ${\sum }_i{c}_i^2/n_i=(\frac{1}{4}+\frac{1}{4}+1)/4=\frac{3}{2}\cdot \frac{1}{4}=\frac{3}{8}$, so

$$SE=\sqrt{6\cdot \frac{3}{8}}=\sqrt{\frac{9}{4}}=\frac{3}{2},t=\frac{21/2}{3/2}=7.$$

Planned: against $t_{0.025,9}=2.262$, $7$ is far into the rejection region. Unplanned: Scheffé's critical value is $\sqrt{(k-1)F_{0.05,2,9}}=\sqrt{2\times 4.256}=\sqrt{8.512}\approx 2.92$ (since $2.92^2=8.53$), and $7>2.92$ as well.

The $t$ critical value is legitimate only if the contrast was fixed before the data were seen. If it was chosen after noticing that fertiliser 3 stood out, it is one of infinitely many contrasts that could have been picked, and only Scheffé's value — which bounds the maximum over all of them simultaneously — controls the error rate. Here the conclusion holds under either standard, so it does not matter which story is true; it often does.

(a) Marginal means and the main-effect sums of squares. Row means: ${\bar{x}}_{1\cdot }=(6+9+12)/3=9$ and ${\bar{x}}_{2\cdot }=(10+11+12)/3=11$. Column means: ${\bar{x}}_{\cdot 1}=(6+10)/2=8$, ${\bar{x}}_{\cdot 2}=10$, ${\bar{x}}_{\cdot 3}=12$. Grand mean $\bar{x}=(9+11)/2=10$, which also equals $(8+10+12)/3$ ✓.

Each row mean averages $bn=15$ observations and each column mean $an=10$, so

$$SSA=bn\sum _i({\bar{x}}_{i\cdot }-\bar{x}{)}^2=15[(-1{)}^2+1^2]=30,SSB=an\sum _j({\bar{x}}_{\cdot j}-\bar{x}{)}^2=10[(-2{)}^2+0+2^2]=80.$$

(b) Departures from additivity. Additivity predicts cell $(i,j)$ to be ${\bar{x}}_{i\cdot }+{\bar{x}}_{\cdot j}-\bar{x}$. Row 1 predictions: $9+8-10=7$, $9+10-10=9$, $9+12-10=11$; the actual means are $6,9,12$, so the departures are $-1,0,+1$. Row 2 predictions: $11+8-10=9$, $11$, $13$; actual $10,11,12$, departures $+1,0,-1$.

Row sums: $-1+0+1=0$ and $1+0-1=0$ ✓. Column sums: $-1+1$, $0+0$, $1-1$, all $0$ ✓. This is forced by the sum-to-zero constraints on the interaction effects, and it is why the interaction has only $(a-1)(b-1)=2$ degrees of freedom — two of the six departures determine the rest.

Each departure is shared by $n=5$ observations, so

$$SSAB=n\sum _{i,j}(\text{departure}{)}^2=5[1+0+1+1+0+1]=20.$$

(c) The table and the tests. Degrees of freedom: $A$ has $a-1=1$; $B$ has $b-1=2$; $AB$ has $(a-1)(b-1)=2$; error has $ab(n-1)=6\times 4=24$. Sum $1+2+2+24=29=abn-1=30-1$ ✓.

Mean squares: $MSA=30$, $MSB=40$, $MSAB=10$, $MSE=96/24=4$. Every effect is tested over $MSE$:

$$F_A=\frac{30}{4}=7.5,F_B=\frac{40}{4}=10,F_{AB}=\frac{10}{4}=\mathrm{2.5.}$$

Against the critical values: $F_A=7.5>4.260$, significant; $F_B=10>3.403$, significant; $F_{AB}=2.5<3.403$, not significant. By the two-way decomposition the total is $SST=30+80+20+96=226$ on $29$ degrees of freedom.

The interaction is not significant, so the main effects may be interpreted as averages over the other factor. Coating $A_2$ produces a response $2$ units higher than $A_1$ on average ($11$ against $9$). Each step up in curing temperature raises the response by $2$ units on average ($8,10,12$).

One caveat belongs in the report. A non-significant interaction is not a proof of additivity: in the sample, temperature raised $A_1$ by $6$ across the range but $A_2$ by only $2$, and with $MSE=4$ that difference is simply not distinguishable from noise at $F=2.5$. The main effects are the right summary, but “no interaction” should be read as “none detected”.

(a) The three plots. Each plot has $B_1$ and $B_2$ as two positions on the horizontal axis, cell mean on the vertical axis, and two line segments: the $A_1$ segment joins $(B_1,{\bar{x}}_{11})$ to $(B_2,{\bar{x}}_{12})$ and the $A_2$ segment joins $(B_1,{\bar{x}}_{21})$ to $(B_2,{\bar{x}}_{22})$. Label the segments and use a common vertical scale from $0$ to $12$ so the three pictures can be compared.

(i) $A_1$ runs from $4$ up to $8$; $A_2$ runs from $6$ up to $10$. The two segments are parallel, both rising by $4$, with $A_2$ a constant $2$ above $A_1$. Draw the vertical gap at both ends and mark it $2$ each time — that equal gap is additivity.

(ii) $A_1$ runs from $4$ up to $8$; $A_2$ runs from $8$ down to $4$. The segments form an X, crossing at the midpoint, height $6$. One slope is $+4$ and the other $-4$; mark both.

(iii) $A_1$ is flat at $4$; $A_2$ rises from $4$ to $10$. The segments start together at $B_1$ and diverge, the gap growing from $0$ to $6$. Mark the shared starting point.

(b) The sums of squares. Throughout, $a=b=2$, $n=3$, so each row and column mean averages $6$ observations and $SSA=6{\sum }_i({\bar{x}}_{i\cdot }-\bar{x}{)}^2$, $SSB=6{\sum }_j({\bar{x}}_{\cdot j}-\bar{x}{)}^2$, and $SSAB=3{\sum }_{i,j}(\text{departure}{)}^2$ with departure ${\bar{x}}_{ij}-{\bar{x}}_{i\cdot }-{\bar{x}}_{\cdot j}+\bar{x}$.

(i) Row means $6,8$; column means $5,9$; grand mean $7$. $SSA=6[1+1]=12$; $SSB=6[4+4]=48$. Predicted cells: $6+5-7=4$, $6+9-7=8$, $8+5-7=6$, $8+9-7=10$ — every cell is exactly predicted, so $SSAB=0$. Between-cell total: $3[(4-7{)}^2+(8-7{)}^2+(6-7{)}^2+(10-7{)}^2]=3[9+1+1+9]=60=12+48+0$ ✓.

(ii) Row means $6,6$; column means $6,6$; grand mean $6$. $SSA=0$ and $SSB=0$. Every cell is predicted to be $6$, so the departures are $-2,+2,+2,-2$ and $SSAB=3[4+4+4+4]=48$. Between-cell total: $3[4+4+4+4]=48=0+0+48$ ✓.

(iii) Row means $4,7$; column means $4,7$; grand mean $\frac{11}{2}$. $SSA=6[\frac{9}{4}+\frac{9}{4}]=27$ and likewise $SSB=27$. Predicted cells: $4+4-\frac{11}{2}=\frac{5}{2}$, $4+7-\frac{11}{2}=\frac{11}{2}$, $7+4-\frac{11}{2}=\frac{11}{2}$, $7+7-\frac{11}{2}=\frac{17}{2}$; actual $4,4,4,10$; departures $+\frac{3}{2},-\frac{3}{2},-\frac{3}{2},+\frac{3}{2}$. So $SSAB=3\cdot 4\cdot \frac{9}{4}=27$. Between-cell total: $3[\frac{9}{4}+\frac{9}{4}+\frac{9}{4}+\frac{81}{4}]=3\cdot 27=81=27+27+27$ ✓.

(c) Reading the pictures. (i) Parallel lines are the picture of additivity, so $SSAB$ had to vanish: the interaction is exactly the departure from parallel, and there is none. Both main effects are visible — the lines rise (a $B$ effect) and are separated (an $A$ effect).

(ii) The X is symmetric: the $A_1$ and $A_2$ segments have the same average height ($6$), so the row means agree and $SSA=0$; and the heights at $B_1$ and $B_2$ average to the same $6$, so the column means agree and $SSB=0$. Everything is interaction. Nothing vanishes in (iii): the lines are not parallel, their average heights differ, and the average heights at the two $B$ levels differ.

In (ii), “$B$ has no effect” reads $SSB=0$ as if $B$ were inert. But $SSB$ measures the effect of $B$ averaged over the levels of $A$, and here it is being averaged over a sign change. The simple effects are: within $A_1$, moving from $B_1$ to $B_2$ raises the response by $8-4=+4$; within $A_2$, it lowers it by $4-8=-4$. Factor $B$ moves the response by $4$ units at every level of $A$ — in opposite directions. When the interaction is present the correct report is these simple effects or the cell means themselves, never the main effect.

(a) Why random. The twelve laboratories are of no interest in themselves; they were drawn from a register so that the conclusion can be about the population of laboratories, and the question is how much laboratories vary. That is the random-effects model,

$$X_{ij}=\mu +A_i+{\epsilon }_{ij},A_i\stackrel{\text{iid}}{\sim }N(0,{\sigma }_A^2),{\epsilon }_{ij}\stackrel{\text{iid}}{\sim }N(0,{\sigma }^2),$$

with the $A_i$ independent of the ${\epsilon }_{ij}$. The null hypothesis is $H_0:{\sigma }_A^2=0$ — laboratories do not vary at all — rather than a statement about twelve particular means.

(b) The variance components. By the expected mean squares under random effects, $E(MSW)={\sigma }^2$ and $E(MSB)={\sigma }^2+n{\sigma }_A^2$. Equating mean squares to their expectations,

$${\hat{\sigma }}^2=MSW=15,{\hat{\sigma }}_A^2=\frac{MSB-MSW}{n}=\frac{75-15}{6}=10.$$

A single measurement has variance $\mathrm{Var}(X_{ij})=\mathrm{Var}(A_i)+\mathrm{Var}({\epsilon }_{ij})={\sigma }_A^2+{\sigma }^2$ by independence, estimated by $10+15=25$: a measurement sent to a randomly chosen laboratory has standard deviation about $5$ concentration units, of which laboratory choice and within-laboratory noise are both substantial parts. The intraclass correlation is

$$\frac{{\hat{\sigma }}_A^2}{{\hat{\sigma }}_A^2+{\hat{\sigma }}^2}=\frac{10}{25}=0.4,$$

so $40\%$ of the variance of a single measurement is between laboratories, and two portions measured by the same laboratory have correlation $0.4$.

(c) The test and what it means. $F=MSB/MSW=75/15=5$ on $k-1=11$ and $k(n-1)=60$ degrees of freedom. Since $5>1.952$, reject $H_0$.

The rejection says ${\sigma }_A^2>0$: laboratories in the population genuinely differ in what they report for the same material. It is a statement about all laboratories on the register, and the natural follow-up is the estimate ${\hat{\sigma }}_A^2=10$ and the intraclass correlation. Had the twelve been fixed — the only twelve the authority uses — the same $F=5$ would say that these particular laboratories do not all have the same mean, and the follow-up would be a post-hoc procedure asking which ones differ. The arithmetic is identical; the estimand is not.

(d) A negative estimate. Now ${\hat{\sigma }}_A^2=(12-15)/6=-\frac{1}{2}$. A variance cannot be negative, but the estimate is not a computational error: the method of moments returns a negative value whenever $MSB<MSW$, and under $H_0$ that happens roughly half the time, since both mean squares then estimate the same ${\sigma }^2$. The usual remedy is to truncate at zero and report ${\hat{\sigma }}_A^2=0$, with the caveat that truncation biases the estimator upward.

Here $F=12/15=0.8<1.952$, so $H_0$ is not rejected. What this indicates: the data provide no evidence of between-laboratory variance. What it does not indicate: that laboratories are identical. A small $F$ is not evidence for the null beyond the non-rejection, and with $n=6$ per laboratory the estimate of ${\sigma }_A^2$ is imprecise.

(a) The covariance structure. Fix a group $i$ and two distinct observations $j\ne l$ in it. The constant $\mu$ contributes nothing to any covariance, so by bilinearity

$$\mathrm{Cov}(X_{ij},X_{il})=\mathrm{Cov}(A_i+{\epsilon }_{ij},A_i+{\epsilon }_{il})=\mathrm{Var}(A_i)+\mathrm{Cov}(A_i,{\epsilon }_{il})+\mathrm{Cov}({\epsilon }_{ij},A_i)+\mathrm{Cov}({\epsilon }_{ij},{\epsilon }_{il}).$$

The last three terms are zero: the $A$'s are independent of the $\epsilon$'s, and distinct $\epsilon$'s are independent. So the covariance is ${\sigma }_A^2$ — the shared group effect is the only thing two group-mates have in common. For the variance, the same expansion with $j=l$ gives $\mathrm{Var}(X_{ij})={\sigma }_A^2+{\sigma }^2$, and therefore

$$\rho =\frac{\mathrm{Cov}(X_{ij},X_{il})}{\sqrt{\mathrm{Var}(X_{ij})\mathrm{Var}(X_{il})}}=\frac{{\sigma }_A^2}{{\sigma }_A^2+{\sigma }^2}.$$

For observations in different groups $i\ne i^{\prime }$, every one of the four terms in the expansion of $\mathrm{Cov}(A_i+{\epsilon }_{ij},A_{i^{\prime }}+{\epsilon }_{i^{\prime }l})$ is zero: distinct $A$'s are independent, $A$'s and $\epsilon$'s are independent, and distinct $\epsilon$'s are independent. So the two observations are uncorrelated. They are moreover independent, because the whole vector $(X_{ij})$ is a linear function of the independent normal variables $A_1,\dots ,A_k,{\epsilon }_{11},\dots ,{\epsilon }_{kn}$ and is therefore jointly normal, and for jointly normal variables zero correlation is independence. $■$

Since ${\sigma }_A^2\ge 0$ is a variance and the denominator is positive, $\rho \ge 0$: this model can never make group-mates negatively correlated. Equality $\rho =0$ holds exactly when ${\sigma }_A^2=0$, which is $H_0$ — under the null the group structure carries no information and the $kn$ observations are simply i.i.d. $N(\mu ,{\sigma }^2)$.

(b) Variance of the grand mean. Averaging the model over all $kn$ observations,

$$\bar{X}=\mu +\frac{1}{k}\sum _{i=1}^k{A}_i+\frac{1}{kn}\sum _{i,j}{\epsilon }_{ij}.$$

The two random sums are independent (one is built from the $A$'s, the other from the $\epsilon$'s), so their variances add:

$$\mathrm{Var}\left(\frac{1}{k}\sum _i{A}_i\right)=\frac{k{\sigma }_A^2}{k^2}=\frac{{\sigma }_A^2}{k},\mathrm{Var}\left(\frac{1}{kn}\sum _{i,j}{\epsilon }_{ij}\right)=\frac{kn{\sigma }^2}{(kn{)}^2}=\frac{{\sigma }^2}{kn},$$

using independence within each sum. Hence $\mathrm{Var}(\bar{X})={\sigma }_A^2/k+{\sigma }^2/(kn)$. $■$ (Equivalently: $\bar{X}$ is the average of the $k$ i.i.d. group means, each with variance ${\sigma }_A^2+{\sigma }^2/n$ by the theorem on expected mean squares under random effects, and dividing by $k$ gives the same thing.)

As $n\to \infty$ with $k$ fixed the second term vanishes and $\mathrm{Var}(\bar{X})\to {\sigma }_A^2/k$, which is not zero unless ${\sigma }_A^2=0$. More observations per group eliminate the measurement noise but leave the $k$ draws of the group effect exactly as they were; the grand mean can never estimate $\mu$ more precisely than an average of $k$ observations from $N(\mu ,{\sigma }_A^2)$ does. If the aim is $\mu$, precision comes from sampling more groups. With a fixed total of $kn$ observations, the design that spreads them over many groups beats the one that concentrates them in a few — for instance $k=100$, $n=2$ gives ${\sigma }_A^2/100+{\sigma }^2/200$, while $k=2$, $n=100$ gives ${\sigma }_A^2/2+{\sigma }^2/200$.

(c) Unbiasedness, and the cost of truncation. By the quoted expectations and linearity,

$$E({\hat{\sigma }}_A^2)=\frac{E(MSB)-E(MSW)}{n}=\frac{{\sigma }^2+n{\sigma }_A^2-{\sigma }^2}{n}={\sigma }_A^2.■$$

Write $T={\hat{\sigma }}_A^2$ and $T^{+}=\max (T,0)$. Then $T^{+}-T=\max (-T,0)\ge 0$ everywhere, and $T^{+}-T=-T>0$ exactly on the event $\{T<0\}=\{MSB<MSW\}$. A non-negative random variable that is strictly positive on an event of positive probability has strictly positive expectation, so if $P(MSB<MSW)>0$ then

$$E(T^{+})-E(T)=E[(T^{+}-T)]>0,\text{i.e.}E[\max ({\hat{\sigma }}_A^2,0)]>{\sigma }_A^2.■$$

Under $H_0$ the ratio $MSB/MSW$ has the $F_{k-1,k(n-1)}$ distribution, which has positive density on all of $(0,\infty )$, so $P(MSB/MSW<1)$ is strictly positive — in fact close to one half, since both mean squares are estimating the same ${\sigma }^2$ and their ratio falls on either side of $1$ with comparable probability. The consequence is uncomfortable: when the true between-group variance is exactly zero, the truncated estimator has strictly positive expectation. This is the awkwardness of the moment estimator that pushes practice toward likelihood-based estimates of variance components.

(a) Step 1 is correct; Steps 2 and 3 are not. Under indicator coding ${\hat{\beta }}_i={\bar{x}}_i-{\bar{x}}_1$, a difference of two independent group means, so its variance is ${\sigma }^2(1/n_1+1/n_i)$ and, estimating ${\sigma }^2$ by $MSW$,

$$SE({\hat{\beta }}_i)=\sqrt{10\left(\frac{1}{10}+\frac{1}{10}\right)}=\sqrt{2}.t=\frac{\mp 2}{\sqrt{2}}=\mp \sqrt{2}\approx \mp \mathrm{1.41.}✓$$

Step 2 misdescribes the $F$-test and then misreads a non-rejection. The $F$-test is a model comparison: it tests the single joint hypothesis ${\beta }_2={\beta }_3=0$ by comparing $SSE$ under the full model with $SSE$ under the intercept-only model. It is not a combination of the two coefficient tests, which are two uncorrected comparisons against the reference — and neither of them tests whether programmes 2 and 3 differ from each other. A joint hypothesis can be rejected while every marginal one survives. Separately, “not significant” is not “equivalent”: failing to reject is not evidence of equality.

Step 3 mistakes a reparameterisation for a contradiction. The coefficients are comparisons against whichever reference was chosen, so of course they change with the reference; what does not change is the set of fitted values, hence $SSE$, hence $F$. The two fits agree perfectly: under reference 2, ${\hat{\beta }}_3={\bar{x}}_3-{\bar{x}}_2=14-10=4$, which is exactly ${\hat{\beta }}_3-{\hat{\beta }}_2=2-(-2)=4$ from the first fit. Choosing which fit to report according to which looks significant is a selection, and it is exactly the uncorrected reference-level comparison the $F$-test exists to replace.

(b) The $F$-test. The grand mean is $(12+10+14)/3=12$, so

$$SSB=10[(12-12{)}^2+(10-12{)}^2+(14-12{)}^2]=10[0+4+4]=80,SSW=27\times 10=270.$$

By the decomposition of the total sum of squares, $SST=350$. The full model fits each group its own mean, so $SS{E}_{\text{full}}=SSW=270$, while the intercept-only model has $SSE=SST=350$. The model-comparison statistic is

$$F=\frac{(350-270)/2}{270/27}=\frac{40}{10}=4.0>\mathrm{3.354.}$$

Reject $H_0$ at $5\%$: the three programmes do not all have the same mean score. Step 2's conclusion is the opposite of the truth.

(c) What $F$ sees. Programme 1 happens to sit in the middle of the three means. Each coefficient in the reference-1 coding measures a gap of $2$, which is $\sqrt{2}\approx 1.41$ standard errors; but the gap between programmes 2 and 3 is $4$, which is $2\sqrt{2}\approx 2.83$ standard errors, and no coefficient in that coding measures it. The $F$-test does not compare anything to a reference: $SSB$ counts programme 2 and programme 3 as each $2$ away from the grand mean on opposite sides, so the $4$-unit spread between them is fully present in the numerator. That is why $F$ rejects while both coefficients fail — the reference coding was looking at the two smallest of the three differences.

Whether the 2-versus-3 comparison survives a familywise correction is a Tukey question. With equal group sizes,

$$\text{HSD}=q_{0.05,3,27}\sqrt{\frac{MSW}{n}}=3.51\sqrt{\frac{10}{10}}=\mathrm{3.51.}$$

The differences are $|12-10|=2$, $|12-14|=2$ and $|10-14|=4$. Only $4>3.51$: programmes 2 and 3 differ at familywise $5\%$, and neither can be separated from programme 1. So the $t=2.83$ in Step 3 was uncorrected, but the finding it pointed at is genuine. The correct report is: the programmes differ ($F=4.0$, $p<0.05$), and the difference the data can locate is between programmes 2 and 3.