Print this page and work through it on paper. Revealed answers are never printed, so you can check one question and still print a clean sheet.

1. 1.Computation · 8 min

   A population of six units has the potential outcomes below — both columns written out, which an experimenter never sees, and which is exactly why the table is useful for checking what an estimator does.

   $$\begin{array}{ccccccc}i& 1& 2& 3& 4& 5& 6\\ Y_i(0)& 4& 6& 8& 10& 12& 14\\ Y_i(1)& 6& 8& 12& 14& 18& 20\end{array}$$

   1. (a)Compute every individual treatment effect ${\tau }_i$ and the average treatment effect $\tau$. Then explain why no experiment on these six units, however large or carefully run, could ever compute a single ${\tau }_i$.
   2. (b)A clinician assigns the treatment to units $4$, $5$ and $6$ — “the strongest candidates”. Compute the naive difference in observed group means, then decompose it into the effect on the treated plus the selection bias, checking that the two pieces add up.
   3. (c)Instead, three of the six units are chosen for treatment uniformly at random, all $\left(\genfrac{}{}{0ex}{}{6}{3}\right)=20$ subsets equally likely. Without listing the twenty assignments, show that the expected value of the naive difference is exactly $\tau$. Then compute the naive difference for the particular draw $\{1,3,5\}$ and say what its disagreement with $\tau$ does and does not mean.
   Show the solution to question 1

   (a) Subtract the columns. ${\tau }_i=Y_i(1)-Y_i(0)$ gives $2,2,4,4,6,6$, so

   $$\tau =\frac{2+2+4+4+6+6}{6}=\frac{24}{6}=4.$$

   No experiment can compute a ${\tau }_i$ because each unit reveals only one column: once unit $i$ is treated it shows $Y_i(1)$ and $Y_i(0)$ becomes counterfactual, and if it is left untreated the reverse. Observing the unit more often, or more precisely, produces more copies of the same column. This is the fundamental problem of causal inference, and it is why the rest of the question works with the average $\tau$, which only needs each column to be observed on some units.

   (b) Naive difference, then decompose. Units $4,5,6$ are treated and reveal $Y(1)=14,18,20$, mean $\frac{52}{3}$. Units $1,2,3$ are controls and reveal $Y(0)=4,6,8$, mean $6=\frac{18}{3}$. The naive difference is

   $${\bar{Y}}_T-{\bar{Y}}_C=\frac{52}{3}-\frac{18}{3}=\frac{34}{3}\approx \mathrm{11.3.}$$

   Effect on the treated: the mean of ${\tau }_i$ over units $4,5,6$ is $\frac{4+6+6}{3}=\frac{16}{3}$.

   Selection bias: $E[Y(0)\mid T=1]-E[Y(0)\mid T=0]$ is the mean of $Y(0)$ over the treated, $\frac{10+12+14}{3}=12$, minus the mean over the controls, $6$; that is $6=\frac{18}{3}$.

   Check: $\frac{16}{3}+\frac{18}{3}=\frac{34}{3}$. ✓

   The naive figure is almost three times the truth $\tau =4$. Most of the gap is selection — the chosen units would have scored $6$ higher than the others under no treatment at all — and even the effect on the treated, $\frac{16}{3}$, exceeds $\tau$ because the strongest units also happen to have the largest ${\tau }_i$.

   (c) Expectation without enumeration. Fix unit $i$. It is in the treated set exactly when the other two treated units are chosen from the remaining five, so it is treated in $\left(\genfrac{}{}{0ex}{}{5}{2}\right)=10$ of the $20$ assignments and $P(T_i=1)=\frac{1}{2}$ for every $i$. The treated mean is $\frac{1}{3}{\sum }_i{T}_i{Y}_i(1)$, and since the table is fixed and only $T$ is random,

   $$E[{\bar{Y}}_T]=\frac{1}{3}\sum _{i=1}^6{Y}_i(1)P(T_i=1)=\frac{1}{6}\sum _i{Y}_i(1)=\frac{78}{6}=13,$$

   using $6+8+12+14+18+20=78$. Identically $E[{\bar{Y}}_C]=\frac{1}{6}{\sum }_i{Y}_i(0)=\frac{54}{6}=9$. Hence $E[{\bar{Y}}_T-{\bar{Y}}_C]=13-9=4=\tau$. This is the theorem Randomisation identifies the average treatment effect in its smallest case: the random draw does not consult the table, so every unit's $Y(1)$ enters at the same weight $\frac{1}{2}$.

   For the draw $\{1,3,5\}$: treated values $Y(1)=6,12,18$, mean $12$; controls $2,4,6$ with $Y(0)=6,10,14$, mean $10$; naive difference $2$. Decomposing as in (b): effect on the treated $\frac{2+4+6}{3}=4$, selection bias $\frac{4+8+12}{3}-\frac{6+10+14}{3}=8-10=-2$, and $4-2=2$. ✓

   What this means: a single randomised draw is not free of selection bias — here it is $-2$, and the estimate $2$ is half the truth. Randomisation promises that the bias is zero on average across draws, which is what “unbiased” says, and the spread of that bias term from draw to draw is exactly what a standard error measures. What it does not mean is that this experiment went wrong: nothing about the draw could have been checked against a truth the experimenter never sees.

2. 2.Critique · 6 min

   The passage below is the methods-and-conclusions section of a trial report. Every sentence after the first contains at least one error of design or of reasoning.

   1. Eighty volunteers aged $20$ to $35$ were assigned to drug or no drug by a coin flip for each volunteer.

   2. The first allocation produced groups whose mean ages differed by four years, so the allocation was re-drawn until the balance table showed no significant difference on any of the fifteen baseline variables.

   3. The groups were then identical at baseline, so any later difference in pain scores must be the effect of the drug.

   4. The clinicians who scored each patient's pain knew which group the patient was in.

   5. The drug reduced pain scores, so we recommend it for elderly patients, in whom the condition is most common.

   1. (a)For each of sentences 2 to 5, state what is wrong, name the concept it violates, and give the procedure that should have been used instead.
   2. (b)Suppose every flaw were repaired. State precisely the strongest claim the trial could then support.
   Show the solution to question 2

   (a) Sentence 2 — re-randomising until the table looks balanced. The theorem Randomisation identifies the average treatment effect needs $T\perp (Y(0),Y(1))$. Discarding every allocation in which age is unbalanced makes the final assignment a function of the covariates: allocations that put older volunteers in one arm are systematically thrown away, so the mechanism is no longer independent of anything correlated with age — and pain is. The imbalance was expected, not a failure: testing fifteen baseline variables at $5\%$ gives a probability $1-0.95^{15}$ of at least one “significant” difference by chance alone, which is roughly even odds (about $54\%$). Correct procedure: keep the first allocation. If age is known to matter, block on it in advance — randomise within age strata — or pre-specify an adjustment for age in the analysis.

   Sentence 3 — “identical”. Randomisation never promises identical groups; with forty per arm they will visibly differ on something. It promises that any difference is accidental, hence governed by known probability, hence exactly what the standard error measures. “No significant difference” is a statement about a test's power, not about equality. And “any later difference must be the drug” ignores everything that happens after assignment, which is the next sentence's problem.

   Sentence 4 — unblinded assessment, and no placebo. Randomisation makes the groups comparable before treatment; blinding keeps them comparable afterwards. Pain is reported by the patient and scored by the clinician, so a patient who knows they received the drug reports differently and a clinician who knows the assignment scores differently, and the comparison is contaminated even though the randomisation was sound. Correct procedure: the no-drug arm receives a placebo indistinguishable from the drug (single-blind), and the assessors do not know assignments (double-blind).

   Sentence 5 — extrapolation. The population randomised was volunteers aged $20$ to $35$. Randomisation buys internal validity — the comparison inside the study is sound — and buys external validity not at all; nothing in the trial speaks to the elderly, whose physiology, dosing and baseline pain differ. Correct procedure: state the conclusion for the population studied, and run a trial in the elderly if they are the population of interest.

   (b) The strongest defensible claim. With one allocation kept as drawn, a placebo arm, double-blind assessment, and the analysis pre-specified: the difference in mean pain score between arms is an unbiased estimate of the average effect of the drug relative to placebo, in volunteers aged $20$ to $35$ of the kind who enrolled, with a standard error that the randomisation licenses. That is a causal claim about that population — and about no other.

3. 3.Computation · 7 min

   Two irrigation schedules are to be compared on field plots. Each plot's yield has variance ${\sigma }^2=36$. Two plots on the same terrace have yields correlated at $\rho =\frac{1}{2}$; plots on different terraces are independent. Twenty plots are available.

   1. (a)Design U takes one plot from each of twenty different terraces and assigns ten to each schedule at random. Find $\mathrm{Var}({\bar{Y}}_T-{\bar{Y}}_C)$, stating the independence you use.
   2. (b)Design B uses ten terraces, two plots on each, and randomises which plot on a terrace gets which schedule. Find $\mathrm{Var}(\bar{D})$ for the mean within-terrace difference, and the ratio of the two variances.
   3. (c)How many plots would Design U need to match the precision of Design B's twenty? And what within-terrace correlation would Design B need for a fourfold variance reduction over Design U?
   4. (d)Suppose the terraces turn out to be irrelevant, $\rho =0$. Compare the two designs' variances and error degrees of freedom, and say which is now the better design and why.
   Show the solution to question 3

   (a) Independent plots, so variances add. Every plot is on its own terrace, so all twenty yields are independent. Each arm's mean is a mean of ten independent plots with variance ${\sigma }^2/10$, and the two arm means are independent of each other, so

   $$\mathrm{Var}({\bar{Y}}_T-{\bar{Y}}_C)=\frac{36}{10}+\frac{36}{10}=\frac{72}{10}=\frac{36}{5}=\mathrm{7.2.}$$

   (b) Difference within a terrace. For terrace $i$ let $D_i=Y_{iT}-Y_{iC}$. Then

   $$\mathrm{Var}(D_i)={\sigma }^2+{\sigma }^2-2\rho {\sigma }^2=36+36-2\cdot \frac{1}{2}\cdot 36=36=2{\sigma }^2(1-\rho ).$$

   The ten terraces are independent, so $\mathrm{Var}(\bar{D})=\frac{36}{10}=\frac{18}{5}=3.6$. The ratio is $\frac{3.6}{7.2}=\frac{1}{2}=1-\rho$, as the theorem Variance reduction from blocking predicts.

   (c) Matching precision, and a fourfold gain. With $m$ plots per schedule Design U has variance $2{\sigma }^2/m=72/m$. Setting $72/m=3.6$ gives $m=20$ per schedule, so $40$ plots — twice Design B's twenty. For a fourfold reduction we need $1-\rho =\frac{1}{4}$, that is $\rho =\frac{3}{4}$.

   (d) Blocking on nothing. With $\rho =0$, $\mathrm{Var}(\bar{D})=2{\sigma }^2/10=7.2$, the same as Design U. But the analyses differ in degrees of freedom: Design U is a two-sample comparison of $10$ against $10$, with $20-2=18$ error degrees of freedom, while Design B is a paired comparison of ten differences with $10-1=9$. The $t$ critical value on $9$ degrees of freedom is larger than on $18$, so Design B is slightly less powerful for the same variance: the degrees of freedom spent on terrace effects bought nothing. Design U is now the better design. Blocking is only worth doing on a variable genuinely related to the outcome — and whichever design is run, it must be analysed as designed: paired data analysed as two independent samples is invalid, not merely inefficient.

4. 4.Computation · 6 min

   Three tyre compounds $A$, $B$, $C$ are to be compared for wear. Three test cars are available on each of three days, giving nine runs. Both the car and the day are believed to affect wear, but neither is of interest in itself.

   1. (a)Lay out a $3\times 3$ Latin square for the nine runs, and say which property of the layout guarantees that neither the car effect nor the day effect can be confounded with the compound comparison.
   2. (b)Count the degrees of freedom for the mean, cars, days, compounds and error. How many runs would a full factorial crossing all three factors require?
   3. (c)A colleague proposes running compound $A$ on car 1, $B$ on car 2 and $C$ on car 3 on every day, arguing that this also uses nine runs and balances the days. Explain exactly what goes wrong.
   4. (d)State the assumption the Latin square makes that the full factorial does not, and explain why the nine runs cannot be used to check it.
   Show the solution to question 4

   (a) One square, and the property that matters. Index rows by day and columns by car:

   $$\begin{array}{cccc}& \text{Car }1& \text{Car }2& \text{Car }3\\ \text{Day }1& A& B& C\\ \text{Day }2& B& C& A\\ \text{Day }3& C& A& B\end{array}$$

   Each compound appears exactly once in every row and exactly once in every column. So the three runs of any one compound cover each day once and each car once, and a day effect (or a car effect) adds the same total to every compound. It therefore cancels in every difference between compounds, which is what “cannot be confounded” means. Any other cyclic shift of the rows works equally well.

   (b) Degrees of freedom. Nine runs give $9$ degrees of freedom in total: $1$ for the mean, $2$ for cars, $2$ for days, $2$ for compounds, leaving

   $$9-1-2-2-2=2$$

   for error. A full factorial crossing three factors at three levels needs $3\times 3\times 3=27$ runs, three times the budget.

   (c) The proposal confounds compound with car. The colleague's grid has every row $ABC$. Rows are fine — each compound once per day — but the columns fail: $A$ appears three times in the car 1 column and never elsewhere. Compound $A$ is only ever observed on car 1, $B$ only on car 2, $C$ only on car 3, so the compound-to-compound difference $A-B$ is inseparable from the car-to-car difference $1-2$. Any car effect looks like a compound effect and nothing in the nine numbers can tell them apart. The layout controls one nuisance (days) and aliases the compound comparison with the other. That is a one-way blocking, not a Latin square.

   (d) Additivity, and why it cannot be checked. The Latin square assumes there are no interactions among day, car and compound — that each acts additively. All $9-1=8$ non-mean degrees of freedom are spent on the three sets of main effects and the $2$ error degrees of freedom; there is no run left over to estimate any interaction, so a car-by-compound interaction, if present, is silently absorbed into the error and the compound estimates. The $27$-run factorial has $26-6=20$ degrees of freedom for interactions and can test them; the square cannot. It buys its efficiency by assuming what the factorial would have measured.

5. 5.Computation · 9 min

   A $2^2$ experiment on the drying time of a coating varies temperature ($A$) and airflow ($B$), each at a low level ($0$) and a high level ($1$), with $n=4$ runs per cell. The cell means are ${\bar{y}}_{00}=40$, ${\bar{y}}_{10}=50$, ${\bar{y}}_{01}=46$, ${\bar{y}}_{11}=50$, where the first index is the level of $A$ and the second of $B$, and the error mean square is $MSE=16$. Use $t_{0.025,12}=2.179$.

   1. (a)Estimate the main effect of $A$, the main effect of $B$, and the interaction. Compute the interaction two ways — from $A$'s effect at each level of $B$, and from $B$'s effect at each level of $A$ — and confirm they agree.
   2. (b)Find the standard error common to the three estimates and the error degrees of freedom, and test each estimate at the $5\%$ level.
   3. (c)In the light of (b), is it legitimate to report “raising the temperature changes drying time by $7$” as a single number? Say what the data show about $A$'s effect at each level of $B$ and how you would word the report.
   4. (d)A one-factor-at-a-time study with the same $16$ runs puts $8$ at $(0,0)$ and $8$ at $(1,0)$. Find the variance of its estimate of $A$'s effect, compare with the factorial's, and list what it cannot estimate.
   Show the solution to question 5

   (a) Effects from the cell means. $A$'s effect at $B$ low is $50-40=10$ and at $B$ high is $50-46=4$, so the main effect of $A$ is the average, $\frac{10+4}{2}=7$. $B$'s effect at $A$ low is $46-40=6$ and at $A$ high is $50-50=0$, so the main effect of $B$ is $\frac{6+0}{2}=3$.

   Interaction from $A$'s two effects: $\frac{4-10}{2}=-3$. From $B$'s two effects: $\frac{0-6}{2}=-3$. ✓ Both are the same contrast $\frac{1}{2}({\bar{y}}_{11}-{\bar{y}}_{10}-{\bar{y}}_{01}+{\bar{y}}_{00})=\frac{1}{2}(50-50-46+40)=-3$, which is why they had to agree.

   (b) One standard error for all three. Each estimate is a contrast in the four independent cell means with coefficients $\pm \frac{1}{2}$, so

   $$\mathrm{Var}=\sum c_i^2\cdot \frac{MSE}{n}=4\cdot \frac{1}{4}\cdot \frac{16}{4}=4,SE=2.$$

   Error degrees of freedom: four cells each contribute $n-1=3$, so $12$. The statistics are $t_A=\frac{7}{2}=3.5$, $t_B=\frac{3}{2}=1.5$, $t_{AB}=\frac{-3}{2}=-1.5$. Against $2.179$: temperature is significant; airflow and the interaction are not.

   (c) Reporting. Because the interaction is not significant, the main effect of $A$ is a legitimate one-number summary: the data do not establish that $A$'s effect depends on $B$. But the point estimates do differ — $10$ at low airflow, $4$ at high — and the test had little power to see it (an estimate of $-3$ against a standard error of $2$). Not significant is not the same as zero. Word it honestly: “Raising the temperature increased drying time by $7$ on average (SE $2$). The increase appeared larger at low airflow ($10$ against $4$), but the difference was not significant ($t=-1.5$ on $12$ degrees of freedom).”

   (d) The one-factor-at-a-time study. Its estimate is ${\bar{y}}_{10}-{\bar{y}}_{00}$ with $8$ runs at each condition, so

   $$\mathrm{Var}=\frac{16}{8}+\frac{16}{8}=4,$$

   exactly the factorial's. That is the theorem Factorial designs estimate more from the same runs with $N=16$: both designs give $4{\sigma }^2/N=64/16=4$ for $A$. The difference is what else is learned. The one-factor study estimates $10$, valid only at low airflow; it cannot estimate $B$'s effect, cannot estimate the interaction, and cannot discover that the $10$ shrinks to $4$ when airflow is high. The factorial got $A$ at the same precision and $B$ and $AB$ for free.

6. 6.Sketch · 8 min

   Two $2^2$ experiments on the same two factors returned the following cell means (first index $A$, second $B$).

   Experiment I: ${\bar{y}}_{00}=10$, ${\bar{y}}_{10}=16$, ${\bar{y}}_{01}=14$, ${\bar{y}}_{11}=20$.

   Experiment II: ${\bar{y}}_{00}=10$, ${\bar{y}}_{10}=16$, ${\bar{y}}_{01}=14$, ${\bar{y}}_{11}=8$.

   An interaction plot puts the level of $A$ on the horizontal axis (low at the left, high at the right) and the cell mean on the vertical axis, and joins the two cells at each level of $B$ by a straight segment — one segment for $B$ low, one for $B$ high.

   1. (a)For each experiment compute the main effect of $A$, the main effect of $B$, and the interaction.
   2. (b)Draw the interaction plot for each experiment on its own axes, labelling every endpoint with its value. Then say which geometric feature of the plot displays each of the three effects, and what “zero interaction” looks like.
   3. (c)In Experiment II the two segments cross. Find the crossing point exactly — how far along from $A$ low to $A$ high, and at what height. Then explain why an experimenter who tested $A$ only at $B$ low would report the same conclusion for both experiments, and what the factorial reports instead for Experiment II.
   Show the solution to question 6

   (a) The three effects. Experiment I: $A=\frac{(16-10)+(20-14)}{2}=\frac{6+6}{2}=6$, $B=\frac{(14-10)+(20-16)}{2}=\frac{4+4}{2}=4$, $AB=\frac{6-6}{2}=0$.

   Experiment II: $A=\frac{(16-10)+(8-14)}{2}=\frac{6-6}{2}=0$, $B=\frac{(14-10)+(8-16)}{2}=\frac{4-8}{2}=-2$, $AB=\frac{-6-6}{2}=-6$.

   (b) The two plots. Experiment I: the $B$-low segment runs from $(\text{low},10)$ to $(\text{high},16)$ and the $B$-high segment from $(\text{low},14)$ to $(\text{high},20)$. Both rise by $6$; they are parallel, with a vertical gap of $4$ at each end. Experiment II: the $B$-low segment again runs from $10$ up to $16$, but the $B$-high segment runs from $14$ down to $8$. The picture is an X.

   What each feature shows. The slope of a segment is $A$'s effect at that level of $B$, so the main effect of $A$ is the average slope: $6$ in I; $0$ in II, where one segment rises by exactly what the other falls. The main effect of $B$ is the average vertical distance from the $B$-low segment up to the $B$-high segment: $4$ in I; in II it is $+4$ at the left end and $-8$ at the right, averaging $-2$. The interaction is half the difference of the slopes: $\frac{6-6}{2}=0$ and $\frac{-6-6}{2}=-6$. Zero interaction therefore looks like parallel segments, as in I: $A$ does the same thing whatever $B$ is. Crossing segments, as in II, mean the interaction is so large that $A$'s effect reverses sign.

   (c) The crossing, and the one-factor report. Let $t\in [0,1]$ measure the way across from $A$ low to $A$ high. The $B$-low segment has height $10+6t$ and the $B$-high segment $14-6t$; they meet when $12t=4$, so $t=\frac{1}{3}$, at height $10+2=12$. The segments cross one third of the way across, at $12$. (The horizontal axis is categorical, so this is a feature of the drawing, not a prediction about an intermediate temperature.)

   An experimenter who tests $A$ at $B$ low sees ${\bar{y}}_{10}-{\bar{y}}_{00}=16-10=6$ in both experiments and reports “raising $A$ adds $6$” for both. In Experiment I that is the whole truth. In Experiment II it is true only at $B$ low; at $B$ high, raising $A$ subtracts $6$, and the one-factor design never ran the cell that would have shown it. The factorial's main effect of $A$ is $0$, which quoted alone would be equally misleading — “$A$ does nothing” is false at both levels of $B$. The honest report is conditional: $A$ adds $6$ at low $B$ and removes $6$ at high $B$, the interaction is $-6$, and no single number for $A$'s effect exists.

7. 7.Proof · 10 min

   Throughout, each unit has potential outcomes $Y(0)$ and $Y(1)$, a treatment indicator $T\in \{0,1\}$ with $0<P(T=1)<1$, and observed outcome $Y=TY(1)+(1-T)Y(0)$. Write $\tau =E[Y(1)-Y(0)]$ for the average treatment effect and ${\tau }_{\mathrm{ATT}}=E[Y(1)-Y(0)\mid T=1]$ for the average effect on the treated.

   1. (a)Prove, with no assumption whatsoever about how $T$ was assigned, the identity $$E[Y\mid T=1]-E[Y\mid T=0]={\tau }_{\mathrm{ATT}}+(E[Y(0)\mid T=1]-E[Y(0)\mid T=0]),$$ and say in words what each term on the right measures.
   2. (b)Now assume $T\perp (Y(0),Y(1))$. Prove that the bracketed term is zero and that ${\tau }_{\mathrm{ATT}}=\tau$, and conclude that the difference in group means is unbiased for $\tau$. Mark the exact step at which independence is used.
   3. (c)Suppose treatment is assigned by independent coin flips with $P(T_i=1)=p$ for every unit, where $p\in (0,1)$ need not equal $\frac{1}{2}$. Does the conclusion of (b) survive? Identify where $p$ could have entered the argument, and say what $p$ does affect.
   Show the solution to question 7

   (a) An identity that holds for every design. On the event $T=1$ the observed outcome is $Y=Y(1)$, and on $T=0$ it is $Y=Y(0)$. Both events have positive probability, so the conditional expectations exist and

   $$E[Y\mid T=1]=E[Y(1)\mid T=1],E[Y\mid T=0]=E[Y(0)\mid T=0].$$

   Now add and subtract $E[Y(0)\mid T=1]$:

   $$E[Y(1)\mid T=1]-E[Y(0)\mid T=0]=(E[Y(1)\mid T=1]-E[Y(0)\mid T=1])+(E[Y(0)\mid T=1]-E[Y(0)\mid T=0]).$$

   By linearity of conditional expectation the first bracket is $E[Y(1)-Y(0)\mid T=1]={\tau }_{\mathrm{ATT}}$, which is the identity. $■$

   In words: ${\tau }_{\mathrm{ATT}}$ is the average effect of the treatment on the units that were actually treated. The second bracket is selection bias — how the treated units would have differed from the controls had nobody been treated, a pure baseline difference. Nothing was assumed, so every design faces this identity; a design can only hope to make the second bracket vanish.

   (b) Randomisation kills the bracket. Independence says the conditional distribution of $(Y(0),Y(1))$ given $T$ is its unconditional distribution. Therefore

   $$E[Y(0)\mid T=1]=E[Y(0)]=E[Y(0)\mid T=0]\text{(independence used here, and only here)},$$

   so the bracket is $0$. By the same fact $E[Y(1)\mid T=1]=E[Y(1)]$ and $E[Y(0)\mid T=1]=E[Y(0)]$, so

   $${\tau }_{\mathrm{ATT}}=E[Y(1)]-E[Y(0)]=E[Y(1)-Y(0)]=\tau .$$

   Putting these into (a): $E[Y\mid T=1]-E[Y\mid T=0]=\tau$. The group mean ${\bar{Y}}_T$ averages $Y_i(1)$ over the treated units, and for each of them $E[Y_i(1)\mid T]=E[Y_i(1)]$ by independence, so $E[{\bar{Y}}_T]=E[Y(1)]$ whenever the treated group is non-empty; likewise $E[{\bar{Y}}_C]=E[Y(0)]$. Hence $E[{\bar{Y}}_T-{\bar{Y}}_C]=\tau$. $■$

   This is the theorem Randomisation identifies the average treatment effect, reached through (a) so that one can see what randomisation is removing: a selection term that is otherwise present and invisible.

   (c) A biased coin is still a coin. The conclusion survives. The proof used one hypothesis — $T\perp (Y(0),Y(1))$ — and a coin with bias $p$ satisfies it for every $p$, because the coin does not consult the potential outcomes. The only place $p$ could have entered is the marked step: the claim $E[Y(0)\mid T=1]=E[Y(0)]$ says the conditional distribution given $T=1$ equals the marginal, and that is true whatever $P(T=1)$ is, provided it is positive so that the conditioning makes sense. The same holds for the $T=0$ side, which is why $p<1$ is needed too.

   What $p$ does affect is precision, not bias. The group sizes are about $Np$ and $N(1-p)$, so the variance of ${\bar{Y}}_T-{\bar{Y}}_C$ is about $\frac{{\sigma }_1^2}{Np}+\frac{{\sigma }_0^2}{N(1-p)}$, which for equal variances is smallest at $p=\frac{1}{2}$ and grows without bound as $p$ approaches $0$ or $1$. Unbiasedness is a property of independence; efficiency is a property of balance.

8. 8.Computation · 8 min

   Two protective coatings are to be compared on steel test panels. A difference in mean lifetime of $\delta =5$ months is worth detecting, the panel-to-panel standard deviation is $\sigma =10$ months, and the test is two-sided at $\alpha =0.05$ with power $0.80$. Use $z_{0.025}=1.96$ and $z_{0.20}=0.84$.

   1. (a)Find the number of replicates per coating and the total number of panels from the theorem Replicates needed to detect an effect, showing the substitution.
   2. (b)Panels can be cut in pairs from the same steel sheet, and paired panels have lifetimes correlated at $\rho =\frac{3}{4}$. Redo the calculation for a paired design, and check that the ratio of the two totals is what the theorem Variance reduction from blocking predicts.
   3. (c)The engineers now decide that a difference of $\delta =2.5$ months matters. Without repeating the full substitution, state the new numbers for both designs and justify the shortcut.
   4. (d)A colleague suggests running the first ten panels, estimating $\sigma$ from them, and then choosing $n$. Say which quantities in the formula must be fixed before data collection, and what is wrong with the suggestion.
   Show the solution to question 8

   (a) Unpaired. $z_{\alpha /2}+z_{\beta }=1.96+0.84=2.80$, so $(z_{\alpha /2}+z_{\beta }{)}^2=7.84$. The theorem gives, per coating,

   $$n\ge \frac{2(7.84)(100)}{25}=\frac{1568}{25}=62.72,$$

   so $63$ replicates per coating and $126$ panels in total. (Check the arithmetic: $2\times 7.84=15.68$, times $100$ is $1568$, and $25\times 62=1550$ with remainder $18$, so $62.72$.)

   (b) Paired. Blocking multiplies the error variance by $1-\rho$, so the effective ${\sigma }^2$ is $100\times \frac{1}{4}=25$, and

   $$n\ge \frac{2(7.84)(25)}{25}=15.68,$$

   so $16$ pairs, $32$ panels. The bounds before rounding are in the ratio $\frac{15.68}{62.72}=\frac{1}{4}=1-\rho$ exactly; after rounding, $\frac{32}{126}=\frac{16}{63}$, which is $\frac{1}{4}$ up to the rounding to whole panels. The pairing has cut the experiment to a quarter of its size, which is the factor $(1-\rho )$ applied to the variance.

   (c) Halving the effect. $\delta$ enters the bound only through ${\delta }^2$ in the denominator, so halving $\delta$ multiplies the bound by $4$: unpaired $4\times 62.72=250.88$, so $251$ per coating and $502$ panels; paired $4\times 15.68=62.72$, so $63$ pairs and $126$ panels. Round the bound, not the rounded $n$ — $4\times 63=252$ would be one panel too many. The standardised effect has gone from $\delta /\sigma =\frac{1}{2}$ to $\frac{1}{4}$, and small effects are expensive.

   (d) What must be fixed in advance. Everything on the right-hand side: $\alpha$, the power $1-\beta$, the effect $\delta$ worth detecting, and $\sigma$. None of them comes from the data being collected. The suggestion goes wrong in two ways. First, an estimate of $\sigma$ from ten panels is itself very noisy — its own sampling error is large — so the chosen $n$ will be badly wrong in one direction or the other, and a design sized by a lucky low estimate will be underpowered. Second, if those ten panels are also part of the final analysis, then the sample size — and so the analysis — was chosen after looking at outcomes, and the test's stated level is no longer guaranteed. The value of $\sigma$ should come from earlier work or from a separate pilot that is not part of the experiment being analysed, and $\delta$ from a judgement about what matters, settled before the first panel is cut.

9. 9.Computation · 9 min

   A registry records the outcome of two treatments for an infection, split by whether the case was mild or severe at presentation. Each entry is successes out of patients treated.

   $$\begin{array}{lccc}& \text{Mild}& \text{Severe}& \text{All cases}\\ \text{New drug}& 40/50& 60/150& 100/200\\ \text{Standard drug}& 105/150& 15/50& 120/200\end{array}$$

   1. (a)Compute all six success rates and confirm that the comparison reverses between the strata and the totals.
   2. (b)Explain the reversal. Identify the confounder, and show from the allocation numbers that it is associated with both the treatment and the outcome.
   3. (c)Compute the adjusted success rate of each drug by stratification, weighting each stratum by its share of all $400$ patients, and state the adjusted difference. Compare it with the unadjusted difference.
   4. (d)State exactly the assumption under which the adjusted difference in (c) is the causal effect. Then give an example of a recorded variable for which the same stratification arithmetic would be an error rather than a correction, and say why.
   Show the solution to question 9

   (a) Six rates. New drug: mild $\frac{40}{50}=80\%$, severe $\frac{60}{150}=40\%$, overall $\frac{100}{200}=50\%$. Standard drug: mild $\frac{105}{150}=70\%$, severe $\frac{15}{50}=30\%$, overall $\frac{120}{200}=60\%$. The totals are consistent: $40+60=100$ of $50+150=200$, and $105+15=120$ of $150+50=200$.

   The new drug is better in mild cases ($80$ against $70$) and better in severe cases ($40$ against $30$), by ten points each time — and worse overall ($50$ against $60$). That is Simpson's paradox.

   (b) Severity is the confounder. It is associated with treatment: the new drug went to severe cases $150$ times out of $200$ ($75\%$), while the standard drug went to mild cases $150$ times out of $200$ ($75\%$). It is associated with outcome under either drug: severe cases succeed at $40\%$ or $30\%$ against $80\%$ or $70\%$ for mild. And it is not on the causal pathway — severity was assessed at presentation, before any drug was given.

   The overall rates are therefore weighted averages with different weights. New drug: $\frac{1}{4}(80)+\frac{3}{4}(40)=20+30=50$. Standard: $\frac{3}{4}(70)+\frac{1}{4}(30)=52.5+7.5=60$. The new drug's overall figure is dragged down by carrying three quarters of its patients in the hard stratum. The reversal is the case mix, not the drug.

   (c) Adjust by stratification. Stratum sizes over both drugs: mild $50+150=200$, severe $150+50=200$, so both weights are $\frac{1}{2}$.

   $$\text{New: }\frac{1}{2}(0.8)+\frac{1}{2}(0.4)=0.6,\text{Standard: }\frac{1}{2}(0.7)+\frac{1}{2}(0.3)=\mathrm{0.5.}$$

   The adjusted difference is $0.6-0.5=+0.10$: the new drug is ten points better, agreeing with both strata. The unadjusted difference was $0.50-0.60=-0.10$. Adjustment has moved the estimate by $0.20$ and flipped its sign; that $0.20$ is the size of the confounding by severity.

   (d) The assumption, and a variable that must not be adjusted for. The adjusted $0.10$ equals the average treatment effect if $T\perp (Y(0),Y(1))\mid \text{severity}$ — no unmeasured confounding: within a severity stratum, which drug a patient got is unrelated to how they would have fared under either. That is the hypothesis of the theorem Adjustment identifies the effect only under no unmeasured confounding, and the table cannot check it. If, say, doctors also gave the new drug to the sicker patients within the severe stratum, the adjusted figure still understates the new drug, and nothing in these numbers reveals it.

   A variable for which the arithmetic would be an error: “fever resolved within $48$ hours of starting treatment”. That is a consequence of the drug, on the pathway from drug to cure. Stratifying on it compares patients whose fever did resolve with each other, and those whose fever did not with each other, which removes the part of the drug's effect that operates through resolving fever; the adjusted difference would be pushed toward zero and could vanish for a drug that works. Whether to condition on a variable is a causal question — a pre-treatment confounder yes, a post-treatment consequence no — and the weighting arithmetic is the same either way, so it cannot decide.

10. 10.Critique · 9 min

    An analyst's memo on an observational study of a voluntary job-training programme reads as follows.

    Step 1. Enrolment was voluntary, but we recorded forty covariates, so we adjusted for all of them — including age, years of schooling, earnings in the two years before the programme, region, earnings in the year after the programme, and whether the participant was promoted in the following year.

    Step 2. After adjustment, participants and non-participants are balanced on every one of the forty variables, so the assumption of no unmeasured confounding has been verified.

    Step 3. The adjusted difference in earnings three years after the programme is therefore the causal effect of training.

    Step 4. It is unbiased in the same sense as a randomised trial's estimate, by the theorem that adjustment identifies the effect.

    Step 5. Should further variables become available we will add them, since adjusting for more can only reduce bias.

    1. (a)Find every error in Steps 1 to 5. For each, name the concept involved and say precisely what the error does to the estimate.
    2. (b)Rewrite the conclusion correctly: what the study can claim, under which assumption, and what would have to be true of the enrolment decision for that assumption to fail.
    3. (c)Name one design-based strategy that could replace the untestable assumption with a different one, and state the assumption it would rely on instead.
    Show the solution to question 10

    (a) The errors, step by step.

    Step 1 — adjusting for post-treatment variables. Two of the forty are consequences of the treatment. Earnings in the year after the programme lie on the causal pathway from training to earnings three years later: a mediator. Conditioning on it compares participants and non-participants who had the same early earnings, which removes the part of training's effect that works through an early earnings gain, and biases the estimate toward zero. Promotion in the following year is caused by training and by the earning-related ability that also drives later earnings: a variable caused by both treatment and outcome is a collider, and conditioning on it manufactures an association between training and earnings that does not exist, in a direction the memo cannot know. Both must be dropped. An adjustment set is a set of pre-treatment confounders, chosen from knowledge of the causal structure before the analysis.

    Step 2 — “verified”. Balance on the measured variables is produced by the adjustment itself; that is what adjusting does, so it is not evidence of anything. The assumption $T\perp (Y(0),Y(1))\mid X$ concerns the variables not in $X$, and it is untestable from the data precisely because those variables were not recorded. No balance table can verify it.

    Step 3 — “therefore”. The causal reading follows only under the assumption of Step 2, which was assumed, not established. With voluntary enrolment the obvious unmeasured confounder is motivation, which raises both the chance of enrolling and later earnings, whatever the training does; the adjusted difference then contains a selection term of unknown size.

    Step 4 — “in the same sense”. A randomised trial earns $T\perp (Y(0),Y(1))$ by construction; the theorem Adjustment identifies the effect only under no unmeasured confounding takes the conditional version as a hypothesis. One estimate is unbiased because of the design, the other only if an unverifiable assumption happens to hold. The theorem is true; the memo has used it as if its hypothesis were free.

    Step 5 — “can only reduce bias”. Step 1 already shows why this is false: adding a mediator removes part of the effect and adding a collider creates a spurious one. More adjustment is not uniformly safer; which variables to adjust for is a causal question, not a statistical one, and “everything available” is not a safe default.

    (b) A corrected conclusion. Dropping the two post-programme variables and adjusting for the pre-enrolment covariates — age, schooling, prior earnings, region and the other variables measured before enrolment — the adjusted difference estimates the average effect of training on earnings three years later under the assumption that, among people with the same values of those covariates, the decision to enrol is independent of what they would have earned with and without training. This assumption cannot be checked from the data. It fails if, among people alike on every recorded variable, those who enrolled were systematically more likely to earn more anyway (the more motivated, the better connected) or systematically less likely (the recently laid off, enrolling out of need); either direction is possible, and the sign of the resulting bias is unknown.

    (c) A design that changes the assumption. If the programme was oversubscribed and places were allocated by lottery among applicants, the lottery is a natural experiment: within the applicant pool, assignment is independent of the potential outcomes by construction, and the comparison of winners against losers is a randomised comparison. The assumption it relies on is that the lottery really was random and its outcome is what determined participation (with non-compliance, the comparison by lottery result remains valid as an intention-to-treat estimate). Alternatively, if eligibility was decided by a threshold — prior earnings below a cutoff — a regression discontinuity compares people just either side of it, under the assumption that they are comparable in everything except eligibility. Each replaces “no unmeasured confounding” with a different assumption, not with none; the gain is that these assumptions can be argued from how the allocation actually worked, rather than hoped for.

11. 11.Proof · 11 min

    An experiment applies each of two temperature settings to $k$ growth chambers, and $m$ plants are measured in every chamber. Model the height of plant $j$ in chamber $i$ as

    $$Y_{ij}=\mu +{\tau }_{g(i)}+C_i+e_{ij},$$

    where $g(i)$ is the setting applied to chamber $i$; the chamber effects $C_i$ are independent with mean $0$ and variance ${\sigma }_c^2$; the plant errors $e_{ij}$ are independent with mean $0$ and variance ${\sigma }_e^2$; and every $C$ is independent of every $e$. Write ${\sigma }^2={\sigma }_c^2+{\sigma }_e^2$ for the variance of a single plant's height and $\rho ={\sigma }_c^2/{\sigma }^2$.

    1. (a)Prove that the chamber mean ${\bar{Y}}_{i\cdot }$ has variance ${\sigma }_c^2+{\sigma }_e^2/m$, and that two plants in the same chamber have correlation $\rho$.
    2. (b)Let ${\bar{Y}}_T-{\bar{Y}}_C$ be the difference between the two settings' means, each taken over all $km$ of its plants. Prove that its true variance is $\frac{2}{k}({\sigma }_c^2+\frac{{\sigma }_e^2}{m})$, while an analyst who treats the $km$ plants as independent units would compute $\frac{2{\sigma }^2}{km}$.
    3. (c)Prove that the ratio of the true variance to the analyst's is $1+(m-1)\rho$. Deduce that the reported standard error is too small by a factor between $1$ and $\sqrt{m}$, and identify exactly when each extreme is attained.
    4. (d)With $k=3$, $m=10$, ${\sigma }_c^2=1$ and ${\sigma }_e^2=2$, find both variances and the factor. Then find the limit of the true variance as $m\to \infty$ with $k$ fixed, and use it to decide the better use of thirty extra plants per setting: three more chambers of ten, or ten more plants in each of the three existing chambers.
    Show the solution to question 11

    (a) Averaging within a chamber. Averaging the model over $j=1,\dots ,m$,

    $${\bar{Y}}_{i\cdot }=\mu +{\tau }_{g(i)}+C_i+{\bar{e}}_{i\cdot },{\bar{e}}_{i\cdot }=\frac{1}{m}\sum _{j=1}^m{e}_{ij}.$$

    The $e_{ij}$ are independent with common variance, so $\mathrm{Var}({\bar{e}}_{i\cdot })=\frac{1}{m^2}\cdot m{\sigma }_e^2=\frac{{\sigma }_e^2}{m}$. The constants have no variance, and $C_i$ is independent of ${\bar{e}}_{i\cdot }$, so the variances add: $\mathrm{Var}({\bar{Y}}_{i\cdot })={\sigma }_c^2+{\sigma }_e^2/m$.

    For two plants $j\ne j^{\prime }$ in the same chamber, $\mathrm{Cov}(Y_{ij},Y_{i{j}^{\prime }})=\mathrm{Cov}(C_i+e_{ij},C_i+e_{i{j}^{\prime }})=\mathrm{Var}(C_i)={\sigma }_c^2$, since every cross term involves two independent mean-zero variables and vanishes. Each plant has variance ${\sigma }_c^2+{\sigma }_e^2={\sigma }^2$, so the correlation is ${\sigma }_c^2/{\sigma }^2=\rho$. $■$

    (b) The true variance and the analyst's. Because every chamber holds the same number $m$ of plants, the mean over all $km$ plants at one setting equals the mean of its $k$ chamber means: ${\bar{Y}}_T=\frac{1}{k}{\sum }_{i\in T}{\bar{Y}}_{i\cdot }$. Different chambers share no $C$ and no $e$, so the chamber means are independent, and by (a)

    $$\mathrm{Var}({\bar{Y}}_T)=\frac{1}{k}({\sigma }_c^2+\frac{{\sigma }_e^2}{m}).$$

    The two settings use disjoint chambers, so ${\bar{Y}}_T$ and ${\bar{Y}}_C$ are independent and $\mathrm{Var}({\bar{Y}}_T-{\bar{Y}}_C)=\frac{2}{k}({\sigma }_c^2+{\sigma }_e^2/m)$.

    The analyst who takes the plants as $km$ independent units of variance ${\sigma }^2$ computes $\mathrm{Var}({\bar{Y}}_T)={\sigma }^2/(km)$ and hence $2{\sigma }^2/(km)$ for the difference. $■$ That is pseudoreplication: the treatment was applied to chambers, so the experimental unit is the chamber and there are $k$ of them per setting, not $km$.

    (c) The ratio. Divide:

    $$\frac{\frac{2}{k}({\sigma }_c^2+{\sigma }_e^2/m)}{2{\sigma }^2/(km)}=\frac{m{\sigma }_c^2+{\sigma }_e^2}{{\sigma }^2}=\frac{m{\sigma }_c^2+{\sigma }^2-{\sigma }_c^2}{{\sigma }^2}=1+(m-1)\frac{{\sigma }_c^2}{{\sigma }^2}=1+(m-1)\rho .$$

    Since $0\le \rho \le 1$ the ratio of variances lies in $[1,m]$, so the ratio of standard errors — true to reported — lies in $[1,\sqrt{m}]$. $■$

    The lower extreme $1$ is attained exactly when $\rho =0$, that is ${\sigma }_c^2=0$: no chamber effect, so plants really are independent units and the analyst was right. The upper extreme $\sqrt{m}$ is attained exactly when $\rho =1$, that is ${\sigma }_e^2=0$: plants in a chamber are identical, so $m$ measurements carry one unit's worth of information. The rule of thumb “too small by about $\sqrt{m}$” is the worst case; the exact factor is $\sqrt{1+(m-1)\rho }$.

    (d) Numbers, the limit, and the budget. Here ${\sigma }^2=3$ and $\rho =\frac{1}{3}$. True variance: $\frac{2}{3}(1+\frac{2}{10})=\frac{2}{3}\cdot \frac{6}{5}=\frac{12}{15}=\frac{4}{5}$. Analyst's: $\frac{2\cdot 3}{30}=\frac{1}{5}$. Ratio $4$, and $1+9\cdot \frac{1}{3}=4$ agrees. ✓ The reported standard error is too small by a factor of $2$: a reported $t=5$ is really $t=2.5$, and on $2k-2=4$ degrees of freedom rather than $58$.

    As $m\to \infty$ with $k$ fixed, ${\sigma }_e^2/m\to 0$ and the true variance tends to $2{\sigma }_c^2/k=\frac{2}{3}$, not to zero. More plants average away the plant error but never the chamber effect, which is shared by every plant in the chamber.

    Thirty extra plants per setting. Three more chambers of ten: $k=6$, $m=10$, variance $\frac{2}{6}\cdot \frac{6}{5}=\frac{2}{5}=0.4$ — halved. Ten more plants in each existing chamber: $k=3$, $m=20$, variance $\frac{2}{3}(1+\frac{2}{20})=\frac{2}{3}\cdot \frac{11}{10}=\frac{11}{15}\approx 0.73$ — down from $\frac{12}{15}$ by one part in twelve, and no number of extra plants could take it below $\frac{2}{3}$. The chambers win by a wide margin. Replication means more experimental units, and the experimental unit is whatever the treatment was applied to.

12. 12.Proof · 11 min

    In a $2^2$ factorial with $n$ replicates per cell, the four cell means ${\bar{y}}_{00},{\bar{y}}_{10},{\bar{y}}_{01},{\bar{y}}_{11}$ (first index $A$, second $B$) are independent, each with variance ${\sigma }^2/n$. The estimates are

    $$A=\frac{1}{2}[({\bar{y}}_{10}-{\bar{y}}_{00})+({\bar{y}}_{11}-{\bar{y}}_{01})],B=\frac{1}{2}[({\bar{y}}_{01}-{\bar{y}}_{00})+({\bar{y}}_{11}-{\bar{y}}_{10})],AB=\frac{1}{2}[({\bar{y}}_{11}-{\bar{y}}_{01})-({\bar{y}}_{10}-{\bar{y}}_{00})],$$

    and the grand mean is $\bar{\mu }=\frac{1}{4}({\bar{y}}_{00}+{\bar{y}}_{10}+{\bar{y}}_{01}+{\bar{y}}_{11})$.

    1. (a)Write each of $A$, $B$ and $AB$ as a contrast ${\sum }_j{c}_j{\bar{y}}_j$ over the four cells, giving the coefficient vector in the order $(00,10,01,11)$, and check that each vector sums to zero.
    2. (b)Prove that each of the three estimates has variance ${\sigma }^2/n$, and that with $N=4n$ runs in total this equals $4{\sigma }^2/N$ — the precision a one-factor-at-a-time study would get for a single factor with all $N$ runs devoted to it.
    3. (c)Prove that $A$, $B$ and $AB$ are pairwise uncorrelated, and explain what this says about “hidden replication”.
    4. (d)Prove that ${\bar{y}}_{11}=\bar{\mu }+\frac{1}{2}A+\frac{1}{2}B+\frac{1}{2}AB$, and write down, with a sign rule, the corresponding expression for each of the other three cells. Explain why this shows the design estimates exactly $2^2-1=3$ effects and no more.
    Show the solution to question 12

    (a) Coefficient vectors. Expanding the brackets,

    $$A=\frac{1}{2}(-{\bar{y}}_{00}+{\bar{y}}_{10}-{\bar{y}}_{01}+{\bar{y}}_{11}),B=\frac{1}{2}(-{\bar{y}}_{00}-{\bar{y}}_{10}+{\bar{y}}_{01}+{\bar{y}}_{11}),AB=\frac{1}{2}({\bar{y}}_{00}-{\bar{y}}_{10}-{\bar{y}}_{01}+{\bar{y}}_{11}),$$

    so in the order $(00,10,01,11)$ the vectors are

    $$c_A=(-\frac{1}{2},\frac{1}{2},-\frac{1}{2},\frac{1}{2}),c_B=(-\frac{1}{2},-\frac{1}{2},\frac{1}{2},\frac{1}{2}),c_{AB}=(\frac{1}{2},-\frac{1}{2},-\frac{1}{2},\frac{1}{2}).$$

    Each has two entries $+\frac{1}{2}$ and two $-\frac{1}{2}$, so each sums to zero. Pattern: $c_A$ is $+$ where $A$ is high, $c_B$ is $+$ where $B$ is high, and $c_{AB}$ is the entrywise product of the two. Summing to zero makes each a contrast: adding the same constant to all four cells changes none of them, which is why the overall level never enters an effect.

    (b) Variance. The cell means are independent, so for any coefficient vector

    $$\mathrm{Var}\left(\sum _j{c}_j{\bar{y}}_j\right)=\sum _j{c}_j^2\mathrm{Var}({\bar{y}}_j)=\frac{{\sigma }^2}{n}\sum _j{c}_j^2.$$

    For each of the three vectors ${\sum }_j{c}_j^2=4\cdot \frac{1}{4}=1$, so the variance is ${\sigma }^2/n$. With $N=4n$ this is ${\sigma }^2/(N/4)=4{\sigma }^2/N$. $■$ A one-factor-at-a-time study with $N/2$ runs at each of two conditions estimates its one effect as a difference of two means, with variance $\frac{{\sigma }^2}{N/2}+\frac{{\sigma }^2}{N/2}=\frac{4{\sigma }^2}{N}$ — the same. This is the theorem Factorial designs estimate more from the same runs: the factorial reaches that precision for all three effects at once.

    (c) Orthogonality. For two coefficient vectors $c$ and $d$, bilinearity of covariance and independence of the cells give

    $$\mathrm{Cov}(\sum _j{c}_j{\bar{y}}_j,\sum _l{d}_l{\bar{y}}_l)=\sum _j\sum _l{c}_j{d}_l\mathrm{Cov}({\bar{y}}_j,{\bar{y}}_l)=\frac{{\sigma }^2}{n}\sum _j{c}_j{d}_j,$$

    since every cross-cell covariance is zero. The three inner products, in the order $(00,10,01,11)$:

    $$c_A\cdot c_B=\frac{1}{4}-\frac{1}{4}-\frac{1}{4}+\frac{1}{4}=0,c_A\cdot c_{AB}=-\frac{1}{4}-\frac{1}{4}+\frac{1}{4}+\frac{1}{4}=0,c_B\cdot c_{AB}=-\frac{1}{4}+\frac{1}{4}-\frac{1}{4}+\frac{1}{4}=0.$$

    All three covariances vanish. $■$

    This is hidden replication made precise. No coefficient in any vector is zero, so every run enters every estimate — nothing is set aside for $B$ that is then unavailable for $A$. And yet the estimates do not compete for information: they are orthogonal, so estimating $B$ costs $A$ nothing in precision, and estimating $AB$ costs neither. Each effect is estimated exactly as well as if it were the only thing the experiment was for.

    (d) Recovering the cells. Write out $\bar{\mu }+\frac{1}{2}A+\frac{1}{2}B+\frac{1}{2}AB$ and collect the coefficient of each cell mean; every term contributes $\pm \frac{1}{4}$.

    ${\bar{y}}_{11}$: $\frac{1}{4}+\frac{1}{4}+\frac{1}{4}+\frac{1}{4}=1$. ${\bar{y}}_{00}$: $\frac{1}{4}-\frac{1}{4}-\frac{1}{4}+\frac{1}{4}=0$. ${\bar{y}}_{10}$: $\frac{1}{4}+\frac{1}{4}-\frac{1}{4}-\frac{1}{4}=0$. ${\bar{y}}_{01}$: $\frac{1}{4}-\frac{1}{4}+\frac{1}{4}-\frac{1}{4}=0$.

    So the expression equals ${\bar{y}}_{11}$. $■$ The sign rule: give each factor the sign $s=+1$ at its high level and $s=-1$ at its low level; then

    $${\bar{y}}_{ab}=\bar{\mu }+\frac{1}{2}{s}_{a}A+\frac{1}{2}{s}_{b}B+\frac{1}{2}{s}_a{s}_{b}AB,$$

    the interaction carrying the product of the signs. Explicitly ${\bar{y}}_{10}=\bar{\mu }+\frac{1}{2}A-\frac{1}{2}B-\frac{1}{2}AB$, ${\bar{y}}_{01}=\bar{\mu }-\frac{1}{2}A+\frac{1}{2}B-\frac{1}{2}AB$, and ${\bar{y}}_{00}=\bar{\mu }-\frac{1}{2}A-\frac{1}{2}B+\frac{1}{2}AB$; each checks by the same coefficient count (for ${\bar{y}}_{00}$ the four contributions to its own coefficient are $\frac{1}{4}+\frac{1}{4}+\frac{1}{4}+\frac{1}{4}$, and every other cell's cancel in pairs).

    Sanity check with cell means $40,50,46,50$: $\bar{\mu }=46.5$, $A=7$, $B=3$, $AB=-3$, and ${\bar{y}}_{11}=46.5+3.5+1.5-1.5=50$, ${\bar{y}}_{00}=46.5-3.5-1.5-1.5=40$. ✓

    Why exactly three effects: the map from the four cell means to $(\bar{\mu },A,B,AB)$ is invertible — part (d) is its inverse — so the four numbers on each side carry the same information. The four coefficient vectors, $(\frac{1}{4},\frac{1}{4},\frac{1}{4},\frac{1}{4})$ and the three contrasts, are mutually orthogonal and so form a basis of the four-dimensional space of cell-mean combinations. One dimension is the overall level; the remaining $2^2-1=3$ are the effects. Any further contrast is a linear combination of $c_A$, $c_B$ and $c_{AB}$, so a “fourth effect” would be a relabelling of these three, not new information. This is the proposition How many effects a two-level factorial estimates in the case $k=2$, with the dimension count made explicit.