Print this page and work through it on paper. Revealed answers are never printed, so you can check one question and still print a clean sheet.

1. 1.Computation · 7 min

   A population consists of the four values $\{2,4,6,8\}$, each equally likely to be drawn. Samples of size $n=2$ are drawn with replacement, so the two draws $X_1,X_2$ are i.i.d.

   1. (a)Classify each of the following as a parameter, a statistic, or neither, with one sentence of justification each: (i) $\mu$; (ii) $\bar{X}$; (iii) $\frac{1}{n}{\sum }_{i=1}^n(X_i-\mu {)}^2$; (iv) $\max (X_1,X_2)$; (v) $\sigma /\sqrt{n}$.
   2. (b)Compute $\mu$ and ${\sigma }^2$, then list all $16$ ordered samples and tabulate the sampling distribution of $\bar{X}$: every value it can take, with its probability.
   3. (c)From your table compute $E(\bar{X})$ and $\mathrm{Var}(\bar{X})$ directly, and confirm that they equal $\mu$ and ${\sigma }^2/n$.
   Show the solution to question 1

   (a) Parameter, statistic, or neither. (i) $\mu =E(X)$ is a fixed numerical feature of the population: a parameter. It does not change from sample to sample and is not known from the data.

   (ii) $\bar{X}=\frac{1}{2}(X_1+X_2)$ can be computed from the data alone: a statistic. It is also a random variable — it takes a different value on a different sample.

   (iii) Neither. It is not a parameter, because it depends on which sample was drawn; and it is not a statistic, because evaluating it needs $\mu$, which the data do not supply. (Replacing $\mu$ by $\bar{X}$ turns it into a statistic.)

   (iv) The sample maximum is a function of the observations alone: a statistic.

   (v) $\sigma /\sqrt{n}$ is a fixed constant determined by the population (and the chosen $n$); it is not random and cannot be computed from a sample. It is a parameter — specifically the standard error of $\bar{X}$ — and the statistic that estimates it is $S/\sqrt{n}$.

   (b) The population and the sampling distribution. The population mean is

   $$\mu =\frac{2+4+6+8}{4}=5,$$

   and the population variance, dividing by $N=4$ because this is the whole population and not a sample, is

   $${\sigma }^2=\frac{(-3{)}^2+(-1{)}^2+1^2+3^2}{4}=\frac{9+1+1+9}{4}=\frac{20}{4}=5.$$

   The $16$ ordered samples $(a,b)$ are equally likely, each with probability $\frac{1}{16}$, and $\bar{x}=\frac{a+b}{2}$. Arranging them in a grid with $a$ down the rows and $b$ across the columns, the sample means are

   $$\begin{array}{ccccc}& 2& 4& 6& 8\\ 2& 2& 3& 4& 5\\ 4& 3& 4& 5& 6\\ 6& 4& 5& 6& 7\\ 8& 5& 6& 7& 8\end{array}$$

   Counting along the anti-diagonals, the sampling distribution of $\bar{X}$ is

   $$\begin{array}{cccccccc}\bar{x}& 2& 3& 4& 5& 6& 7& 8\\ P(\bar{X}=\bar{x})& \frac{1}{16}& \frac{2}{16}& \frac{3}{16}& \frac{4}{16}& \frac{3}{16}& \frac{2}{16}& \frac{1}{16}\end{array}$$

   — a triangle centred on $5$. The probabilities sum to $\frac{16}{16}=1$.

   (c) Mean and variance from the table.

   $$E(\bar{X})=\frac{2\cdot 1+3\cdot 2+4\cdot 3+5\cdot 4+6\cdot 3+7\cdot 2+8\cdot 1}{16}=\frac{2+6+12+20+18+14+8}{16}=\frac{80}{16}=5=\mu .$$

   For the variance, the deviations $\bar{x}-5$ are $-3,-2,-1,0,1,2,3$ with frequencies $1,2,3,4,3,2,1$, so

   $$\mathrm{Var}(\bar{X})=\frac{1\cdot 9+2\cdot 4+3\cdot 1+4\cdot 0+3\cdot 1+2\cdot 4+1\cdot 9}{16}=\frac{40}{16}=\frac{5}{2}=\frac{{\sigma }^2}{2}=\frac{{\sigma }^2}{n}.$$

   Both theorems check. Note that the averaging halved the variance ($5$ to $\frac{5}{2}$) — this is the whole reason for taking a mean — and that the independence of the two draws is what licensed the ${\sigma }^2/n$; the equality $E(\bar{X})=\mu$ would have held even for dependent draws.

2. 2.Computation · 7 min

   Keep the population $\{2,4,6,8\}$, with $\mu =5$ and ${\sigma }^2=5$, and the $16$ equally likely ordered samples $(a,b)$ of size $n=2$ drawn with replacement. Two candidate estimators of ${\sigma }^2$ are the sample variance $S^2=\frac{1}{n-1}\sum (X_i-\bar{X}{)}^2$ and the divide-by-$n$ version ${\hat{\sigma }}^2=\frac{1}{n}\sum (X_i-\bar{X}{)}^2$.

   1. (a)Show that for a sample $(a,b)$ of size $2$, $S^2=\frac{(a-b{)}^2}{2}$ and ${\hat{\sigma }}^2=\frac{(a-b{)}^2}{4}$.
   2. (b)Tabulate $(a-b{)}^2$ over the $16$ samples, and hence compute $E(S^2)$ and $E({\hat{\sigma }}^2)$ exactly. Which estimator is unbiased, and by what factor is the other one off?
   3. (c)Compute $E(S)$ exactly and compare it with $\sigma =\sqrt{5}$. State the general fact this illustrates.
   Show the solution to question 2

   (a) The two formulas. With $\bar{x}=\frac{a+b}{2}$ the deviations are

   $$a-\bar{x}=\frac{a-b}{2},b-\bar{x}=\frac{b-a}{2},$$

   equal and opposite, as deviations from the sample mean must be (they always sum to zero). Each squares to $\frac{(a-b{)}^2}{4}$, so the sum of squared deviations is $\frac{(a-b{)}^2}{2}$. Dividing by $n-1=1$ gives $S^2=\frac{(a-b{)}^2}{2}$; dividing by $n=2$ gives ${\hat{\sigma }}^2=\frac{(a-b{)}^2}{4}$.

   (b) The expectations. Over the $16$ ordered samples, $|a-b|$ takes the value $0$ on the $4$ diagonal samples $(2,2),(4,4),(6,6),(8,8)$; the value $2$ on the $6$ samples $(2,4),(4,2),(4,6),(6,4),(6,8),(8,6)$; the value $4$ on the $4$ samples $(2,6),(6,2),(4,8),(8,4)$; and the value $6$ on the $2$ samples $(2,8),(8,2)$. Check: $4+6+4+2=16$. Hence

   $$\sum _{\text{samples}}(a-b{)}^2=4\cdot 0+6\cdot 4+4\cdot 16+2\cdot 36=0+24+64+72=160,E\left[(a-b{)}^2\right]=\frac{160}{16}=10.$$

   (Cross-check without the table: $a-b$ has mean $0$ and, being a difference of independent draws, variance ${\sigma }^2+{\sigma }^2=10$, so $E[(a-b{)}^2]=\mathrm{Var}(a-b)=10$. The table agrees.)

   Therefore

   $$E(S^2)=\frac{10}{2}=5={\sigma }^2,E({\hat{\sigma }}^2)=\frac{10}{4}=\frac{5}{2}.$$

   $S^2$ is unbiased. The divide-by-$n$ estimator is biased downward by the factor $\frac{n-1}{n}=\frac{1}{2}$: on average it reports half the true variance. The reason is visible in the diagonal samples, where both observations coincide with $\bar{x}$ and the estimator reports $0$; deviations from the sample's own centre are systematically too small.

   (c) $E(S)$. Since $S=\sqrt{S^2}=\frac{|a-b|}{\sqrt{2}}$,

   $$E|a-b|=\frac{4\cdot 0+6\cdot 2+4\cdot 4+2\cdot 6}{16}=\frac{12+16+12}{16}=\frac{40}{16}=\frac{5}{2},E(S)=\frac{5}{2\sqrt{2}}=\frac{5\sqrt{2}}{4}.$$

   Compare with $\sigma =\sqrt{5}$ by squaring both (both are positive): ${\left(\frac{5\sqrt{2}}{4}\right)}^2=\frac{50}{16}=\frac{25}{8}=3.125<5$. So $E(S)<\sigma$ — roughly $1.77$ against $2.24$.

   The general fact: $S^2$ is unbiased for ${\sigma }^2$, but $S$ is not unbiased for $\sigma$. Unbiasedness is not preserved by taking square roots, because the square root is strictly concave and $E(\sqrt{S^2})<\sqrt{E(S^2)}$ whenever $S^2$ is not constant. The sample standard deviation underestimates $\sigma$ on average.

3. 3.Computation · 7 min

   A town's $2000$ workers split into two strata of $1000$ each. Those living in the town have a daily commute with mean $12$ km and standard deviation $6$ km; those living outside have mean $28$ km and standard deviation $6$ km. A sample of $n=100$ workers is to be asked their commute distance.

   1. (a)Compute the population mean $\mu$, the within-strata and between-strata components of the population variance, and ${\sigma }^2$ itself.
   2. (b)Find the standard error of $\bar{X}$ under simple random sampling, and under stratified sampling with proportional allocation. State the stratum sample sizes.
   3. (c)How many simple-random-sample interviews would be needed to match the stratified standard error? Then say what would happen to the gain from stratifying if the two strata had the same mean commute, and why.
   Show the solution to question 3

   (a) Decompose the variance. The stratum weights are $W_1=W_2=\frac{1000}{2000}=\frac{1}{2}$, so

   $$\mu =\frac{1}{2}(12)+\frac{1}{2}(28)=20.$$

   The within-strata component is ${\sum }_h{W}_h{\sigma }_h^2=\frac{1}{2}(36)+\frac{1}{2}(36)=36$. The between-strata component is

   $$\sum _h{W}_h({\mu }_h-\mu {)}^2=\frac{1}{2}(12-20{)}^2+\frac{1}{2}(28-20{)}^2=\frac{1}{2}(64)+\frac{1}{2}(64)=64.$$

   By the law of total variance applied to the stratum label, ${\sigma }^2=36+64=100$, so $\sigma =10$. Almost two-thirds of the population variance is between the groups.

   (b) The two standard errors. Under simple random sampling (treating $n/N=0.05$ as small enough to ignore the finite population correction),

   $$\mathrm{Var}({\bar{X}}_{\text{SRS}})=\frac{{\sigma }^2}{n}=\frac{100}{100}=1,SE=1\text{ km}.$$

   Proportional allocation takes $n_h=n{W}_h=100\cdot \frac{1}{2}=50$ from each stratum. The strata are sampled independently, so

   $$\mathrm{Var}({\bar{X}}_{\text{str}})=\sum _h{W}_h^2\frac{{\sigma }_h^2}{n_h}=\frac{1}{4}\cdot \frac{36}{50}+\frac{1}{4}\cdot \frac{36}{50}=\frac{36}{100}=0.36,$$

   which is exactly $\frac{1}{n}{\sum }_h{W}_h{\sigma }_h^2=\frac{36}{100}$, the within-strata variance divided by $n$. Hence $SE=\sqrt{0.36}=0.6$ km. Stratifying cut the standard error from $1$ to $0.6$ for the same hundred interviews.

   (c) The price in interviews, and the source of the gain. An SRS matches $0.36$ when $\frac{100}{n}\le 0.36$, i.e. $n\ge \frac{100}{0.36}=\frac{2500}{9}\approx 277.8$, so $278$ interviews — nearly three times as many.

   The difference between the two variances is exactly the between-strata term divided by $n$: $1-0.36=0.64=\frac{64}{100}$. If both strata had mean $20$, the between-strata component would be $0$, ${\sigma }^2$ would be $36$, and both designs would give $\mathrm{Var}(\bar{X})=\frac{36}{100}$. Stratification would then buy nothing. Stratification deletes the between-strata variance and only that, so the variable you stratify on must be one that separates the stratum means; strata that differ only in spread are worthless.

4. 4.Critique · 7 min

   A city council wants to know what proportion of its residents oppose a new road toll. It posts a survey on its website; $50,000$ residents respond, of whom $39,000$ oppose. The council's report reads:

   Step 1. The sample proportion is $\hat{p}=39,000/50,000=0.78$.

   Step 2. With $n=50,000$ the standard error is $\sqrt{0.78\times 0.22/50,000}\approx 0.0019$, so the $95\%$ margin of error is about $\pm 0.4\%$.

   Step 3. Therefore between $77.6\%$ and $78.4\%$ of all residents oppose the toll. A sample this large has essentially no sampling error, so the figure is almost exact.

   The arithmetic in Steps 1 and 2 is correct.

   1. (a)Identify what is wrong, naming the kinds of sampling bias involved. Say precisely which claim in Step 3 is false, and which phrase in it is doing the damage.
   2. (b)A councillor proposes keeping the survey open until $200,000$ responses arrive. Using the way bias and sampling variability each depend on $n$, explain why this does not help.
   3. (c)Describe a design under which Steps 2 and 3 would be legitimate, and compute the $95\%$ margin of error it would deliver with $n=400$ and the conservative planning value $p^{\ast }=0.5$ (take $z^{\ast }=1.96$). Name the one source of bias that even this design must still plan for.
   Show the solution to question 4

   (a) Where the argument fails. Two design failures, neither of them arithmetical.

   First, undercoverage: the sampling frame is "people who visit the council website", not "residents of the city". Residents who never visit the site cannot be selected at all.

   Second, voluntary response bias: nobody selected these $50,000$ — they selected themselves, and people who feel strongly (here, drivers who would pay the toll) are far more likely to bother. The sample is also a convenience sample from the council's point of view.

   Both failures shift $E(\hat{p})$ away from $p$: the survey estimates, quite precisely, the proportion of self-selected website visitors who oppose the toll, which is a different parameter from the one the council wants.

   Step 2's number is the standard error $\hat{p}$ would have if the $50,000$ were an i.i.d. Bernoulli$(p)$ sample. The formula's hypothesis fails, so the number describes nothing.

   The false claim in Step 3 is that the interval $(0.776,0.784)$ contains the population proportion. The phrase doing the damage is "no sampling error". That phrase is nearly true — sampling variability really is tiny at $n=50,000$ — but variability is not the error here. The error is bias, which the margin of error does not measure at all. The report has a very narrow interval around the wrong centre: a precise answer to the wrong question.

   (b) Why more responses cannot help. Sampling variability shrinks like $1/\sqrt{n}$: quadrupling $n$ halves the standard error, from about $0.0019$ to about $0.0009$. Bias is $E(\hat{p})-p$, and it does not depend on $n$ at all: the extra $150,000$ self-selected responses come from the same distorted frame with the same self-selection, so they estimate the same wrong quantity. The councillor would obtain a still narrower interval around the same wrong centre. (This is the Literary Digest mistake: $2.4$ million responses, off by nearly twenty points.)

   (c) A legitimate design. Draw a simple random sample from a complete frame of residents — the electoral roll or the register of residential addresses — with the council choosing the respondents, each with equal probability, rather than the respondents choosing themselves. (Stratifying by district would do even better.) Then the responses are i.i.d. Bernoulli$(p)$ and the formulas of Steps 2 and 3 apply.

   With $n=400$ and $p^{\ast }=0.5$,

   $$SE=\sqrt{\frac{0.5\times 0.5}{400}}=\sqrt{\frac{1}{1600}}=\frac{1}{40}=0.025,ME=1.96\times 0.025=0.049,$$

   about $\pm 5$ points. Four hundred properly chosen residents give an honest $\pm 5\%$; fifty thousand self-selected ones give a meaningless $\pm 0.4\%$.

   The remaining hazard is nonresponse bias: some of the $400$ selected residents will decline, and if decliners differ from responders the estimate shifts again. Plan for it in advance — a follow-up protocol on a random subsample of the non-responders — rather than hoping it away.

5. 5.Proof · 9 min

   Let $X_1,\dots ,X_n$ each have mean $\mu$ and finite variance ${\sigma }^2>0$, and let $\bar{X}=\frac{1}{n}{\sum }_{i=1}^n{X}_i$. You may use linearity of expectation and the expansion $\mathrm{Var}(aX+bY)=a^2\mathrm{Var}(X)+b^2\mathrm{Var}(Y)+2ab\mathrm{Cov}(X,Y)$.

   1. (a)Assuming the $X_i$ are independent, prove that $E(\bar{X})=\mu$ and $\mathrm{Var}(\bar{X})={\sigma }^2/n$. Mark the one step in which independence is used, and say what the argument gives without it.
   2. (b)Now drop independence and suppose instead that $\mathrm{Cov}(X_i,X_j)=\rho {\sigma }^2$ for every pair $i\ne j$ — the situation for units drawn from a single cluster. Prove that $$\mathrm{Var}(\bar{X})=\frac{{\sigma }^2}{n}(1+(n-1)\rho ).$$
   3. (c)Deduce from (b) that necessarily $\rho \ge -\frac{1}{n-1}$, and that when $\rho >0$ the variance of $\bar{X}$ does not tend to $0$ as $n\to \infty$. Say what this means for someone who quotes $\sigma /\sqrt{n}$ as the standard error of a clustered sample.
   Show the solution to question 5

   (a) Mean and variance of $\bar{X}$. The mean. By linearity of expectation, pulling the constant $\frac{1}{n}$ out of the sum,

   $$E(\bar{X})=\frac{1}{n}\sum _{i=1}^{n}E(X_i)=\frac{1}{n}\cdot n\mu =\mu .$$

   Only the fact that every $X_i$ has mean $\mu$ was used; independence plays no role.

   The variance. Write $D_i=X_i-\mu$, so $E(D_i)=0$, $E(D_i^2)={\sigma }^2$ and $E(D_i{D}_j)=\mathrm{Cov}(X_i,X_j)$. Then $\sum X_i-n\mu =\sum D_i$, and expanding the square of the sum (which is the two-variable expansion applied term by term),

   $$\mathrm{Var}\left(\sum _{i=1}^n{X}_i\right)=E\left[(\sum _i{D}_i{)}^2\right]=\sum _{i}E(D_i^2)+\sum _{i\ne j}E(D_i{D}_j)=n{\sigma }^2+\sum _{i\ne j}\mathrm{Cov}(X_i,X_j).$$

   Independence is used here, and only here: independent variables have $E(X_i{X}_j)=E(X_i)E(X_j)$, so every covariance is $0$ and the double sum vanishes, leaving $\mathrm{Var}(\sum X_i)=n{\sigma }^2$. Finally, scaling by the constant $\frac{1}{n}$ multiplies a variance by $\frac{1}{n^2}$:

   $$\mathrm{Var}(\bar{X})=\frac{1}{n^2}\cdot n{\sigma }^2=\frac{{\sigma }^2}{n}.■$$

   Without independence the mean result survives intact, and the variance becomes $\frac{{\sigma }^2}{n}+\frac{1}{n^2}{\sum }_{i\ne j}\mathrm{Cov}(X_i,X_j)$ — the formula ${\sigma }^2/n$ is exactly the uncorrelated case.

   (b) Equicorrelated observations. Return to the general expansion of (a). There are $n(n-1)$ ordered pairs $(i,j)$ with $i\ne j$, and each contributes $\rho {\sigma }^2$, so

   $$\mathrm{Var}\left(\sum _{i=1}^n{X}_i\right)=n{\sigma }^2+n(n-1)\rho {\sigma }^2=n{\sigma }^2(1+(n-1)\rho ).$$

   Dividing by $n^2$,

   $$\mathrm{Var}(\bar{X})=\frac{{\sigma }^2}{n}(1+(n-1)\rho ).■$$

   Sanity checks: $\rho =0$ recovers ${\sigma }^2/n$; $\rho =1$ (all observations identical) gives ${\sigma }^2$, the variance of a single observation, which is right because $n$ copies of one number carry one number's worth of information.

   (c) Consequences. A variance is never negative and ${\sigma }^2/n>0$, so (b) forces $1+(n-1)\rho \ge 0$, that is $\rho \ge -\frac{1}{n-1}$. Observations cannot be strongly negatively correlated with all of each other at once; for large $n$ the common correlation is essentially non-negative.

   For $\rho >0$, split the formula:

   $$\mathrm{Var}(\bar{X})=\frac{{\sigma }^2}{n}+\rho {\sigma }^2\cdot \frac{n-1}{n}⟶0+\rho {\sigma }^2=\rho {\sigma }^2>0(n\to \infty ).$$

   The standard error has a floor of $\sigma \sqrt{\rho }$ that no amount of further sampling from the same cluster can breach: once the shared component of the cluster has been learned, extra units only repeat it.

   Someone quoting $\sigma /\sqrt{n}$ for such data understates the true standard error by the factor $\sqrt{1+(n-1)\rho }$. With $\rho =0.1$ and $n=100$ that factor is $\sqrt{10.9}>3$: the reported uncertainty is less than a third of the real one, and the confidence intervals built on it are far too narrow. The correct hypothesis for ${\sigma }^2/n$ is uncorrelated observations, and a cluster design breaks exactly that.

6. 6.Computation · 9 min

   A population of measurements has standard deviation $\sigma =12$. Give every standard error exactly.

   1. (a)For a sample of $n=36$ i.i.d. observations, find $SE(\bar{X})$. Then find the smallest $n$ for which $SE(\bar{X})\le 0.5$, and say by what factor the sample had to grow to cut the standard error by a factor of $4$.
   2. (b)Now suppose the population is finite, with $N=81$ units, and the $n=36$ observations are drawn without replacement. Find $SE(\bar{X})$. Is the usual convention for ignoring the correction applicable here? What happens to $SE(\bar{X})$ when $n=N$, and why is that the right answer?
   3. (c)Verify the without-replacement formula by direct enumeration: from the population $\{2,4,6,8\}$ (so $\mu =5$ and ${\sigma }^2=5$) list the samples of size $2$ drawn without replacement, compute $\mathrm{Var}(\bar{X})$ from the list, and compare it with what the formula predicts.
   Show the solution to question 6

   (a) The square-root law. For i.i.d. observations $SE(\bar{X})=\sigma /\sqrt{n}$, so at $n=36$

   $$SE=\frac{12}{\sqrt{36}}=\frac{12}{6}=2.$$

   For $SE\le \frac{1}{2}$ we need $\frac{12}{\sqrt{n}}\le \frac{1}{2}$, i.e. $\sqrt{n}\ge 24$, i.e. $n\ge 576$. The smallest such $n$ is $576$. The standard error fell by a factor of $4$ (from $2$ to $\frac{1}{2}$) and the sample grew by a factor of $\frac{576}{36}=16=4^2$: dividing the standard error by $k$ costs a factor $k^2$ in sample size, because $n$ sits under a square root.

   (b) Finite population correction. Without replacement from $N=81$,

   $$\mathrm{Var}(\bar{X})=\frac{{\sigma }^2}{n}\cdot \frac{N-n}{N-1}=\frac{144}{36}\cdot \frac{81-36}{81-1}=4\cdot \frac{45}{80}=4\cdot \frac{9}{16}=\frac{9}{4},SE=\frac{3}{2}.$$

   The convention is to ignore the correction when $n/N\le 0.05$. Here $n/N=\frac{36}{81}=\frac{4}{9}\approx 0.44$, so it is not applicable: ignoring it would report $2$ instead of $1.5$, overstating the uncertainty by a third. Sampling without replacement is more precise than the i.i.d. formula says, because later draws cannot repeat information already collected.

   At $n=N$ the factor $\frac{N-n}{N-1}$ is $0$, so $SE=0$. That is correct: a sample of the whole population is a census, $\bar{X}$ equals $\mu$ identically, and there is no sampling variability left to measure.

   (c) Enumeration check. Drawing $2$ of the $4$ values without replacement, the $6$ unordered samples are equally likely (each ordered sample has probability $\frac{1}{4}\cdot \frac{1}{3}=\frac{1}{12}$, and each unordered sample corresponds to two of them). Their means:

   $$\{2,4\}\to 3,\{2,6\}\to 4,\{2,8\}\to 5,\{4,6\}\to 5,\{4,8\}\to 6,\{6,8\}\to 7.$$

   The mean of the six values is $\frac{3+4+5+5+6+7}{6}=\frac{30}{6}=5=\mu$, so $\bar{X}$ is still unbiased. The deviations from $5$ are $-2,-1,0,0,1,2$, with squares $4,1,0,0,1,4$ summing to $10$, so

   $$\mathrm{Var}(\bar{X})=\frac{10}{6}=\frac{5}{3}.$$

   The formula predicts

   $$\frac{{\sigma }^2}{n}\cdot \frac{N-n}{N-1}=\frac{5}{2}\cdot \frac{4-2}{4-1}=\frac{5}{2}\cdot \frac{2}{3}=\frac{5}{3},$$

   in exact agreement. With replacement the same population gives $\mathrm{Var}(\bar{X})=\frac{5}{2}$; forbidding repeats removed the two extreme samples $(2,2)$ and $(8,8)$ and their kin, and the variance dropped by the factor $\frac{2}{3}$.

7. 7.Proof · 10 min

   Let $X_1,\dots ,X_n$ be i.i.d. with mean $\mu$ and finite variance ${\sigma }^2$, with $n\ge 2$, and let $S^2=\frac{1}{n-1}{\sum }_{i=1}^n(X_i-\bar{X}{)}^2$. You may quote $E(\bar{X})=\mu$ and $\mathrm{Var}(\bar{X})={\sigma }^2/n$.

   1. (a)Prove the identity $$\sum _{i=1}^n(X_i-\bar{X}{)}^2=\sum _{i=1}^n(X_i-c{)}^2-n(\bar{X}-c{)}^2$$ for every real constant $c$, and deduce that $\bar{X}$ is the value of $c$ that minimises ${\sum }_i(X_i-c{)}^2$.
   2. (b)Take $c=\mu$ and prove that $E(S^2)={\sigma }^2$. Then find $E({\hat{\sigma }}^2)$ for ${\hat{\sigma }}^2=\frac{1}{n}{\sum }_i(X_i-\bar{X}{)}^2$, and say in one sentence where the missing piece of ${\sigma }^2$ went.
   3. (c)Prove that $E(S)\le \sigma$, with equality only when $S$ is constant with probability $1$. Use nothing beyond (b) and the fact that a variance is never negative.
   Show the solution to question 7

   (a) The identity. Insert and remove $c$ inside the square: $X_i-\bar{X}=(X_i-c)-(\bar{X}-c)$. Squaring and summing over $i$,

   $$\sum _{i=1}^n(X_i-\bar{X}{)}^2=\sum _{i=1}^n(X_i-c{)}^2-2(\bar{X}-c)\sum _{i=1}^n(X_i-c)+n(\bar{X}-c{)}^2,$$

   where $(\bar{X}-c)$ came out of the middle sum because it does not depend on $i$. Now ${\sum }_i(X_i-c)={\sum }_i{X}_i-nc=n\bar{X}-nc=n(\bar{X}-c)$, so the middle term is $-2n(\bar{X}-c{)}^2$, and

   $$\sum _{i=1}^n(X_i-\bar{X}{)}^2=\sum _{i=1}^n(X_i-c{)}^2-2n(\bar{X}-c{)}^2+n(\bar{X}-c{)}^2=\sum _{i=1}^n(X_i-c{)}^2-n(\bar{X}-c{)}^2.■$$

   Rearranged, ${\sum }_i(X_i-c{)}^2={\sum }_i(X_i-\bar{X}{)}^2+n(\bar{X}-c{)}^2\ge {\sum }_i(X_i-\bar{X}{)}^2$, with equality exactly when $c=\bar{X}$. So the sample mean is the constant about which the sum of squared deviations is smallest — which is why deviations measured from $\bar{X}$ are systematically smaller than deviations measured from $\mu$.

   (b) Unbiasedness of $S^2$. Put $c=\mu$ in the identity and take expectations of both sides, using linearity:

   $$E\left[\sum _{i=1}^n(X_i-\bar{X}{)}^2\right]=\sum _{i=1}^{n}E\left[(X_i-\mu {)}^2\right]-nE\left[(\bar{X}-\mu {)}^2\right].$$

   Each $E[(X_i-\mu {)}^2]=\mathrm{Var}(X_i)={\sigma }^2$, since every $X_i$ has the population distribution; that gives $n{\sigma }^2$. And because $E(\bar{X})=\mu$, the quantity $E[(\bar{X}-\mu {)}^2]$ is $\mathrm{Var}(\bar{X})={\sigma }^2/n$ — this is where independence enters, through the variance formula. Hence

   $$E\left[\sum _{i=1}^n(X_i-\bar{X}{)}^2\right]=n{\sigma }^2-n\cdot \frac{{\sigma }^2}{n}=(n-1){\sigma }^2,E(S^2)=\frac{(n-1){\sigma }^2}{n-1}={\sigma }^2.■$$

   For the divide-by-$n$ estimator,

   $$E({\hat{\sigma }}^2)=\frac{(n-1){\sigma }^2}{n}={\sigma }^2-\frac{{\sigma }^2}{n},$$

   biased low by exactly ${\sigma }^2/n$. The missing piece is $\mathrm{Var}(\bar{X})$: it is the expected squared error incurred by measuring deviations from $\bar{X}$ instead of from $\mu$ — one observation's worth of variance, spent on estimating the mean.

   (c) $S$ is biased for $\sigma$. First, $E(S)$ is finite: $S\ge 0$ and $s\le 1+s^2$ for every $s\ge 0$, so $E(S)\le 1+E(S^2)=1+{\sigma }^2<\infty$. So $S$ has a well-defined variance, and by (b),

   $$0\le \mathrm{Var}(S)=E(S^2)-[E(S){]}^2={\sigma }^2-[E(S){]}^2.$$

   Hence $[E(S){]}^2\le {\sigma }^2$, and since $E(S)\ge 0$ and $\sigma \ge 0$, taking square roots gives $E(S)\le \sigma$. Equality holds if and only if $\mathrm{Var}(S)=0$, which is if and only if $S$ equals a constant with probability $1$. $■$

   For any population in which $S$ actually varies — which is every population that takes more than one value — the inequality is strict: the sample standard deviation underestimates $\sigma$ on average, even though its square is unbiased. Nothing about concavity or Jensen's inequality was needed; the non-negativity of a variance already contains the whole fact.

8. 8.Computation · 8 min

   Markov's inequality: for $Y\ge 0$ with finite mean and any $a>0$, $P(Y\ge a)\le E(Y)/a$. Chebyshev's inequality: for $X$ with mean $\mu$ and variance ${\sigma }^2$ and any $k>0$, $P(|X-\mu |\ge k)\le {\sigma }^2/k^2$. Both hold for every distribution; this question is about how tight they are.

   1. (a)A non-negative random variable $Y$ has $E(Y)=4$. Bound $P(Y\ge 20)$ from above. Then construct a distribution for $Y$ with $E(Y)=4$ for which the bound is attained exactly, and say what that shows about improving Markov's inequality.
   2. (b)Observations are i.i.d. with unknown shape, unknown mean $\mu$ and known $\sigma =1$. Using only Chebyshev's inequality, find the smallest $n$ that guarantees $P(|{\bar{X}}_n-\mu |\ge 0.25)\le 0.01$. Say which two earlier results feed into the bound you use.
   3. (c)Construct a distribution with mean $\mu$ and standard deviation $\sigma$ for which $P(|X-\mu |\ge 2\sigma )$ equals $\frac{1}{4}$ exactly, and verify both moments. For a normal distribution the same probability is about $0.046$; explain why Chebyshev must be loose for it.
   Show the solution to question 8

   (a) Markov, and a case of equality. Markov's inequality with $a=20$ gives

   $$P(Y\ge 20)\le \frac{E(Y)}{20}=\frac{4}{20}=\frac{1}{5}.$$

   To attain it, let $Y$ take only the values $0$ and $20$, with $P(Y=20)=\frac{1}{5}$ and $P(Y=0)=\frac{4}{5}$. Then $E(Y)=20\cdot \frac{1}{5}+0\cdot \frac{4}{5}=4$ as required, and $P(Y\ge 20)=\frac{1}{5}$ — equality.

   Why this is the equality case: Markov's proof rests on the pointwise inequality $Y\ge a1\{Y\ge a\}$, and that is an equality precisely when $Y$ is $0$ whenever it is below $a$ and exactly $a$ whenever it is at or above $a$. Since a distribution with $E(Y)=4$ achieves the bound, no inequality of the form $P(Y\ge a)\le C\cdot E(Y)/a$ with $C<1$ can hold for all non-negative $Y$: Markov's constant cannot be improved without adding hypotheses.

   (b) Chebyshev applied to the sample mean. Two earlier results are needed. First, the theorem on the mean and variance of the sample mean: for i.i.d. observations $E({\bar{X}}_n)=\mu$ and $\mathrm{Var}({\bar{X}}_n)={\sigma }^2/n=1/n$. Second, Chebyshev's inequality applied to the random variable ${\bar{X}}_n$ with $k=0.25$:

   $$P(|{\bar{X}}_n-\mu |\ge 0.25)\le \frac{\mathrm{Var}({\bar{X}}_n)}{(0.25{)}^2}=\frac{1/n}{1/16}=\frac{16}{n}.$$

   Requiring $\frac{16}{n}\le \frac{1}{100}$ gives $n\ge 1600$; the smallest such $n$ is $1600$. This is a guarantee valid for every population with $\sigma =1$, whatever its shape. (A normal approximation would ask for only about a hundred observations; the difference is the price of assuming nothing.)

   (c) Chebyshev's constant is also sharp. Take the three-point distribution

   $$X=\{\begin{array}{ll}\mu -2\sigma & \text{with probability }\frac{1}{8},\\ \mu & \text{with probability }\frac{3}{4},\\ \mu +2\sigma & \text{with probability }\frac{1}{8}.\end{array}$$

   Mean: the distribution is symmetric about $\mu$, so $E(X)=\mu$; explicitly, $E(X-\mu )=\frac{1}{8}(-2\sigma )+\frac{3}{4}(0)+\frac{1}{8}(2\sigma )=0$.

   Variance: $E[(X-\mu {)}^2]=\frac{1}{8}(4{\sigma }^2)+\frac{3}{4}(0)+\frac{1}{8}(4{\sigma }^2)=\frac{{\sigma }^2}{2}+\frac{{\sigma }^2}{2}={\sigma }^2$. ✓

   The tail: $P(|X-\mu |\ge 2\sigma )=\frac{1}{8}+\frac{1}{8}=\frac{1}{4}=\frac{1}{2^2}$, exactly Chebyshev's bound with $k=2\sigma$.

   Chebyshev's inequality must hold for this distribution as well as for the normal, so the constant $\frac{1}{k^2}$ cannot be lowered. The normal distribution keeps most of its mass close to $\mu$ and puts only about $0.046$ beyond $2\sigma$ — a factor of more than five below the bound. The bound is loose for the normal precisely because it is tight for the worst case: universality is paid for in slack.

9. 9.Sketch · 9 min

   A sequence of coin flips begins with four heads and then alternates strictly for ever: $H,H,H,H,T,H,T,H,T,H,\dots$. (The sequence is chosen by hand rather than by a coin, so it is a picture of a mechanism, not a random experiment.) Let $H_n$ be the number of heads among the first $n$ flips, ${\hat{p}}_n=H_n/n$ the running proportion, and $H_n-\frac{n}{2}$ the running excess of heads.

   1. (a)For $n\ge 4$ find closed forms for $H_n$, ${\hat{p}}_n$ and the excess $H_n-\frac{n}{2}$, treating even $n=2j$ and odd $n=2j+1$ separately, and show that in both cases ${\hat{p}}_n-\frac{1}{2}=\frac{H_n-n/2}{n}$.
   2. (b)Sketch ${\hat{p}}_n$ against $n$ for $1\le n\le 30$, on axes running from $0$ to $1$, with the horizontal lines $y=0.4$, $y=0.5$ and $y=0.6$ drawn. Find the last $n$ at which ${\hat{p}}_n$ is not strictly inside the band $0.4<{\hat{p}}_n<0.6$, and prove that it stays strictly inside for every later $n$.
   3. (c)On separate axes sketch the excess $H_n-\frac{n}{2}$ for $1\le n\le 30$. Explain from your formulas why the excess never tends to $0$ although ${\hat{p}}_n\to \frac{1}{2}$, and say which of the two quantities the Law of Large Numbers makes a claim about.
   Show the solution to question 9

   (a) Closed forms. For $n\le 4$ every flip is a head: $H_n=n$, ${\hat{p}}_n=1$, and the excess is $n-\frac{n}{2}=\frac{n}{2}$, i.e. $\frac{1}{2},1,\frac{3}{2},2$.

   From flip $5$ onward the flips come in pairs $(T,H)$, each pair contributing exactly one head.

   Even $n=2j$ with $j\ge 2$. Flips $5,\dots ,2j$ form $j-2$ complete pairs, so

   $$H_n=4+(j-2)=j+2,{\hat{p}}_n=\frac{j+2}{2j}=\frac{1}{2}+\frac{1}{j}=\frac{1}{2}+\frac{2}{n},H_n-\frac{n}{2}=(j+2)-j=2.$$

   Odd $n=2j+1$ with $j\ge 2$. One more flip, a tail, so $H_n=j+2$ still, and

   $${\hat{p}}_n=\frac{j+2}{2j+1},H_n-\frac{n}{2}=(j+2)-\frac{2j+1}{2}=\frac{3}{2},{\hat{p}}_n-\frac{1}{2}=\frac{2(j+2)-(2j+1)}{2(2j+1)}=\frac{3}{2(2j+1)}=\frac{3}{2n}.$$

   In both cases ${\hat{p}}_n-\frac{1}{2}=\frac{H_n-n/2}{n}$ — which is in fact an identity for every $n$, since $\frac{H_n}{n}-\frac{1}{2}=\frac{H_n-n/2}{n}$. The excess is $2$ at even $n$ and $\frac{3}{2}$ at odd $n$, for ever.

   (b) The sketch of ${\hat{p}}_n$, and the band. The drawing must show: four dots at height $1$ for $n=1,2,3,4$; then a sawtooth that descends towards $0.5$ from above and never crosses it, since ${\hat{p}}_n-\frac{1}{2}$ is always positive. The even-$n$ points lie on the upper envelope $\frac{1}{2}+\frac{2}{n}$ and the odd-$n$ points on the lower envelope $\frac{1}{2}+\frac{3}{2n}$; some values to plot are ${\hat{p}}_5=\frac{4}{5}=0.8$, ${\hat{p}}_6=\frac{5}{6}$, ${\hat{p}}_8=\frac{3}{4}$, ${\hat{p}}_{10}=0.7$, ${\hat{p}}_{12}=\frac{2}{3}$, ${\hat{p}}_{20}=0.6$, ${\hat{p}}_{29}=\frac{16}{29}\approx 0.55$ and ${\hat{p}}_{30}=\frac{17}{30}\approx 0.57$. The teeth shrink as $n$ grows. Draw the three horizontal lines; the line $y=0.4$ is never approached, because the deviation is one-sided.

   Entering the band. Strictly inside means $0<{\hat{p}}_n-\frac{1}{2}<\frac{1}{10}$. For even $n$ this is $\frac{2}{n}<\frac{1}{10}$, i.e. $n>20$, so even $n\ge 22$ are inside and even $n\le 20$ are not: ${\hat{p}}_{16}=\frac{10}{16}=0.625$, ${\hat{p}}_{18}=\frac{11}{18}\approx 0.611$, and ${\hat{p}}_{20}=\frac{12}{20}=0.6$ sits exactly on the boundary. For odd $n$ it is $\frac{3}{2n}<\frac{1}{10}$, i.e. $n>15$, so odd $n\ge 17$ are inside: ${\hat{p}}_{17}=\frac{10}{17}\approx 0.588$ and ${\hat{p}}_{19}=\frac{11}{19}\approx 0.579$. The sequence therefore dips into the band at $17$, leaves at $18$, re-enters at $19$, touches the edge at $20$, and the last $n$ not strictly inside is $n=20$. Mark $17$–$20$ on the sketch: this in-and-out is exactly the behaviour the weak law permits.

   Proof that it stays inside. For every $n\ge 21$, whichever parity, $0<{\hat{p}}_n-\frac{1}{2}\le \frac{2}{n}\le \frac{2}{21}<\frac{1}{10}$, using $\frac{3}{2n}<\frac{2}{n}$ for the odd case. So ${\hat{p}}_n\in (0.5,0.6)\subset (0.4,0.6)$ for all $n\ge 21$. $■$

   (c) The sketch of the excess, and the moral. The drawing shows points at $\frac{1}{2},1,\frac{3}{2},2$ for $n=1,\dots ,4$, and then a flat zigzag between $\frac{3}{2}$ (odd $n$) and $2$ (even $n$) all the way to $n=30$ and beyond. It never approaches the axis $y=0$; in fact it never drops below $\frac{3}{2}$ again.

   Why the two pictures differ: after the fourth flip a surplus of two heads exists, and nothing in the later flips ever pays it back — each $(T,H)$ pair has zero net effect on the excess. Yet ${\hat{p}}_n-\frac{1}{2}=\frac{\text{excess}}{n}$, a bounded numerator over a growing denominator, so the proportion converges to $\frac{1}{2}$ all the same. The proportion returns to $\frac{1}{2}$ by dilution, not by compensation.

   The Law of Large Numbers is a statement about the proportion — the sample mean of the $0/1$ indicators — and nothing else. It says nothing about the count. For a genuinely random coin the situation is worse than in this hand-made sequence: the excess is not even bounded, its standard deviation being $\frac{1}{2}\sqrt{n}$, which grows without limit. Anyone expecting a run of heads to be "corrected" by later tails is expecting the count to converge, and it never does.

10. 10.Proof · 11 min

    Let $X_1,X_2,\dots$ be i.i.d. fair coin flips coded $1$ for heads and $0$ for tails, let $H_n={\sum }_{i=1}^n{X}_i$ be the number of heads, and let

    $$Z_n=\frac{H_n-n/2}{\sqrt{n}/2}.$$

    You may quote: the MGF of a sum of independent variables is the product of their MGFs; $M_{cX}(t)=M_X(ct)$; the MGF of $N(0,1)$ is $e^{t^2/2}$; the continuity theorem for MGFs; the series $\cosh u={\sum }_{k\ge 0}\frac{u^{2k}}{(2k)!}$; the limit ${\left(1+\frac{a}{n}+o(1/n)\right)}^n\to e^a$; and $\Phi (2)=0.9772$.

    1. (a)Find $\mu$ and ${\sigma }^2$ for a single flip, and show that $Z_n$ is both the standardised sample mean $\frac{{\bar{X}}_n-\mu }{\sigma /\sqrt{n}}$ and the standardised sum. Then show that $Z_n=\frac{1}{\sqrt{n}}{\sum }_{i=1}^n{Y}_i$, where $Y_i=2X_i-1$ takes the values $\pm 1$ with probability $\frac{1}{2}$ each.
    2. (b)Compute the MGF $m(t)=E\left(e^{t{Y}_i}\right)$ and show that $M_{Z_n}(t)=[\cosh (t/\sqrt{n}){]}^n$.
    3. (c)Prove that $M_{Z_n}(t)\to e^{t^2/2}$ for every fixed $t$, controlling the remainder of the $\cosh$ series explicitly, and conclude that $Z_n\stackrel{d}{\to }N(0,1)$.
    4. (d)Deduce $\underset{n\to \infty }{\lim }P\left(H_n\ge \frac{n}{2}+\sqrt{n}\right)$, and reconcile the fact that this limit is positive with the Law of Large Numbers.
    Show the solution to question 10

    (a) Standardising. A single flip has $\mu =E(X_i)=1\cdot \frac{1}{2}+0\cdot \frac{1}{2}=\frac{1}{2}$ and $E(X_i^2)=\frac{1}{2}$, so ${\sigma }^2=\frac{1}{2}-\frac{1}{4}=\frac{1}{4}$ and $\sigma =\frac{1}{2}$.

    The standardised mean is

    $$\frac{{\bar{X}}_n-\frac{1}{2}}{\frac{1}{2}/\sqrt{n}}=2\sqrt{n}\left(\frac{H_n}{n}-\frac{1}{2}\right)=\frac{2}{\sqrt{n}}\left(H_n-\frac{n}{2}\right)=\frac{H_n-n/2}{\sqrt{n}/2}=Z_n,$$

    and the standardised sum is $\frac{H_n-n\mu }{\sigma \sqrt{n}}=\frac{H_n-n/2}{\sqrt{n}/2}$, the same expression: mean and sum standardise to one and the same variable.

    With $Y_i=2X_i-1$: heads gives $Y_i=1$, tails gives $Y_i=-1$, each with probability $\frac{1}{2}$, and the $Y_i$ are independent because the $X_i$ are. Summing, ${\sum }_i{Y}_i=2H_n-n=2\left(H_n-\frac{n}{2}\right)$, so

    $$\frac{1}{\sqrt{n}}\sum _{i=1}^n{Y}_i=\frac{2(H_n-n/2)}{\sqrt{n}}=Z_n.✓$$

    (b) The MGF. $m(t)=E\left(e^{tY}\right)=\frac{1}{2}{e}^t+\frac{1}{2}{e}^{-t}=\cosh t$. Since $Z_n={\sum }_i(Y_i/\sqrt{n})$ is a sum of independent variables, its MGF is the product of theirs, and each factor is $M_{Y_i/\sqrt{n}}(t)=m(t/\sqrt{n})$ by the scaling rule. Hence

    $$M_{Z_n}(t)=\prod _{i=1}^n\cosh \left(\frac{t}{\sqrt{n}}\right)={\left[\cosh \left(\frac{t}{\sqrt{n}}\right)\right]}^n.$$

    (c) The limit, with the remainder controlled. Fix $t$ and write $u=t/\sqrt{n}$, so $|u|\le |t|$ for all $n\ge 1$. From the series,

    $$\cosh u=1+\frac{u^2}{2}+R(u),R(u)=\sum _{k\ge 2}\frac{u^{2k}}{(2k)!}\ge 0.$$

    For $k\ge 2$, $(2k)!=(2k)(2k-1)(2k-2)(2k-3)\cdot (2k-4)!\ge 4\cdot 3\cdot 2\cdot 1\cdot (2k-4)!=24(2k-4)!$, so

    $$R(u)\le \frac{u^4}{24}\sum _{k\ge 2}\frac{u^{2k-4}}{(2k-4)!}=\frac{u^4}{24}\sum _{m\ge 0}\frac{u^{2m}}{(2m)!}=\frac{u^4}{24}\cosh u\le \frac{u^4}{24}\cosh t.$$

    Substituting $u=t/\sqrt{n}$ gives $0\le R(t/\sqrt{n})\le \frac{t^4\cosh t}{24n^2}$, which is $O(1/n^2)$ and therefore $o(1/n)$ as $n\to \infty$ with $t$ fixed. So

    $$\cosh \left(\frac{t}{\sqrt{n}}\right)=1+\frac{t^2/2}{n}+o\left(\frac{1}{n}\right),M_{Z_n}(t)={\left[1+\frac{t^2/2}{n}+o\left(\frac{1}{n}\right)\right]}^n⟶e^{t^2/2}$$

    by the quoted limit with $a=t^2/2$. The right-hand side is the MGF of $N(0,1)$, and the convergence holds for every real $t$, in particular on a neighbourhood of $0$; by the continuity theorem for MGFs, $Z_n\stackrel{d}{\to }N(0,1)$. $■$

    This is the Central Limit Theorem for a fair coin, and the proof is the general MGF argument with every step made explicit: the population is symmetric, so $m$ has no odd terms, and $\cosh$ is entire, so the remainder is under complete control rather than merely $o(t^2)$.

    (d) A tail probability for the count. The event rewrites as

    $$H_n\ge \frac{n}{2}+\sqrt{n}⟺H_n-\frac{n}{2}\ge \sqrt{n}⟺Z_n=\frac{H_n-n/2}{\sqrt{n}/2}\ge \frac{\sqrt{n}}{\sqrt{n}/2}=2.$$

    Since $2$ is a continuity point of $\Phi$, convergence in distribution gives

    $$\underset{n\to \infty }{\lim }P\left(H_n\ge \frac{n}{2}+\sqrt{n}\right)=\underset{n\to \infty }{\lim }P(Z_n\ge 2)=1-\Phi (2)=1-0.9772=\mathrm{0.0228.}$$

    Reconciling with the Law of Large Numbers. The event says the surplus of heads is at least $\sqrt{n}$ — and that stays about $2.3\%$ likely however large $n$ becomes, because the excess count $H_n-\frac{n}{2}$ is of order $\sqrt{n}$ and does not converge. But in terms of the proportion the same event is ${\hat{p}}_n-\frac{1}{2}\ge \frac{\sqrt{n}}{n}=\frac{1}{\sqrt{n}}$, a deviation that shrinks to $0$. The Law of Large Numbers says $P(|{\hat{p}}_n-\frac{1}{2}|\ge \epsilon )\to 0$ for each fixed $\epsilon$, and there is no conflict: for any fixed $\epsilon$, eventually $\frac{1}{\sqrt{n}}<\epsilon$, so the CLT event lies inside the band the LLN says is almost certain. The two theorems look at the same deviation at different magnifications — the LLN at a fixed scale, where it vanishes; the CLT at scale $1/\sqrt{n}$, where it is a bell curve.

11. 11.Critique · 8 min

    An insurer's individual claim sizes, in thousands, are independent draws from a strongly right-skewed population with mean $\mu =4$ and standard deviation $\sigma =6$; every claim is non-negative. A trainee analyses a batch of $n=36$ claims and writes:

    Step 1. Since $n=36\ge 30$, the Central Limit Theorem applies, so each claim is approximately $N(4,36)$.

    Step 2. Hence the probability that a single claim exceeds $22$ is $P\left(Z>\frac{22-4}{6}\right)=P(Z>3)\approx 0.0013$.

    Step 3. The total of the $36$ claims has mean $36\times 4=144$ and standard deviation $6/\sqrt{36}=1$, so the probability that the total exceeds $216$ is $P(Z>72)$, which is zero for all practical purposes.

    You may use $\Phi (2)=0.9772$ and $\Phi (3)=0.9987$.

    1. (a)Identify every error. For each, name the theorem or result being misapplied and the hypothesis or conclusion being misread.
    2. (b)With the information given, what can honestly be said about the probability that a single claim exceeds $22$? Obtain one bound from Markov's inequality and another from Chebyshev's inequality, and say which is sharper.
    3. (c)Correct Step 3: find the approximate probability that the total exceeds $216$. Then say, with reference to the population's shape and where the event sits, how far that figure should be trusted.
    Show the solution to question 11

    (a) Three errors.

    Step 1 applies the Central Limit Theorem to the wrong object. The theorem's conclusion is about the sample mean ${\bar{X}}_n$ (equivalently the sum $S_n$): the standardised mean converges in distribution to $N(0,1)$. It says nothing whatever about an individual $X_i$, which has the population's distribution — strongly right-skewed — no matter how many other claims are collected alongside it. Collecting $36$ claims does not reshape any one of them.

    Step 1 also misreads "$n\ge 30$". That is a rule of thumb calibrated on moderately skewed populations, not a hypothesis of the theorem, whose actual hypotheses are i.i.d. observations with $0<{\sigma }^2<\infty$. For a strongly right-skewed population the normal approximation to ${\bar{X}}_n$ may need $n$ in the hundreds, and it fails first in the tails. Even for the mean, $n=36$ earns only cautious trust here.

    Step 2 inherits the error. The figure $0.0013$ is the upper tail of a normal distribution that does not describe a claim. Nothing in the data supports it.

    Step 3 confuses the standard deviation of the mean with that of the sum. The standard error $\sigma /\sqrt{n}=1$ belongs to ${\bar{X}}_{36}$. By the CLT for sums, $S_n$ is approximately $N(n\mu ,n{\sigma }^2)$, with standard deviation $\sigma \sqrt{n}=6\times 6=36$, not $1$. The spread of a total grows with $n$. A $z$-score of $72$ should itself have triggered a sanity check: it claims that a total only $50\%$ above its mean is seventy-two standard deviations out, which no real quantity ever is.

    (b) What can honestly be said. Let $X$ be one claim. Since $X\ge 0$ with $E(X)=4$, Markov's inequality with $a=22$ gives

    $$P(X\ge 22)\le \frac{4}{22}=\frac{2}{11}\approx \mathrm{0.18.}$$

    Chebyshev's inequality with $k=22-4=18$ gives $P(|X-4|\ge 18)\le \frac{36}{18^2}=\frac{36}{324}=\frac{1}{9}$; and since $\{X\ge 22\}\subseteq \{|X-4|\ge 18\}$,

    $$P(X\ge 22)\le \frac{1}{9}\approx \mathrm{0.11.}$$

    Chebyshev is the sharper of the two here ($\frac{1}{9}<\frac{2}{11}$, since $9\cdot 2=18>11$), because it uses the variance as well as the mean. The honest statement is: the probability is at most about $11\%$, and could be anything below that. It may well be much smaller than $\frac{1}{9}$, but there is no basis for $0.0013$ — a heavy right tail is exactly the kind of population that puts several percent of its mass beyond $22$.

    (c) Step 3 corrected. $S_{36}$ has mean $144$ and standard deviation $36$, so

    $$P(S_{36}>216)\approx P\left(Z>\frac{216-144}{36}\right)=P(Z>2)=1-\Phi (2)=\mathrm{0.0228.}$$

    How far to trust it. The Central Limit Theorem licenses the approximation, but its accuracy at $n=36$ depends on the population's skewness, and this population is strongly right-skewed — the regime where the rule of thumb is least reliable. Worse, the event is a right-tail probability of about $2\%$, and the tails are where a skewed sample mean departs from normality most stubbornly; for a right-skewed population the normal figure typically understates the right-tail probability at moderate $n$. The figure $0.0228$ is therefore a rough guide — "a few percent, plausibly somewhat more" — and not a number to underwrite a reserve on. A decision that hinges on it needs an exact calculation from the claim distribution or a bootstrap, not the CLT.

12. 12.Computation · 10 min

    Throughout, use $z^{\ast }=1.96$ for $95\%$ confidence, with $(1.96{)}^2=3.8416$, and keep the arithmetic exact.

    1. (a)Prove that $p(1-p)\le \frac{1}{4}$ for $0\le p\le 1$, with equality only at $p=\frac{1}{2}$, and explain why this makes $p^{\ast }=\frac{1}{2}$ the safe planning value in a sample-size calculation.
    2. (b)A poll is to estimate a proportion with $95\%$ margin of error at most $0.02$. Find the required $n$ with the conservative planning value, and again with a prior estimate $p^{\ast }=0.1$. For the second, check the conditions for the normal approximation, and say exactly what goes wrong if the true proportion turns out to be $\frac{1}{2}$.
    3. (c)Two independent trials record a side effect in $40$ of $400$ patients on treatment A and in $80$ of $400$ patients on treatment B. Check the conditions, find the standard error of ${\hat{p}}_B-{\hat{p}}_A$, give a $95\%$ confidence interval for $p_B-p_A$, and state the interval a student who added the two standard errors would have reported and why it is wrong.
    Show the solution to question 12

    (a) The worst case is $p=\frac{1}{2}$. Complete the square:

    $$p(1-p)=p-p^2=\frac{1}{4}-\left(p^2-p+\frac{1}{4}\right)=\frac{1}{4}-{\left(p-\frac{1}{2}\right)}^2.$$

    Since ${\left(p-\frac{1}{2}\right)}^2\ge 0$, this is at most $\frac{1}{4}$, with equality exactly when $p=\frac{1}{2}$. $■$

    The required sample size for margin $m$ is $n=\frac{(z^{\ast }{)}^2{p}^{\ast }(1-p^{\ast })}{m^2}$, which increases with $p^{\ast }(1-p^{\ast })$. Choosing $p^{\ast }=\frac{1}{2}$ therefore gives the largest $n$ any $p$ could demand, so the margin $m$ is achieved whatever the true proportion turns out to be. Any other planning value gambles that $p$ is at least as far from $\frac{1}{2}$ as the guess.

    (b) Two sample sizes. With the conservative $p^{\ast }=\frac{1}{2}$ and $m=0.02$,

    $$n=\frac{3.8416\times \frac{1}{4}}{(0.02{)}^2}=\frac{0.9604}{0.0004}=2401,$$

    which happens to be an exact integer, so $n=2401$. (It is $(1.96/0.04{)}^2=49^2$.)

    With the prior estimate $p^{\ast }=0.1$,

    $$n=\frac{3.8416\times 0.1\times 0.9}{0.0004}=\frac{0.345744}{0.0004}=864.36,$$

    and rounding up gives $n=865$ — about $36\%$ of the conservative figure. Conditions: $n{p}^{\ast }=865\times 0.1=86.5\ge 10$ and $n(1-p^{\ast })=865\times 0.9=778.5\ge 10$, both comfortably satisfied, so the normal approximation behind the formula is legitimate.

    If the truth is $p=\frac{1}{2}$, the standard error at $n=865$ is $\sqrt{\frac{0.25}{865}}$ rather than the planned $\sqrt{\frac{0.09}{865}}$ — larger by the factor $\sqrt{\frac{0.25}{0.09}}=\frac{5}{3}$. The margin of error scales the same way: $0.02\times \frac{5}{3}=\frac{1}{30}\approx 0.033$, well over the $0.02$ that was promised. The poll would be $865$ interviews spent on a guarantee it cannot keep.

    (c) Difference of two proportions. The estimates are ${\hat{p}}_A=\frac{40}{400}=0.1$ and ${\hat{p}}_B=\frac{80}{400}=0.2$. Conditions: successes and failures are $40$ and $360$ for A, $80$ and $320$ for B, all at least $10$, so both sampling distributions may be treated as normal.

    Standard errors of the two estimates:

    $$S{E}_A=\sqrt{\frac{0.1\times 0.9}{400}}=\sqrt{\frac{0.09}{400}}=\frac{0.3}{20}=0.015,S{E}_B=\sqrt{\frac{0.2\times 0.8}{400}}=\sqrt{\frac{0.16}{400}}=\frac{0.4}{20}=\mathrm{0.02.}$$

    The two samples are independent, so the variances add — the minus sign in ${\hat{p}}_B-{\hat{p}}_A$ is squared away:

    $$\mathrm{Var}({\hat{p}}_B-{\hat{p}}_A)=S{E}_B^2+S{E}_A^2=0.0004+0.000225=0.000625,SE=\sqrt{0.000625}=\mathrm{0.025.}$$

    (The three numbers $0.015$, $0.02$, $0.025$ are a $3$–$4$–$5$ triangle: the standard error of a difference is the hypotenuse, not the sum of the legs.) The point estimate is $0.2-0.1=0.1$, the margin is $1.96\times 0.025=0.049$, and the $95\%$ interval is

    $$0.1\pm 0.049=(0.051,0.149).$$

    It excludes $0$, so the data are reasonably strong evidence that treatment B produces the side effect more often — by somewhere between $5$ and $15$ points.

    The student who adds standard errors would take $SE=0.015+0.02=0.035$, margin $1.96\times 0.035=0.0686$, and report $(0.031,0.169)$: an interval $40\%$ too wide. It is wrong because standard deviations of independent quantities do not add; variances do. Adding them overstates the uncertainty of every comparison and quietly throws away the power to detect a real difference. For two equal standard errors the inflation is exactly $\sqrt{2}$; here it is $\frac{0.035}{0.025}=1.4$.