Nonparametric Methods

(a) Both gaps at every jump. The empirical distribution function $F_4$ is a staircase that equals $(i-1)/4$ just before $x_{(i)}$ and $i/4$ just after it. Since $F_0$ is continuous, the largest vertical distance between the two functions occurs at one of these eight values.

The “after” gaps $|i/4-F_0(x_{(i)})|$ are

$$|0.25-0.10|=0.15,|0.50-0.45|=0.05,|0.75-0.55|=0.20,|1.00-0.70|=\mathrm{0.30.}$$

The “before” gaps $|(i-1)/4-F_0(x_{(i)})|$ are

$$|0-0.10|=0.10,|0.25-0.45|=0.20,|0.50-0.55|=0.05,|0.75-0.70|=\mathrm{0.05.}$$

The largest of the eight is $0.30$, so $D_4=0.30$, attained just after the fourth observation: the staircase has reached $1$ while $F_0$ is still at $0.70$.

(b) Move the last point. Only the two gaps at $x_{(4)}$ change. After the jump, $|1.00-0.97|=0.03$; before it, $|0.75-0.97|=0.22$. The other six gaps are as in (a), the largest of them being $0.20$ (twice). So now

$$D_4=0.22,$$

attained just before the fourth observation. In (a) the maximum came from the after-side of the last jump; here it comes from the before-side of the same jump, and a computation that checked only the after-gaps would have reported $0.20$ instead of $0.22$. The supremum of $|F_n-F_0|$ is always at a jump, but it can be on either side, so both must be checked at every observation.

(c) Distribution-free. By the probability integral transform, if $X$ has continuous distribution function $F_0$ then $U=F_0(X)$ is Uniform$(0,1)$. Since $F_0$ is non-decreasing, the empirical distribution function of the transformed values $u_i=F_0(x_i)$, evaluated at $u=F_0(x)$, equals $F_4(x)$; hence

$$D_4=\underset{x}{\sup }|F_4(x)-F_0(x)|=\underset{u\in (0,1)}{\sup }|G_4(u)-u|,$$

where $G_4$ is the empirical distribution function of four uniforms. Nothing about the particular $F_0$ survives the substitution, so the null distribution of $D_4$ is the same for every continuous $F_0$ — which is why a single table of critical values serves every goodness-of-fit problem of this type. Notice that the data in (a) were given to you already in the form $F_0(x_{(i)})$: those four numbers are exactly the transformed sample $u_{(i)}$.

(a) Expected counts. Under independence $E_{ij}=(\text{row total})(\text{column total})/n$, so

$$E_{11}=\frac{5\times 6}{10}=3,E_{12}=\frac{5\times 4}{10}=2,E_{21}=3,E_{22}=2.$$

All four are below $5$. The ${\chi }_1^2$ distribution for $X^2$ is asymptotic and needs expected counts of roughly $5$ or more; with $n=10$ the statistic takes only a handful of distinct values, and the continuous approximation cannot be trusted. Fisher's exact test is the right tool.

(b) The possible tables. The row totals are $5$ and $5$ and the column totals are $6$ and $4$. Since the improved column has total $6$ and each row has only $5$ patients, $a\le 5$ and $6-a\le 5$, so $a\in \{1,2,3,4,5\}$. By the hypergeometric formula with $a+b=5$, $c+d=5$ and $a+c=6$,

$$P(a)=\frac{\left(\genfrac{}{}{0ex}{}{5}{a}\right)\left(\genfrac{}{}{0ex}{}{5}{6-a}\right)}{\left(\genfrac{}{}{0ex}{}{10}{6}\right)},\left(\genfrac{}{}{0ex}{}{10}{6}\right)=210.$$

The numerators are

$$a=1:\left(\genfrac{}{}{0ex}{}{5}{1}\right)\left(\genfrac{}{}{0ex}{}{5}{5}\right)=5,a=2:\left(\genfrac{}{}{0ex}{}{5}{2}\right)\left(\genfrac{}{}{0ex}{}{5}{4}\right)=50,a=3:\left(\genfrac{}{}{0ex}{}{5}{3}\right)\left(\genfrac{}{}{0ex}{}{5}{3}\right)=100,a=4:\left(\genfrac{}{}{0ex}{}{5}{4}\right)\left(\genfrac{}{}{0ex}{}{5}{2}\right)=50,a=5:\left(\genfrac{}{}{0ex}{}{5}{5}\right)\left(\genfrac{}{}{0ex}{}{5}{1}\right)=5,$$

and $5+50+100+50+5=210$, so the probabilities $\frac{5}{210},\frac{50}{210},\frac{100}{210},\frac{50}{210},\frac{5}{210}$ sum to $1$. ✓ The observed table has probability $\frac{5}{210}=\frac{1}{42}$.

(c) The $p$-values. “Treatment improves the outcome” means large $a$. The observed $a=5$ is the largest possible value, so the only table at least as extreme is the observed one:

$$p_{\text{one-sided}}=\frac{1}{42}\approx \mathrm{0.024.}$$

For the two-sided test, the tables with probability at most $\frac{5}{210}$ are $a=5$ and $a=1$ (the mirror image, in which the control group does all the improving). So

$$p_{\text{two-sided}}=\frac{5+5}{210}=\frac{10}{210}=\frac{1}{21}\approx \mathrm{0.048.}$$

(d) The approximation, for comparison. With the observed counts $5,0,1,4$ and expected counts $3,2,3,2$,

$$X^2=\frac{(5-3{)}^2}{3}+\frac{(0-2{)}^2}{2}+\frac{(1-3{)}^2}{3}+\frac{(4-2{)}^2}{2}=\frac{4}{3}+2+\frac{4}{3}+2=\frac{20}{3}\approx 6.67,$$

which is well beyond $3.841$ and corresponds to a $p$-value near $0.01$. The exact two-sided $p$ is $0.048$ — nearly five times larger. This is the small expected counts limitation: with a discrete, skewed statistic the ${\chi }^2$ approximation rejects too often, exactly as the rule of thumb warns. (The other limitation, arbitrariness of binning, does not arise because the data are genuinely categorical.) Note what not to do: there is nothing here to pool, and pooling would change the hypothesis anyway.

(a) Study I. Degrees of freedom $(3-1)(3-1)=4$, and $32>9.488$, so independence is rejected at the $5\%$ level (the $p$-value is far below $0.001$). For the strength, $\min (r-1,c-1)=2$, so

$$V=\sqrt{\frac{X^2}{n\min (r-1,c-1)}}=\sqrt{\frac{32}{400\times 2}}=\sqrt{0.04}=\mathrm{0.2.}$$

For a fixed pattern of association $X^2$ is proportional to $n$, so at $n=100$ the statistic is $X^2=32\times \frac{100}{400}=8$, which is below $9.488$: not significant. But

$$V=\sqrt{\frac{8}{100\times 2}}=\sqrt{0.04}=0.2$$

exactly as before. The statistic measures evidence and tracks the sample size; $V$ measures strength and does not — and $V=0.2$ is a weak association whether or not $n$ is large enough to make it significant.

(b) Study II. With $b=7$ and $c=1$,

$$X^2=\frac{(b-c{)}^2}{b+c}=\frac{36}{8}=4.5>3.841,$$

so McNemar's test rejects at the $5\%$ level.

Exactly: under $H_0$ each changer is equally likely to change in either direction, so conditional on $b+c=8$ changers, $b\sim \text{Binomial}(8,\frac{1}{2})$. The one-sided tail is

$$P(b\ge 7)=\frac{\left(\genfrac{}{}{0ex}{}{8}{7}\right)+\left(\genfrac{}{}{0ex}{}{8}{8}\right)}{2^8}=\frac{8+1}{256}=\frac{9}{256},$$

and doubling gives the two-sided $p=\frac{18}{256}=\frac{9}{128}\approx 0.070$ — not significant at $5\%$.

The verdicts differ because ${\chi }_1^2$ is an asymptotic approximation to a statistic that, with only $8$ discordant pairs, takes very few values; the expected count of each kind of change is only $4$, below the working threshold of $5$. The exact binomial calculation is the one to trust. The chi-square version is the same statistic with a continuous distribution stretched over nine possible outcomes, and it rejects too readily, as approximations to discrete counts do.

(c) Why the concordant pairs drop out. The question is whether the marginal proportion of yes answers moved. A subject who answered the same way twice contributes one yes (or one no) to both margins and therefore cannot move the difference between them, whichever way the campaign worked. Under the null the only random quantity is the direction of each change, and that is what the test conditions on — the same logic as pairing in the paired $t$-test, where within-pair differences discard between-subject variation.

A chi-square test of independence on the before-by-after table would ask a different question: whether a person's before answer is associated with their after answer. With $192$ of $200$ unchanged it would reject overwhelmingly, reporting only that people tend to agree with themselves — which says nothing about whether the campaign shifted opinion.

(a) Sign test. Seven differences are positive and one is negative, no ties. Under $H_0$ the number of positives is $S\sim \text{Binomial}(8,\frac{1}{2})$, which needs only that each difference is positive with probability $\frac{1}{2}$ — no symmetry, no shape. Then

$$P(S\ge 7)=\frac{\left(\genfrac{}{}{0ex}{}{8}{7}\right)+\left(\genfrac{}{}{0ex}{}{8}{8}\right)}{2^8}=\frac{9}{256}\approx 0.035,$$

so the two-sided $p$-value is $\frac{18}{256}=\frac{9}{128}\approx 0.070$. Not significant at the $5\%$ level.

(b) Signed-rank test. Order the absolute differences and rank them:

$$0.3(1),0.7(2),0.8(3),1.9(4),2.4(5),3.3(6),4.0(7),5.1(8).$$

The only negative difference is $-0.3$, with rank $1$, so $W^{-}=1$ and, since the ranks sum to $\frac{8\cdot 9}{2}=36$,

$$W^{+}=36-1=35.$$

By the null moments of the signed-rank statistic with $n=8$,

$$E(W^{+})=\frac{8\cdot 9}{4}=18,\mathrm{Var}(W^{+})=\frac{8\cdot 9\cdot 17}{24}=\frac{1224}{24}=51.$$

So $z=\frac{35-18}{\sqrt{51}}=\frac{17}{\sqrt{51}}$. Since $7^2=49<51<51.84=7.2^2$, the standard deviation lies between $7$ and $7.2$, hence

$$2.36<\frac{17}{7.2}<z<\frac{17}{7}<2.43,$$

and every value in that range exceeds $1.96$. Reject $H_0$ at the $5\%$ level; $z\approx 2.38$, two-sided $p\approx 0.017$.

(c) Exact $p$. Under the null (differences symmetric about $0$) each of the $2^8=256$ sign patterns is equally likely. $W^{+}\ge 35$ means $W^{-}\le 1$, which happens for exactly two patterns: no negatives ($W^{-}=0$) or the rank-$1$ difference alone negative ($W^{-}=1$). So

$$P(W^{+}\ge 35)=\frac{2}{256}=\frac{1}{128},p_{\text{two-sided}}=\frac{2}{128}=\frac{1}{64}\approx 0.016,$$

in close agreement with the normal approximation.

The two tests disagree because the sign test sees only that $7$ of $8$ differences are positive, and with $n=8$ that pattern is simply not rare enough ($p=0.070$). The signed-rank test also sees that the one negative difference is the smallest in magnitude while the positives include all the large ones. That extra information is what pushes the evidence past the $5\%$ line — and it is bought with the extra assumption of symmetry, which the sign test does not need.

(a) The moments. Rank the absolute differences $|d_i|$ from $1$ to $n$ and let $Z_i=1$ if the difference holding absolute rank $i$ is positive, $Z_i=0$ otherwise. Then $W^{+}$, the sum of the ranks of the positive differences, is ${\sum }_{i=1}^{n}i{Z}_i$.

Under symmetry about zero, each difference is equally likely to be positive or negative, and its sign is independent of its magnitude: symmetry says the conditional distribution of the sign given $|d_i|$ is still a fair coin. Hence the signs are independent of the ranking, and the $Z_i$ are independent Bernoulli$(\frac{1}{2})$ variables. This is the step that needs symmetry. Under a skewed null with median zero the signs are still fair coins, but they are no longer independent of the magnitudes — large $|d|$ would be more often positive, say — so the $Z_i$ attached to large ranks would not be Bernoulli$(\frac{1}{2})$, and neither formula would hold.

By linearity,

$$E(W^{+})=\sum _{i=1}^{n}iE(Z_i)=\frac{1}{2}\sum _{i=1}^{n}i=\frac{1}{2}\cdot \frac{n(n+1)}{2}=\frac{n(n+1)}{4}.$$

By independence the variance of the sum is the sum of the variances, and $\mathrm{Var}(Z_i)=\frac{1}{2}\cdot \frac{1}{2}=\frac{1}{4}$, so

$$\mathrm{Var}(W^{+})=\sum _{i=1}^n{i}^2\mathrm{Var}(Z_i)=\frac{1}{4}\cdot \frac{n(n+1)(2n+1)}{6}=\frac{n(n+1)(2n+1)}{24}.■$$

Linearity did not need independence; the variance step did.

(b) Symmetry of the null distribution. The ranks of the negative differences are those $i$ with $Z_i=0$, so

$$W^{-}=\sum _{i=1}^{n}i(1-Z_i)=\frac{n(n+1)}{2}-W^{+}.$$

Under the null the vector $(Z_1,\dots ,Z_n)$ consists of independent fair coins, and so does $(1-Z_1,\dots ,1-Z_n)$: the two vectors have the same joint distribution, namely uniform over the $2^n$ sign patterns. Since $W^{+}$ is the function $\sum i{Z}_i$ of the first vector and $W^{-}$ is the same function of the second, $W^{-}$ and $W^{+}$ have the same distribution. Therefore, for any $w$,

$$P(W^{+}\ge w)=P(W^{-}\ge w)=P\left(\frac{n(n+1)}{2}-W^{+}\ge w\right)=P\left(W^{+}\le \frac{n(n+1)}{2}-w\right).■$$

The null distribution is symmetric about $\frac{n(n+1)}{4}$, which is why one tail of a table suffices.

(c) The floor. With $n=5$ the ranks sum to $15$ and there are $2^5=32$ equally likely sign patterns. The most extreme value $W^{+}=15$ arises from exactly one pattern (all positive), so the smallest one-sided $p$-value is $\frac{1}{32}=0.03125$. By (b) the lower tail $W^{+}\le 0$ has the same probability, so the smallest two-sided $p$-value is $\frac{2}{32}=\frac{1}{16}=0.0625$, which exceeds $0.05$: no two-sided signed-rank test on five pairs can reject at the $5\%$ level, whatever the data.

In general the smallest two-sided $p$ is $\frac{2}{2^n}$, and $\frac{2}{2^n}\le 0.05$ requires $2^n\ge 40$, that is $n\ge 6$: with six pairs the extreme pattern gives $p=\frac{2}{64}=0.03125$. (One-sided, $2^n\ge 20$ suffices, so $n=5$ is enough.) A rank test cannot be more decisive than the number of sign patterns allows, and with tiny samples that ceiling binds before any question about power arises.

(a) The identity. Let the first-sample observations in increasing order have combined ranks $r_{(1)}<r_{(2)}<\cdots <r_{(n_1)}$. The $k$-th smallest first-sample value has $k-1$ first-sample values below it, so the remaining $r_{(k)}-k$ observations below it all belong to the second sample: $r_{(k)}-k$ is the number of $y_j$ it exceeds. Summing over $k$,

$$\#\{(i,j):x_i>y_j\}=\sum _{k=1}^{n_1}\left(r_{(k)}-k\right)=W-\frac{n_1(n_1+1)}{2}.■$$

Its range is $0$ to $n_1{n}_2$, and $W$ alone ranges from $\frac{n_1(n_1+1)}{2}$ upward, which is exactly why the offset matters.

(b) Three errors.

Step 1 confuses $W$ with $U$. By the definition of the Mann-Whitney statistic, $U=W-\frac{n_1(n_1+1)}{2}=17-15=2$, not $17$. The null moments $E(U)=\frac{n_1{n}_2}{2}$ and $\mathrm{Var}(U)=\frac{n_1{n}_2(n_1+n_2+1)}{12}$ are moments of $U$; feeding them $W$ compares a statistic with the wrong centre. (Note that $W=17$ is only $2$ above its minimum of $15$: the first sample holds nearly all the lowest ranks, which is the opposite of “no evidence”.)

Step 3 turns non-rejection into acceptance. Failing to reject $H_0$ means the data are compatible with $H_0$; it never establishes it. With $n_1=5$ and $n_2=7$ the test has modest power, so a non-significant result would be weak evidence of anything.

Step 3 also misstates the hypothesis. The Mann-Whitney null is that the two distributions are identical, and the alternative is a stochastic ordering, $P(X<Y)\ne \frac{1}{2}$. A conclusion about medians is valid only under the extra assumption that the two distributions differ by a pure shift and are otherwise the same shape — an assumption the student never stated.

(c) The correct test. With $U=2$,

$$z=\frac{2-17.5}{\sqrt{455/12}}=\frac{-15.5}{\sqrt{455/12}}.$$

Compare squares: $|z|>1.96$ if and only if $15.5^2>1.96^2\times \frac{455}{12}$. The left side is $240.25$. On the right, $1.96^2=3.8416$ and $\frac{455}{12}<38$, so the right side is below $3.8416\times 38=145.98$. Since $240.25>146$, $|z|>1.96$ comfortably; in fact $6.1^2=37.21<\frac{455}{12}<38.44=6.2^2$ gives $2.50<|z|<2.54$, so $z\approx -2.52$ and the two-sided $p$ is about $0.012$.

Reject $H_0$ at the $5\%$ level. The justified conclusion: the two samples are unlikely to come from the same distribution, and the direction is that first-sample values tend to be smaller — only $2$ of the $35$ pairs have the first-sample value on top. Any statement about medians requires adding, and stating, the shift assumption.

(a) Spearman. With $r_i=1,2,3,4,5$ and $s_i=3,1,2,5,4$, the rank differences $d_i=r_i-s_i$ are

$$-2,+1,+1,-1,+1,\text{so}\sum d_i^2=4+1+1+1+1=8.$$

There are no ties, so the computing formula applies with $n(n^2-1)=5\times 24=120$:

$${\rho }_s=1-\frac{6\times 8}{120}=1-\frac{48}{120}=1-\frac{2}{5}=\frac{3}{5}.$$

(b) Kendall. Listing the second-exam ranks in first-exam order, $3,1,2,5,4$, a pair $(i,j)$ with $i<j$ is concordant when the second rank increases too. The $\left(\genfrac{}{}{0ex}{}{5}{2}\right)=10$ pairs:

$$(1,2)3>1D,(1,3)3>2D,(1,4)3<5C,(1,5)3<4C,(2,3)1<2C,$$

$$(2,4)1<5C,(2,5)1<4C,(3,4)2<5C,(3,5)2<4C,(4,5)5>4D.$$

So $C=7$, $D=3$ and

$$\tau =\frac{C-D}{\left(\genfrac{}{}{0ex}{}{5}{2}\right)}=\frac{4}{10}=\frac{2}{5}.$$

As usual $|\tau |<|{\rho }_s|$ for the same data; here $\tau =\frac{2}{3}{\rho }_s$ exactly, the rough rule of thumb landing on the nose. The interpretation of $\tau$ is direct: a randomly chosen pair of students is ordered the same way by both exams with probability $0.7$ and the opposite way with probability $0.3$, and $\tau$ is the difference.

(c) Correcting the error. With the true score the second-exam values are $6,1,3,100,40$. Since $100$ is still the largest, student $4$ still has rank $5$ and every other rank is unchanged: the rank vector is $3,1,2,5,4$ exactly as before. Every $d_i$ is unchanged, so ${\rho }_s=\frac{3}{5}$ still; every pair is concordant or discordant exactly as before, so $\tau =\frac{2}{5}$ still. A rank depends only on order, and correcting $1000$ to $100$ preserves the order of the five scores. The same is true of any strictly increasing transformation of the scores — squaring them, taking logarithms — which is why rank correlation detects any monotone relationship and not merely a linear one.

Pearson's $r$ is built from products of deviations, $S_{xy}=\sum (x_i-\bar{x})(y_i-\bar{y})$, and with $y_4=1000$ that single term dominates $S_{xy}$ while $(y_4-\bar{y}{)}^2$ dominates $S_{yy}$. In the limit of an ever larger $y_4$, $r$ tends to the correlation between $x$ and the indicator of student $4$: the $x$-deviations are $-2,-1,0,1,2$, the indicator's deviations from its mean $0.2$ are $-0.2,-0.2,-0.2,0.8,-0.2$, the cross-products sum to $0.4+0.2+0+0.8-0.4=1$, and with $S_{xx}=10$ and $S_{II}=4(0.04)+0.64=0.8$ the limit is $r=\frac{1}{\sqrt{8}}=\frac{1}{2\sqrt{2}}\approx 0.35$. So $r$ is being reported by one observation, and correcting that observation to $100$ moves $r$ appreciably, while a second error putting it at $10^6$ would move it again. The rank statistics bound the influence of any one observation — the erroneous score can never contribute more than a rank difference of $4$ — and that robustness is a different property from being assumption-free.

(a) The sum of squared deviations of $1,\dots ,n$. Expand the square and use the two quoted sums:

$$\sum _{i=1}^n{\left(i-\frac{n+1}{2}\right)}^2=\sum i^2-(n+1)\sum i+n\cdot \frac{(n+1{)}^2}{4}=\frac{n(n+1)(2n+1)}{6}-\frac{n(n+1{)}^2}{2}+\frac{n(n+1{)}^2}{4}.$$

The last two terms combine to $-\frac{n(n+1{)}^2}{4}$. Factor out $n(n+1)$ and put everything over $12$:

$$n(n+1)\left[\frac{2(2n+1)-3(n+1)}{12}\right]=n(n+1)\cdot \frac{4n+2-3n-3}{12}=\frac{n(n+1)(n-1)}{12}=\frac{n(n^2-1)}{12}.■$$

(b) The computing formula. Because there are no ties, each rank vector is a permutation of $1,\dots ,n$. So both have the same mean $\bar{r}=\bar{s}=\frac{n+1}{2}$ and, by (a), the same sum of squared deviations,

$$S_{rr}=S_{ss}=\frac{n(n^2-1)}{12}.$$

(With ties, averaged ranks would spoil both of these — the means would still agree but the sums of squares would be smaller, and the formula would need a correction.)

Now write $d_i=(r_i-\bar{r})-(s_i-\bar{s})$, which is legitimate because the two means are equal, and expand:

$$\sum d_i^2=\sum (r_i-\bar{r}{)}^2-2\sum (r_i-\bar{r})(s_i-\bar{s})+\sum (s_i-\bar{s}{)}^2=S_{rr}-2S_{rs}+S_{ss}=2S_{rr}-2S_{rs}.$$

Hence $S_{rs}=S_{rr}-\frac{1}{2}\sum d_i^2$, and Pearson's correlation of the ranks is

$${\rho }_s=\frac{S_{rs}}{\sqrt{S_{rr}{S}_{ss}}}=\frac{S_{rr}-\frac{1}{2}\sum d_i^2}{S_{rr}}=1-\frac{\sum d_i^2}{2S_{rr}}=1-\frac{\sum d_i^2}{2\cdot n(n^2-1)/12}=1-\frac{6\sum d_i^2}{n(n^2-1)}.■$$

(c) Reversed rankings. If $s_i=n+1-r_i$ then

$$d_i=r_i-(n+1-r_i)=2r_i-(n+1)=2\left(r_i-\frac{n+1}{2}\right),$$

so, since the $r_i$ are a permutation of $1,\dots ,n$, part (a) gives

$$\sum d_i^2=4\sum _{i=1}^n{\left(i-\frac{n+1}{2}\right)}^2=4\cdot \frac{n(n^2-1)}{12}=\frac{n(n^2-1)}{3},$$

and the computing formula returns ${\rho }_s=1-\frac{6}{n(n^2-1)}\cdot \frac{n(n^2-1)}{3}=1-2=-1$. $■$

For the bound: ${\rho }_s$ is a Pearson correlation, so by the Cauchy–Schwarz inequality ${\rho }_s\ge -1$ for any two rankings. Substituting the computing formula, $1-\frac{6\sum d_i^2}{n(n^2-1)}\ge -1$ rearranges to $\sum d_i^2\le \frac{n(n^2-1)}{3}$, with equality exactly when ${\rho }_s=-1$ — which the reversed ranking attains. (Equivalently: reversing the order minimises $\sum r_i{s}_i$, the rearrangement inequality in rank form.)

Check, $n=3$. The bound is $\frac{3\cdot 8}{3}=8$. Reversing $1,2,3$ to $3,2,1$ gives $d=-2,0,2$ and $\sum d_i^2=4+0+4=8$, so ${\rho }_s=1-\frac{6\cdot 8}{24}=-1$. ✓ The other five permutations give $\sum d_i^2\in \{0,2,2,6,6\}$, all below $8$, so the bound is attained only by the reversal. ✓

(a) Reduce to the group sum. The pooled total is $1+2+4+6+9=22$. If the first group's two values sum to $s$, the second group's three values sum to $22-s$, so

$$T=\bar{y}-\bar{x}=\frac{22-s}{3}-\frac{s}{2}=\frac{2(22-s)-3s}{6}=\frac{44-5s}{6}.$$

This is a strictly decreasing function of $s$, so “$T$ at least as large as observed” is exactly “$s$ at least as small as observed”, and the two tests have identical $p$-values. Tabulating $s$ is better because it takes integer values, and because the argument shows that any statistic that is a monotone function of the first group's sum — a difference of totals, of means, a $t$-statistic with pooled variance — yields the same test.

(b) The ten relabellings. Choosing which two of $\{1,2,4,6,9\}$ go to the first group:

$$\{1,2\}\to 3,\{1,4\}\to 5,\{2,4\}\to 6,\{1,6\}\to 7,\{2,6\}\to 8,\{1,9\}\to 10,\{4,6\}\to 10,\{2,9\}\to 11,\{4,9\}\to 13,\{6,9\}\to 15.$$

The observed labelling is $\{1,2\}$, so $s_{\text{obs}}=3$, the smallest value in the list. The ten values sum to $3+5+6+7+8+10+10+11+13+15=88$, so their average is $8.8$; the pooled mean is $22/5=4.4$ and $n_1\times 4.4=8.8$. ✓ (Under exchangeability each of the two first-group slots holds a randomly chosen pooled value, so $E(s)=2\times$ pooled mean.)

The chart. Horizontal axis $s$ from $3$ to $15$ with integer ticks; vertical axis “number of relabellings” from $0$ to $2$. Draw a bar of height $1$ at each of $s=3,5,6,7,8,11,13,15$ and a bar of height $2$ at $s=10$: nine bars, total height $10$. Mark the bar at $s=3$ as the observed value — it stands alone at the far left. The rejection region defined by the data is $\{s\le 3\}$, which is just that one bar, so shade it. Draw a vertical dashed line at $E(s)=8.8$; it falls between the bars at $8$ and $10$.

(c) Reading off. Exactly one of the ten equally likely relabellings has $s\le 3$, so

$$p_{\text{one-sided}}=\frac{1}{10}.$$

This is also the smallest $p$-value the design can produce: with $n_1=2$ and $n_2=3$ nothing can reach $\alpha =0.05$, however cleanly the groups separate.

For the two-sided rule, the observed distance is $|3-8.8|=5.8$, so a relabelling counts if $s\le 3$ or $s\ge 14.6$. From the chart that is $s=3$ and $s=15$, two of ten: $p_{\text{two-sided}}=\frac{2}{10}$.

Why no symmetry. In the chapter's equal-sizes example every relabelling has a mirror image obtained by swapping the two groups, which negates $T$, so the null distribution is symmetric about $0$. Here the groups have sizes $2$ and $3$, and swapping them is not a relabelling of the same design, so nothing forces symmetry. The chart shows it: the bars to the left of $8.8$ sit at distances $5.8,3.8,2.8,1.8,0.8$ while those to the right sit at $1.2,1.2,2.2,4.2,6.2$, and the value $9$ stretches the right tail. Exactness does not need symmetry — the proof that the permutation test has the stated level uses only that the ten relabellings are equally likely — but reading a two-sided $p$-value from an asymmetric distribution does require saying which rule is being used.

(a) Argument A: Step 2 is the fatal one. Step 1 is correct, and Step 3 is only wrong as a consequence of Step 2.

The resample maximum is at most the observed maximum, because a resample contains only observed values. It equals $4.8$ unless all five draws miss the largest observation, and each draw misses it with probability $\frac{4}{5}$ independently, so

$$P(\text{resample max}=4.8)=1-{\left(\frac{4}{5}\right)}^5=1-\frac{1024}{3125}=\frac{2101}{3125}\approx \mathrm{0.672.}$$

So the bootstrap distribution of the maximum puts about two-thirds of its mass on the single point $4.8$ and the rest at $4.4$, $3.1$, $2.3$ or $0.9$ — a distribution that lives entirely at or below the estimate. Its $97.5\%$ quantile is therefore $4.8$ itself (indeed every quantile above the $33\%$ point is $4.8$), so the percentile interval has the form $(\text{something},4.8]$.

That is fatal because $\theta \ge 4.8$ with certainty — the population cannot produce a value larger than $\theta$ — and $\theta >4.8$ with probability one. An interval whose upper endpoint is $4.8$ contains $\theta$ with probability zero, not $0.95$. The bootstrap fails here because the statistic depends on an extreme order statistic, and the empirical distribution is a poor stand-in for the population exactly at its boundary. (As $n$ grows the defect does not go away: $1-(1-\frac{1}{n}{)}^n\to 1-e^{-1}\approx 0.63$.)

Correction. The sampling distribution of the maximum is not awkward at all: for a Uniform$(0,\theta )$ sample of size $n$, $P(X_{(n)}\le t)=(t/\theta {)}^n$ for $0\le t\le \theta$. Use that exact distribution to build the interval, as the estimation of $\theta$ for the uniform is done in closed form; the bootstrap is the wrong tool for a boundary parameter. Step 3's claim of “no assumption” is also empty: the argument already assumed a uniform population to justify the maximum in Step 1.

(b) Argument B: Step 2 is the fatal one. Step 1 is a fair observation, and Step 3 merely acts on Step 2.

The null hypothesis of the Wilcoxon signed-rank test is that the differences are symmetric about zero, not merely that their median is zero. The null distribution of $W^{+}$ — $E(W^{+})=\frac{n(n+1)}{4}$, $\mathrm{Var}(W^{+})=\frac{n(n+1)(2n+1)}{24}$, and the uniform distribution over the $2^n$ sign patterns — is derived by writing $W^{+}=\sum i{Z}_i$ with the signs $Z_i$ independent fair coins independent of the magnitudes. Symmetry is what delivers that independence.

On these data the assumption is visibly false. Ranking the absolute values gives $0.1(1),0.2(2),0.3(3),0.4(4),0.5(5),0.7(6),0.8(7),9(8),12(9),15(10)$; the two negatives hold ranks $1$ and $3$, so $W^{-}=4$ and $W^{+}=55-4=51$, against a null mean of $27.5$ and null standard deviation $\sqrt{96.25}\approx 9.8$ — about $2.4$ standard deviations up, a “significant” result. But look at where the evidence comes from: the three largest magnitudes, $9,12,15$, carry ranks $8,9,10$ and contribute $27$ of the $51$ on their own, while the small differences are split almost evenly in sign. Under a right-skewed population with median zero that pattern is typical, not evidence: the long right tail supplies large magnitudes far more often than the short left tail does, so the signs attached to the largest ranks are not fair coins, $E(W^{+})$ exceeds $\frac{n(n+1)}{4}$ under the null, and the test rejects a true null far more often than $\alpha$. The user would be “detecting” a shift whose existence the test had assumed away.

Correction. With skewed differences, use the sign test: $S=8$ positives out of $10$, and under the null that the median difference is zero $S\sim \text{Binomial}(10,\frac{1}{2})$, with no further assumption — that null needs only $P(d>0)=\frac{1}{2}$. Then $P(S\ge 8)=\frac{\left(\genfrac{}{}{0ex}{}{10}{8}\right)+\left(\genfrac{}{}{0ex}{}{10}{9}\right)+\left(\genfrac{}{}{0ex}{}{10}{10}\right)}{1024}=\frac{45+10+1}{1024}=\frac{56}{1024}=\frac{7}{128}\approx 0.055$, two-sided about $0.11$: not significant, which is the honest reading of eight positives in ten.

“Distribution-free” promises only that the null distribution of the statistic can be computed without knowing the population — for the sign test under any continuous distribution with median zero, for the signed-rank test under any distribution symmetric about zero. It does not mean assumption-free, and the assumption a rank test does make is exactly what fixes which null distribution one is entitled to use.

(a) Exactness. Under the null the $n_1+n_2$ pooled observations are independent draws from one distribution, so their joint distribution is unchanged by any permutation of the labels: the observations are exchangeable. This is the only place the null hypothesis enters. Condition on the multiset of pooled values. Given that multiset, each of the $N$ ways of choosing which $n_1$ values carry the first label has the same conditional probability $1/N$, so the observed labelling is a uniform draw from the $N$ relabellings, and $T_{\text{obs}}$ is a uniform draw from the $N$ distinct values $t_{(1)}<t_{(2)}<\cdots <t_{(N)}$ that $T$ takes.

If $T_{\text{obs}}=t_{(N-k+1)}$, the $k$-th largest value, then exactly $k$ relabellings have $T\ge T_{\text{obs}}$ and $p=k/N$. Hence $p$ is uniform on $\{\frac{1}{N},\frac{2}{N},\dots ,1\}$, and

$$P(p\le \alpha )=P\left(k\le \alpha N\right)=\frac{\#\{k\in \{1,\dots ,N\}:k\le \alpha N\}}{N}=\frac{\lfloor \alpha N\rfloor }{N}\le \alpha .$$

This is the conditional probability given the multiset; since it does not depend on the multiset, it is the unconditional probability too. Equality holds precisely when $\alpha N$ is an integer. $■$

So “exact” means: the rejection probability under the null is known exactly, for every sample size, with no approximation — it is $\lfloor \alpha N\rfloor /N$, and it equals $\alpha$ whenever $\alpha$ is one of the attainable levels $k/N$. It does not mean that every $\alpha$ is attainable, and it says nothing about power. Nothing about the form of $T$ was used, which is why any statistic may be permuted.

(b) The floor. The smallest $p$ occurs when $T_{\text{obs}}$ is the largest of the $N$ values, giving $p=1/N$; no relabelling can have $p$ smaller, since the observed one always counts. For $n_1=n_2=m$, $N=\left(\genfrac{}{}{0ex}{}{2m}{m}\right)$ runs $2,6,20,70,252,\dots$ for $m=1,2,3,4,5,\dots$

One-sided at $\alpha =0.05$ requires $\frac{1}{N}\le 0.05$, i.e. $N\ge 20$: $m=2$ gives $N=6$ and fails, $m=3$ gives $N=20$ and succeeds, with $p=\frac{1}{20}=0.05$ exactly. So $m=3$ is the smallest, and only the single most extreme relabelling rejects.

Two-sided with $p$ doubled requires $\frac{2}{N}\le 0.05$, i.e. $N\ge 40$: $m=3$ fails and $m=4$ gives $N=70$, $p=\frac{2}{70}=\frac{1}{35}\approx 0.029$. So $m=4$ is the smallest.

(c) The example. The pooled values are $1,\dots ,6$ and $N=\left(\genfrac{}{}{0ex}{}{6}{3}\right)=20$. Since $\bar{y}-\bar{x}=\frac{(21-s)-s}{3}=\frac{21-2s}{3}$ where $s$ is the first group's sum, $T$ is largest when $s$ is smallest, and the observed labelling $\{1,2,3\}$ has the smallest possible sum $s=6$. It is the unique relabelling with $T\ge T_{\text{obs}}$, so $p=\frac{1}{20}=0.05$, exactly the floor from (b).

Replacing values by ranks. The values $1,\dots ,6$ are their own ranks, so nothing changes and $p=\frac{1}{20}$. More generally, replacing observations by ranks is a strictly increasing transformation, and under it the ordering of relabellings by any statistic that is monotone in the group sum is preserved whenever, as here, the observed group holds the $n_1$ smallest values — that relabelling is the unique extreme one before and after ranking.

Replacing $T$ by $U$. With $W$ the rank sum of the second sample, $U_y=W-\frac{n_2(n_2+1)}{2}=(4+5+6)-6=9$, which equals $n_1{n}_2=9$, the maximum possible: every $y$ exceeds every $x$. Across relabellings $U_y$ is a strictly increasing function of the second group's rank sum, hence a strictly decreasing function of $s$, so it orders the $20$ relabellings exactly as $T$ does, and again exactly one relabelling is at least as extreme: $p=\frac{1}{20}$.

The lesson. The Mann-Whitney test is a permutation test — the permutation test whose statistic is the rank sum, applied to the ranks. Its tabulated null distribution is precisely the permutation distribution of $W$ over the $N$ relabellings, which is why it is the same for every continuous population: ranking has already removed the data's values, so the enumeration can be done once. A permutation test on the raw values keeps the magnitudes and can use any statistic, at the price of redoing the enumeration for each data set. Both are exact for the same reason, exchangeability under the null; they differ in what they throw away, and hence in power against particular alternatives.

(a) Counting. Each of the three positions takes one of three values, so there are $3^3=27$ equally likely ordered resamples. Ignoring order, a resample is a multiset of size $3$ from $3$ types, of which there are $\left(\genfrac{}{}{0ex}{}{3+3-1}{3}\right)=\left(\genfrac{}{}{0ex}{}{5}{3}\right)=10$ — but these are not equally likely. A resample contains all three distinct values in $3!=6$ of the $27$ orderings, so the probability is $\frac{6}{27}=\frac{2}{9}$; the other $\frac{7}{9}$ of resamples repeat at least one observation, which is what makes sampling with replacement produce any variation at all.

(b) The median. The resample median is the middle value of the three draws.

The median is $1$ exactly when at least two of the three draws are $1$: two of them, $\left(\genfrac{}{}{0ex}{}{3}{2}\right){\left(\frac{1}{3}\right)}^2\frac{2}{3}=\frac{6}{27}$, or all three, ${\left(\frac{1}{3}\right)}^3=\frac{1}{27}$. So $P(\text{median}=1)=\frac{7}{27}$, and by the same count $P(\text{median}=8)=\frac{7}{27}$.

The median is $3$ in the remaining cases — at least two $3$s ($\frac{7}{27}$ again) or all three values distinct ($\frac{6}{27}$, from (a)), which are disjoint — giving $P(\text{median}=3)=\frac{13}{27}$.

Check: $\frac{7+13+7}{27}=1$. ✓

(c) Moments of the bootstrap median. Writing $M^{\ast }$ for the resample median,

$$E(M^{\ast })=\frac{7\cdot 1+13\cdot 3+7\cdot 8}{27}=\frac{7+39+56}{27}=\frac{102}{27}=\frac{34}{9},$$

$$E\left(M^{\ast 2}\right)=\frac{7\cdot 1+13\cdot 9+7\cdot 64}{27}=\frac{7+117+448}{27}=\frac{572}{27}.$$

Hence

$$\mathrm{Var}(M^{\ast })=\frac{572}{27}-{\left(\frac{34}{9}\right)}^2=\frac{1716}{81}-\frac{1156}{81}=\frac{560}{81},$$

and the bootstrap standard error of the median is

$${\text{SE}}^{\ast }=\sqrt{\frac{560}{81}}=\frac{\sqrt{16\cdot 35}}{9}=\frac{4\sqrt{35}}{9}\approx \mathrm{2.63.}$$

Note that the bootstrap mean $\frac{34}{9}\approx 3.78$ is above the observed median $3$: the resampling distribution has inherited the right skew of the data, since a resample that picks up the $8$ twice moves the median much further than one that picks up the $1$ twice.

(d) The percentile interval. The bootstrap distribution has only three atoms. Its cumulative probabilities are $P(M^{\ast }\le 1)=\frac{7}{27}\approx 0.26$ and $P(M^{\ast }\le 3)=\frac{20}{27}\approx 0.74$. The $2.5\%$ quantile is the smallest value with cumulative probability at least $0.025$, which is $1$; the $97.5\%$ quantile is the smallest value with cumulative probability at least $0.975$, which is $8$. The $95\%$ percentile interval is $[1,8]$ — the entire range of the data. Indeed every percentile interval wider than the middle $48\%$ is $[1,8]$: the distribution is far too coarse to yield anything else. The bootstrap has not manufactured information three observations do not contain; a resample median is always one of the observed values, so with tiny $n$ the bootstrap distribution of the median is a handful of points and its quantiles are crude.

The mean, without enumeration. A resample mean is the average of three independent draws from the empirical distribution, which puts mass $\frac{1}{3}$ on each of $1,3,8$. That distribution has mean $4$ and variance

$$\frac{(1-4{)}^2+(3-4{)}^2+(8-4{)}^2}{3}=\frac{9+1+16}{3}=\frac{26}{3},$$

so the variance of the average of three independent draws is $\frac{26}{3}\cdot \frac{1}{3}=\frac{26}{9}$ and

$${\text{SE}}^{\ast }({\bar{x}}^{\ast })=\frac{\sqrt{26}}{3}\approx \mathrm{1.70.}$$