Contents / Statistics / Descriptive Statistics

Chapter 1

Descriptive Statistics

Measures of center, spread, shape, and data visualization.

Introduction

A dataset arrives as a list of numbers, and a list of numbers is not yet

knowledge. Descriptive statistics is the discipline of compressing that list

into a few quantities and a picture, in such a way that the compression is

honest: what survives should be the features of the data that matter, and what

is discarded should be the part a reader would not have used anyway.

Two questions organise the whole subject. Where is the data? — a question

about location, answered by the mean, the median, the mode. *How spread out is

it?* — a question about scale, answered by the variance, the standard

deviation, the interquartile range. A third question, what shape is it?,

decides which of the first two answers you are allowed to trust.

This chapter takes those summaries seriously as mathematical objects rather than

as recipes. The mean is not merely "add up and divide"; it is the unique number

that minimises total squared distance to the data, which is exactly why it turns

up wherever squares are minimised. The median is the corresponding minimiser for

absolute distance, which is exactly why it ignores outliers. The sample variance

divides by $n-1$ and not by $n$ for a reason that can be written down and

proved, and we prove it. By the end you should be able to say, of every formula

here, what it optimises and what it assumes.

1.1Populations, Samples and Notation

Almost every confusion in elementary statistics traces back to one blurred

distinction: the difference between the thing you want to know about and the

thing you actually measured.

Definition 1.1 (Population, sample, parameter, statistic). A population is the entire collection of units under study; a sample is a subset of it that we actually observe.

A parameter is a number computed from the whole population. It is a fixed, usually unknown constant — the population mean $\mu$ , the population variance $\sigma^{2}$ .

A statistic is a number computed from the sample. It is random: draw a different sample and you get a different value. The sample mean $\bar{x}$ and sample variance $s^{2}$ are statistics.

The parameter is fixed and unknown; the statistic is known and variable. Every

inferential technique in later chapters is a way of managing that trade. Here we

only need the distinction to keep two formulas apart, but keeping them apart is

worth the effort, because the divisor in the variance depends on which of the

two you are computing.

Notation (Parameters versus statistics). Throughout this chapter and the rest of the subject:

quantity	population (parameter)	sample (statistic)
size	$N$	$n$
mean	$\mu$	$\bar{x}$
variance	$\sigma^{2}$	$s^{2}$
standard deviation	$\sigma$	$s$
proportion	$p$	$\hat{p}$

Greek letters for parameters, Roman letters (often with a bar or a hat) for statistics. A capital $X_i$ denotes the $i$ -th observation regarded as a random variable, before it is observed; a lower-case $x_i$ denotes the number it turned out to be. The distinction matters in a statement like $E[\bar{X}] = \mu$ , which is a claim about the random variable $\bar{X}$ and is meaningless for the observed number $\bar{x}$ .

We will use one modelling assumption repeatedly, and it deserves a name.

Definition 1.2 (Random sample). $X_1, \dots, X_n$ is a random sample (equivalently, the $X_i$ are i.i.d.) from a distribution with mean $\mu$ and variance $\sigma^{2}$ if the $X_i$ are independent of one another and each has that same distribution, so that $E[X_i] = \mu$ and $\operatorname{Var}(X_i) = \sigma^{2}$ for every $i$ .

Notice what this does not assume: nothing about normality, nothing about

symmetry, nothing about the shape of the distribution at all. Only independence

and a common distribution with a finite variance. Most of the theorems in this

chapter need exactly that much and no more, and it is worth watching which ones

need more.

Definition 1.3 (Estimator, bias, unbiasedness). An estimator of a parameter $\theta$ is a statistic $\hat{\theta}$ used to guess it. Its bias is

\operatorname{Bias}(\hat\theta) = E[\hat{\theta}] - \theta,

and $\hat\theta$ is unbiased when the bias is zero, i.e. $E[\hat{\theta}] = \theta$ .

Intuition. Unbiasedness is a statement about the long run, not about your particular sample. It says: if you could repeat the sampling forever and average all the estimates you got, you would land exactly on the truth. It does not say your one estimate is close. An estimator can be unbiased and wildly noisy, or slightly biased and far more reliable. Unbiasedness is one desirable property among several, not a certificate of quality.

Exercises for §1.1 →

1.2Mean

Definition 1.4 (Population mean and sample mean). The population mean of $N$ values $x_1,\dots,x_N$ is

\mu = \frac{1}{N}\sum_{i=1}^{N} x_i,

and the sample mean of $n$ observed values is

\bar{x} = \frac{1}{n}\sum_{i=1}^{n} x_i.

The two formulas are identical; only the set being averaged differs.

The mean uses every data point, with equal weight. That is its great strength —

no observation is thrown away, and the algebra that follows is clean — and its

one great weakness: a single extreme value gets the same vote as every other, so

it can drag the mean far from anything typical.

Intuition. Picture the values as unit weights placed along a rod. The mean is the point where the rod balances. Slide one weight far to the right and the balance point must move right too, even though every other weight stayed put. That is the whole story of the mean's sensitivity to outliers, and it is literally true: the identity $\sum (x_i - \bar{x}) = 0$ below is the statement that the moments about $\bar{x}$ cancel.

The balance property is the first thing to prove, because everything else in the

section leans on it.

Proposition 1.5 (Deviations from the mean sum to zero). For any values $x_1,\dots,x_n$ with mean $\bar{x}$ ,

\sum_{i=1}^{n} (x_i - \bar{x}) = 0.

Proof. Split the sum and use that $\bar{x}$ does not depend on $i$ :

\sum_{i=1}^n (x_i - \bar x) = \sum_{i=1}^n x_i - n\bar x = n\bar x - n\bar x = 0,

where the middle step is just the definition $\sum x_i = n\bar{x}$ rearranged.∎

This is why raw deviations are useless as a measure of spread: they always

cancel, for every dataset, tight or scattered. Any workable measure of spread

must first destroy the signs, by squaring (variance) or by taking absolute

values (mean absolute deviation).

The mean as a minimiser

The following theorem is the real definition of the mean. It explains, in one

line, why least-squares regression produces means, why the variance is defined

around $\bar{x}$ rather than around anything else, and why the mean is the

natural centre whenever the cost of being wrong grows with the square of the

error.

Theorem 1.6 (The mean minimises total squared deviation). For fixed data $x_1,\dots,x_n$ , define $S(c) = \sum_{i=1}^n (x_i - c)^2$ . Then $S$ is minimised uniquely at $c = \bar{x}$ , and

S(c) = S(\bar{x}) + n(c-\bar{x})^2.

Proof. Insert $\bar{x}$ and expand, writing $x_i - c = (x_i - \bar x) + (\bar x - c)$ :

S(c) = \sum_{i=1}^n \bigl[(x_i-\bar x) + (\bar x - c)\bigr]^2 = \sum_{i=1}^n (x_i-\bar x)^2 + 2(\bar x - c)\sum_{i=1}^n (x_i - \bar x) + n(\bar x - c)^2.

The middle term vanishes by Proposition Deviations from the mean sum to zero — this is exactly where that identity earns its keep. What remains is

S(c) = S(\bar x) + n(c - \bar x)^2.

The second term is non-negative and equals zero only when $c = \bar{x}$ , so $S(c) > S(\bar{x})$ for every $c \neq \bar{x}$ .∎

Remark. The same computation done with calculus is shorter but says less: $S'(c) = -2\sum(x_i - c) = 0$ gives $c = \bar{x}$ , and $S''(c) = 2n > 0$ confirms a minimum. The algebraic version is better because it produces the exact excess $n(c-\bar{x})^2$ incurred by centring at the wrong place, an identity we use again when proving why the sample variance needs $n-1$ .

Linear transformations

Changing units — Celsius to Fahrenheit, dollars to thousands of dollars — is a

linear transformation $y_i = a x_i + b$ . Summaries must transform predictably or

they would be useless.

Proposition 1.7 (Mean under a linear transformation). If $y_i = a x_i + b$ for constants $a, b$ and all $i$ , then $\bar{y} = a\bar{x} + b$ .

Proof. Directly from the definition,

\bar y = \frac1n \sum_{i=1}^n (a x_i + b) = \frac{a}{n}\sum_{i=1}^n x_i + \frac{1}{n}\cdot nb = a\bar x + b. \qquad

∎

So the mean is equivariant: it moves with a shift and scales with a scale.

This is the property the median also has, and it is what lets us standardise

data later without worrying about units.

Weighted and combined means

Definition 1.8 (Weighted mean). Given weights $w_i > 0$ attached to values $x_i$ , the weighted mean is

\bar{x}_w = \frac{\sum_{i=1}^n w_i x_i}{\sum_{i=1}^n w_i}.

The ordinary mean is the case $w_i = 1$ for every $i$ . Weights are the right

tool when observations carry different amounts of information — a course grade

where assignments are worth different point totals, a survey where respondents

represent different numbers of people.

Proposition 1.9 (Combined mean of two groups). If group $A$ has $n_A$ values with mean $\bar{x}_A$ and group $B$ has $n_B$ values with mean $\bar{x}_B$ , then the mean of the pooled data is

\bar{x} = \frac{n_A \bar{x}_A + n_B \bar{x}_B}{n_A + n_B}.

Proof. The pooled sum is the sum of the two sums, and each group's sum is its size times its mean: $\sum_A x_i = n_A \bar{x}_A$ and $\sum_B x_i = n_B\bar{x}_B$ . Dividing the total $n_A\bar{x}_A + n_B\bar{x}_B$ by the total count $n_A + n_B$ gives the claim.∎

The pooled mean is therefore a weighted mean of the group means, with the group

sizes as weights. Averaging the two means directly is correct only when the

groups are the same size.

Pitfall. Averaging averages is one of the most common errors in applied work, and it does not merely lose precision — it can reverse a conclusion. If a small group has a high mean and a large group a low one, the naive average of the two means sits far above the true pooled mean. The same mechanism, applied to rates rather than means, produces Simpson's paradox.

Definition 1.10 (Trimmed mean). A $100\alpha\%$ trimmed mean discards the lowest $\alpha$ fraction and the highest $\alpha$ fraction of the sorted values and averages what remains.

Trimming interpolates between the two extremes: $\alpha = 0$ gives the ordinary

mean, and pushing $\alpha$ toward $1/2$ gives the median. A $10\%$ trimmed mean

keeps most of the mean's efficiency on clean data while refusing to be dragged

by a handful of extreme values.

The sample mean as an estimator

So far $\bar{x}$ has been a descriptive number. Regarded as an estimator of an

unknown $\mu$ , it has two properties that make it the default choice, and both

have short proofs. Note carefully which hypothesis each one needs.

Theorem 1.11 (The sample mean is unbiased). Let $X_1,\dots,X_n$ be a random sample from a distribution with mean $\mu$ . Then

E[\bar{X}] = \mu.

Only $E[X_i] = \mu$ and linearity of expectation are used; independence is not needed.

Proof. By linearity of expectation, which holds whether or not the $X_i$ are independent,

E[\bar X] = E\!\left[\frac1n\sum_{i=1}^n X_i\right] = \frac1n\sum_{i=1}^n E[X_i] = \frac1n \cdot n\mu = \mu. \qquad

∎

Theorem 1.12 (Variance of the sample mean). If in addition the $X_i$ are independent, each with variance $\sigma^{2} < \infty$ , then

\operatorname{Var}(\bar{X}) = \frac{\sigma^2}{n}, \qquad \text{so} \qquad \operatorname{SD}(\bar{X}) = \frac{\sigma}{\sqrt{n}}.

Proof. For a constant $a$ , $\operatorname{Var}(aX) = a^{2}\operatorname{Var}(X)$ , and for independent summands the variance of a sum is the sum of the variances. Hence

\operatorname{Var}(\bar X) = \frac{1}{n^2}\operatorname{Var}\!\left(\sum_{i=1}^n X_i\right) = \frac{1}{n^2}\sum_{i=1}^n \operatorname{Var}(X_i) = \frac{n\sigma^2}{n^2} = \frac{\sigma^2}{n}. \qquad

∎

Here independence is doing genuine work: with correlated observations the cross

terms $2\sum_{i<j}\operatorname{Cov}(X_i,X_j)$ do not vanish, and a positively

correlated sample has a sample mean far noisier than $\sigma/\sqrt{n}$ suggests.

This is why cluster sampling needs its own variance formulas, and why treating

$100$ measurements on $10$ subjects as $100$ independent observations

understates the uncertainty.

Intuition. The $\sqrt{n}$ is the single most consequential fact in applied statistics. Precision improves with the square root of effort: to halve the noise in an estimate you must quadruple the sample. Every cost–benefit argument about sample size is an argument about this square root.

Example 1.13 (Mean, weighted mean, and units). Five service times, in minutes, are $\{3, 7, 7, 2, 9\}$ . Find the mean. Then find the mean of the same times recorded in seconds.

Solution.

Sum: $3+7+7+2+9 = 28$ , so $\bar{x} = 28/5 = 5.6$ minutes.
Seconds are $y_i = 60 x_i$ , a linear transformation with $a = 60$ , $b = 0$ . By Proposition Mean under a linear transformation, $\bar{y} = 60(5.6) = 336$ seconds.
Check: $5.6$ lies between the minimum $2$ and maximum $9$ , as any mean must, and $336$ seconds is $5.6$ minutes. ✓

□

Example 1.14 (Combined mean). Group $A$ has $n_A = 10$ values with mean $15$ ; group $B$ has $n_B = 20$ values with mean $25$ . Find the pooled mean.

Solution.

Total sum $= 10(15) + 20(25) = 150 + 500 = 650$ .
Pooled mean $= 650/30 = 65/3 \approx 21.67$ .
Check: the naive average of the means is $20$ , but $B$ is twice as large, so the truth is pulled toward $B$ 's mean of $25$ . Indeed $21.67 > 20$ . ✓

□

Example 1.15 (Using the minimising property). A dataset of $n = 8$ values has $\bar{x} = 5$ and $\sum (x_i - 5)^2 = 32$ . Without the raw data, find $\sum (x_i - 7)^2$ .

Solution.

By Theorem The mean minimises total squared deviation, $S(c) = S(\bar{x}) + n(c - \bar{x})^2$ .
With $c = 7$ : $S(7) = 32 + 8(7-5)^2 = 32 + 32 = 64$ .
Check: $S(7) > S(5)$ , as it must be, since $5$ is the unique minimiser. ✓

□

Exercises for §1.2 →

1.3Median and Mode

Definition 1.16 (Median). Sort the data as $x_{(1)} \le x_{(2)} \le \dots \le x_{(n)}$ . The median is

\text{med} = \begin{cases} x_{((n+1)/2)}, & n \text{ odd},\\[4pt] \tfrac12\bigl(x_{(n/2)} + x_{(n/2+1)}\bigr), & n \text{ even}.\end{cases}

When $n$ is even any number between the two middle values splits the data

equally, so the median is not unique as a splitting point; taking the midpoint

is a convention chosen to make the median a continuous function of the data.

Intuition. Line everyone up from smallest to largest and ask what the person in the middle has. Doubling the richest person's fortune moves the mean but does not move the middle person at all. That insensitivity is not a defect to be corrected — it is the entire point of the median.

The median has its own minimisation property, exactly parallel to the mean's,

with squares replaced by absolute values. This one theorem explains every

robustness claim made about the median.

Theorem 1.17 (The median minimises total absolute deviation). For fixed data $x_1,\dots,x_n$ , the function $A(c) = \sum_{i=1}^n |x_i - c|$ is minimised at any median of the data. If $n$ is odd the minimiser is unique and equals $x_{((n+1)/2)}$ ; if $n$ is even every $c$ in $[x_{(n/2)}, x_{(n/2+1)}]$ is a minimiser.

Proof. $A$ is a sum of convex functions, hence convex and continuous, and it is piecewise linear with kinks only at the data values. So it suffices to examine how $A$ changes as $c$ moves.

Take $c$ not equal to any data value, and let $L(c)$ be the number of $x_i < c$ and $R(c)$ the number with $x_i > c$ , so $L + R = n$ . Increasing $c$ by a small $h > 0$ that crosses no data value increases each of the $L$ terms with $x_i < c$ by exactly $h$ and decreases each of the $R$ terms by exactly $h$ :

A(c+h) - A(c) = h\bigl(L(c) - R(c)\bigr).

Thus $A$ is strictly decreasing while $L < R$ (more points still to the right) and strictly increasing once $L > R$ . The minimum is therefore attained where the counts on the two sides balance, which is precisely the median.

Concretely, for $n$ odd, $L < R$ for every $c < x_{((n+1)/2)}$ and $L > R$ for every $c > x_{((n+1)/2)}$ , so the unique minimiser is that middle order statistic. For $n$ even, $L = R = n/2$ throughout the interval $(x_{(n/2)}, x_{(n/2+1)})$ , so $A$ has slope zero there and every point of the closed interval is a minimiser.∎

Remark (Why the two theorems explain everything). Squared loss punishes one huge error far more than many small ones, so the minimiser must chase an outlier to keep its squared distance down. Absolute loss charges an outlier in proportion to its distance, and moving the centre toward it costs exactly as much on the other side; the trade is a wash, so the minimiser does not move. Robustness is not a property bolted onto the median — it is a consequence of the loss function it minimises.

Definition 1.18 (Mode). The mode is the most frequently occurring value (for discrete or categorical data) or the location of a peak of the density (for continuous data). A distribution is unimodal, bimodal, or multimodal according to the number of peaks.

The mode is the only one of the three centres that makes sense for purely

categorical data: you cannot sort eye colours, so there is no median, and you

certainly cannot average them.

Pitfall. A bimodal distribution usually signals two mixed subpopulations — two machines, two age groups, two experimental conditions. Reporting a single mean for it describes a value that may occur in neither group. Always look at the shape before quoting a centre; this is the one case where no single number is an honest summary.

Example 1.19 (Median, odd and even counts). Find the median of $\{3, 7, 7, 2, 9\}$ and of $\{2, 4, 6, 8\}$ .

Solution.

Sorted: $2, 3, 7, 7, 9$ . With $n=5$ odd the median is the $3$ rd value, $7$ .
Sorted: $2,4,6,8$ . With $n=4$ even the median is $(4+6)/2 = 5$ .
Check: with an even count the median need not be an observed value; here $5$ is not in the data, which is fine. ✓

□

Example 1.20 (Mean versus median under an outlier). Twelve help-desk service times (minutes) are $\{2, 5, 6, 7, 8, 9, 10, 11, 12, 13, 15, 42\}$ , already sorted. Compute both centres, then recompute after deleting the $42$ .

Solution.

Sum $= 140$ , so $\bar{x} = 140/12 = 35/3 \approx 11.67$ .
With $n = 12$ even, the median is $(x_{(6)} + x_{(7)})/2 = (9+10)/2 = 9.5$ .
Deleting the $42$ : the sum falls to $98$ over $11$ values, so $\bar{x} \approx 8.91$ — a shift of $2.76$ minutes. The new median is the $6$ th of $11$ values, $9$ — a shift of $0.5$ .
Check: one observation out of twelve moved the mean by more than five times as much as it moved the median. ✓

□

Exercises for §1.3 →

1.4Variance and Standard Deviation

The crudest measure of spread is the range, $\max - \min$ . It uses two

observations out of $n$ , throws away everything between them, and is determined

entirely by the two least typical values in the dataset. It is worth quoting and

never worth relying on.

Since deviations from the mean sum to zero, a useful measure must remove the

signs. Squaring is the choice that keeps the algebra differentiable and connects

to the minimisation theorem above.

Definition 1.21 (Variance and standard deviation). For a population of $N$ values with mean $\mu$ ,

\sigma^2 = \frac{1}{N}\sum_{i=1}^{N}(x_i - \mu)^2, \qquad \sigma = \sqrt{\sigma^2}.

For a sample of $n$ values with mean $\bar{x}$ ,

s^2 = \frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})^2, \qquad s = \sqrt{s^2}.

The divisor $n-1$ is Bessel's correction; the next section proves why it is there. The standard deviation restores the original units of the data, which the variance — measured in units squared — does not.

Intuition. The standard deviation is roughly "the typical distance from the mean". It is not exactly the average distance (that would be the mean absolute deviation), because squaring weights large deviations more heavily, but it is the right order of magnitude and it is the number the empirical rule and every normal-theory method are written in terms of.

For a random variable the same definition reads

$\operatorname{Var}(X) = E[(X-\mu)^2]$ , and it has a computational form that

saves a pass through the data and is used constantly in later chapters.

Theorem 1.22 (Computational formula for the variance). For a random variable $X$ with finite second moment and $\mu = E[X]$ ,

\operatorname{Var}(X) = E[X^2] - (E[X])^2.

In particular $E[X^{2}] \ge (E[X])^2$ , with equality only if $X$ is constant.

Proof. Expand the square inside the expectation and use linearity, remembering that $\mu$ is a constant so $E[\mu X] = \mu E[X]$ and $E[\mu^{2}] = \mu^{2}$ :

E[(X-\mu)^2] = E[X^2 - 2\mu X + \mu^2] = E[X^2] - 2\mu E[X] + \mu^2 = E[X^2] - 2\mu^2 + \mu^2 = E[X^2] - \mu^2.

Since $\operatorname{Var}(X) = E[(X-\mu)^2] \ge 0$ as the expectation of a non-negative quantity, $E[X^{2}] \ge \mu^{2}$ ; equality forces $E[(X-\mu)^2] = 0$ , which for a non-negative random variable means $X = \mu$ with probability one.∎

Corollary 1.23 (Computational formula for a sample). For any values $x_1,\dots,x_n$ with mean $\bar{x}$ ,

\sum_{i=1}^n (x_i - \bar{x})^2 = \sum_{i=1}^n x_i^2 - n\bar{x}^2 = \sum_{i=1}^n x_i^2 - \frac{\bigl(\sum_i x_i\bigr)^2}{n}.

Proof. Expand: $\sum (x_i - \bar x)^2 = \sum x_i^{2} - 2\bar x \sum x_i + n\bar x^{2}$ . Substituting $\sum x_i = n\bar{x}$ turns the middle term into $-2n\bar{x}^2$ , so the last two terms combine to $-n\bar{x}^2$ . The final form follows from $\bar{x} = (\sum x_i)/n$ .∎

Pitfall. The computational formula is exact in exact arithmetic and treacherous in floating point. When the mean is large compared with the spread — timestamps, years, sensor readings offset from zero — $\sum x_i^{2}$ and $n\bar{x}^2$ are two huge nearly equal numbers, and subtracting them destroys most of the significant digits. It can even return a negative variance. Use it for hand computation and for algebra; use the two-pass or Welford formula in code.

Linear transformations of the variance

Proposition 1.24 (Variance under a linear transformation). If $y_i = a x_i + b$ for constants $a, b$ , then

s_y^2 = a^2 s_x^2, \qquad s_y = |a|\, s_x.

In particular a shift ( $a=1$ ) leaves the spread unchanged, and the same holds for $\sigma^{2}$ and for $\operatorname{Var}(aX+b) = a^{2}\operatorname{Var}(X)$ .

Proof. By Proposition Mean under a linear transformation, $\bar{y} = a\bar{x} + b$ . Hence each deviation transforms as

y_i - \bar y = (a x_i + b) - (a\bar x + b) = a(x_i - \bar x),

the $b$ cancelling exactly. Squaring gives $(y_i - \bar y)^2 = a^{2} (x_i-\bar x)^2$ , and summing and dividing by the same $n-1$ gives $s_y^{2} = a^{2} s_x^{2}$ . Taking square roots, and noting a standard deviation is non-negative, gives $s_y = |a| s_x$ .∎

Remark. The absolute value is not decoration. Multiplying data by $-3$ reflects and stretches it; the spread triples, so $s_y = 3 s_x$ , not $-3 s_x$ . A negative standard deviation would be meaningless.

Comparing spreads across scales

Definition 1.25 (Coefficient of variation). For data with $\bar{x} > 0$ , the coefficient of variation is

\mathrm{CV} = \frac{s}{\bar{x}},

usually quoted as a percentage. It is dimensionless, since $s$ and $\bar{x}$ carry the same units.

Being dimensionless, the CV compares variability across datasets measured in

different units or on wildly different scales. It requires a meaningful zero and

a positive mean — it is a ratio-scale tool, and quoting a CV for temperatures in

Celsius is nonsense, because shifting to Fahrenheit changes it.

Example 1.26 (Sample variance two ways). For $\{2, 4, 4, 4, 5, 5, 7, 9\}$ ( $n = 8$ ), find $s^{2}$ and $s$ using the definition, then confirm with the computational formula.

Solution.

Mean: $\sum x_i = 40$ , so $\bar{x} = 5$ .
Deviations: $-3,-1,-1,-1,0,0,2,4$ ; squares $9,1,1,1,0,0,4,16$ summing to $32$ .
$s^{2} = 32/7 \approx 4.571$ and $s = \sqrt{32/7} \approx 2.138$ .
Computational check: $\sum x_i^{2} = 4+16+16+16+25+25+49+81 = 232$ and $n\bar{x}^2 = 8(25) = 200$ , so $\sum(x_i-\bar{x})^2 = 232 - 200 = 32$ . ✓
Had this been a whole population we would divide by $8$ : $\sigma^{2} = 4$ .

□

Example 1.27 (Rescaling a summary). A dataset has $\bar{x} = 10$ and $s = 3$ . Give the mean and standard deviation of $y_i = -2x_i + 7$ .

Solution.

Mean: $\bar{y} = -2(10) + 7 = -13$ .
Standard deviation: $s_y = |-2|\cdot 3 = 6$ , and $s_y^{2} = 36$ .
Check: the shift by $7$ affected only the mean, and the sign of $a$ affected only the mean too — the spread depends on $|a|$ , not on $a$ . ✓

□

Example 1.28 (Coefficient of variation). Heights have $\bar{x} = 170$ cm with $s = 8.5$ cm; weights have $\bar{x} = 70$ kg with $s = 8.4$ kg. Which is more variable?

Solution.

Heights: $\mathrm{CV} = 8.5/170 = 0.05 = 5\%$ .
Weights: $\mathrm{CV} = 8.4/70 = 0.12 = 12\%$ .
The raw standard deviations are nearly equal, but relative to their own scales weights vary more than twice as much. Comparing $8.5$ cm with $8.4$ kg directly would have been meaningless. ✓

□

Exercises for §1.4 →

1.5Why the Sample Variance Divides by $n-1$

This is the single most-asked question in the chapter, and it has a complete

answer. The claim is not that $n-1$ is more cautious, or that it "corrects for

small samples", or that it makes the answer bigger. The claim is exact:

dividing by $n-1$ makes $s^{2}$ an unbiased estimator of $\sigma^{2}$ , and dividing

by $n$ does not.

The source of the problem is that $\sigma^{2}$ is defined as a spread about the

true mean $\mu$ , but we can only measure spread about the sample mean

$\bar{x}$ — and by Theorem The mean minimises total squared deviation,

$\bar{x}$ is the one centre that makes that sum of squares as small as it can

possibly be. Measuring spread about the minimiser systematically understates it.

The next lemma computes the shortfall exactly.

Lemma 1.29 (Expected sum of squares about the sample mean). Let $X_1,\dots,X_n$ be a random sample from a distribution with mean $\mu$ and finite variance $\sigma^{2}$ , and let $\bar{X}$ be the sample mean. Then

E\!\left[\sum_{i=1}^n (X_i - \bar{X})^2\right] = (n-1)\sigma^2.

Independence is required: the proof uses $\operatorname{Var}(\bar X) = \sigma^{2}/n$ .

Proof. Apply the identity of Theorem The mean minimises total squared deviation with $c = \mu$ , read in the direction $S(\mu) = S(\bar X) + n(\mu - \bar X)^2$ , that is

\sum_{i=1}^n (X_i - \bar X)^2 = \sum_{i=1}^n (X_i - \mu)^2 - n(\bar X - \mu)^2.

This is an algebraic identity, true sample by sample. Now take expectations of both sides and use linearity.

For the first term, $E[(X_i - \mu)^2] = \operatorname{Var}(X_i) = \sigma^{2}$ for each $i$ , so it contributes $n\sigma^{2}$ .

For the second, $E[\bar X] = \mu$ by Theorem The sample mean is unbiased, so $E[(\bar X - \mu)^2] = \operatorname{Var}(\bar X) = \sigma^{2}/n$ by Theorem Variance of the sample mean. That term contributes $n \cdot \sigma^{2}/n = \sigma^{2}$ .

Therefore

E\!\left[\sum_{i=1}^n (X_i-\bar X)^2\right] = n\sigma^2 - \sigma^2 = (n-1)\sigma^2. \qquad

∎

Theorem 1.30 (Unbiasedness of the sample variance). Under the same hypotheses — a random sample from a distribution with finite variance $\sigma^{2}$ —

E[s^2] = E\!\left[\frac{1}{n-1}\sum_{i=1}^n (X_i-\bar X)^2\right] = \sigma^2.

No assumption of normality is needed.

Proof. Pull the constant $1/(n-1)$ out of the expectation and apply Lemma Expected sum of squares about the sample mean:

E[s^2] = \frac{1}{n-1}\,E\!\left[\sum_{i=1}^n (X_i - \bar X)^2\right] = \frac{(n-1)\sigma^2}{n-1} = \sigma^2. \qquad

∎

Corollary 1.31 (The n-divisor estimator is biased downward). Let $\hat\sigma^{2} = \frac1n \sum_{i=1}^n (X_i - \bar X)^2$ . Then

E[\hat\sigma^2] = \frac{n-1}{n}\,\sigma^2, \qquad \operatorname{Bias}(\hat\sigma^2) = -\frac{\sigma^2}{n}.

Proof. $\hat\sigma^{2} = \frac{n-1}{n}s^{2}$ , so $E[\hat\sigma^{2}] = \frac{n-1}{n}E[s^{2}] = \frac{n-1}{n}\sigma^{2}$ by Theorem Unbiasedness of the sample variance. Subtracting $\sigma^{2}$ gives the bias $-\sigma^{2}/n$ .∎

So the $n$ -divisor version underestimates the variance by a factor $(n-1)/n$ , on

average, every time. At $n = 2$ it is off by half; at $n = 100$ by $1\%$ ; as

$n \to \infty$ the bias vanishes, which is why the distinction stops mattering

for large samples and matters enormously for small ones.

Intuition (Degrees of freedom). The $n$ deviations $x_i - \bar{x}$ are not $n$ free numbers. They satisfy one linear constraint, $\sum (x_i - \bar{x}) = 0$ from Proposition Deviations from the mean sum to zero: knowing any $n-1$ of them determines the last. So the deviation vector carries only $n-1$ independent pieces of information about spread, and the honest thing is to average over $n-1$ of them, not $n$ .

This is the same bookkeeping that gives $n-1$ degrees of freedom to a one-sample $t$ -test, $n-2$ to a simple regression (two parameters estimated), and $n-p$ in general. Every estimated parameter costs one degree of freedom, and the cost is exactly the shrinkage this section quantifies.

Pitfall. Unbiasedness does not pass through a square root. Since $\sqrt{\cdot}$ is strictly concave, Jensen's inequality gives $E[s] = E[\sqrt{s^{2}}] < \sqrt{E[s^{2}]} = \sigma$ whenever $s^{2}$ is genuinely random, so $s$ underestimates $\sigma$ on average even though $s^{2}$ is unbiased for $\sigma^{2}$ . Nobody corrects for this in practice — the correction factor depends on the distribution — but "we use $n-1$ so the standard deviation is unbiased" is a false statement.

Example 1.32 (Bias of the two divisors). A population has $\sigma^{2} = 40$ . Samples of size $n = 5$ are drawn repeatedly and $\hat\sigma^{2} = \frac15\sum(x_i-\bar{x})^2$ is computed each time. What value does $\hat\sigma^{2}$ average to, and what is its bias?

Solution.

By Corollary The n-divisor estimator is biased downward, $E[\hat\sigma^{2}] = \frac{n-1}{n}\sigma^{2} = \frac45 (40) = 32$ .
Bias $= 32 - 40 = -8$ , which agrees with $-\sigma^{2}/n = -40/5 = -8$ .
Check: the bias is negative, as it must be, since the sum of squares is taken about the minimising centre $\bar{x}$ . ✓

□

Example 1.33 (Recovering the sum of squares). A sample of $n = 16$ has $s = 6$ . Find $\sum (x_i - \bar{x})^2$ and the value the $n$ -divisor formula would have reported for the variance.

Solution.

$s^{2} = 36$ and $\sum (x_i-\bar{x})^2 = (n-1)s^{2} = 15(36) = 540$ .
The $n$ -divisor version gives $540/16 = 33.75$ .
Check: $33.75 = \frac{15}{16}(36)$ , matching the factor $(n-1)/n$ . ✓

□

Exercises for §1.5 →

1.6Percentiles, Quartiles and the IQR

Location and spread can also be described positionally, by where a value falls

in the sorted order rather than by how far it is from a centre. Positional

summaries are the natural partners of the median, and they inherit its

robustness.

Definition 1.34 (Percentile and quantile). The $p$ -th percentile of a dataset is a value below which approximately $p\%$ of the observations fall. Equivalently, the $q$ -quantile with $q = p/100$ is a value $x_q$ with at least a fraction $q$ of the data at or below it and at least a fraction $1-q$ at or above it.

The quartiles are $Q_1$ (the $25$ th percentile), $Q_2$ (the median), and $Q_3$ (the $75$ th percentile). The five-number summary is $\{\min, Q_1, Q_2, Q_3, \max\}$ .

The word approximately is not laziness. For finite data no value in general

has exactly $25\%$ below it, and different software resolves the ambiguity

differently — there are nine commonly implemented quantile definitions. They

agree to within one observation and disagree in the last decimal place; this

chapter uses the median-of-halves convention throughout.

Method 1.35 (Quartiles by the median-of-halves rule).

Sort the data.
Find the median $Q_2$ .
Form the lower half: all values strictly below the median's position. If $n$ is odd, exclude the median value itself.
Form the upper half the same way.
$Q_1$ is the median of the lower half; $Q_3$ is the median of the upper half.

Definition 1.36 (Interquartile range). The interquartile range is $\mathrm{IQR} = Q_3 - Q_1$ : the width of the interval containing the middle half of the data.

The IQR is to the median what the standard deviation is to the mean — a measure

of scale matched to its centre. It is determined entirely by the middle of the

data, so moving the largest observation to infinity does not change it at all,

whereas it sends $s$ to infinity too.

The box plot

The five-number summary is exactly what a box plot draws.

Figure 1.37. A box plot of the twelve service times

\{2,5,6,7,8,9,10,11,12,13,15,42\}

. The box spans

Q_1 = 6.5

Q_3 = 12.5

with the median at

9.5

; whiskers reach the most extreme values inside the upper fence

Q_3 + 1.5\,\mathrm{IQR} = 21.5

, and the value

42

is plotted separately as an outlier.

Read off the figure what the box plot is good at and what it hides. It shows

centre, spread, skew (the median sits left of centre in the box, and the right

whisker is longer) and outliers, in a form compact enough to stack a dozen

groups side by side. What it cannot show is shape: a bimodal dataset and a

uniform one can produce identical box plots, because five numbers do not

determine a distribution.

Where the 1.5 comes from

The outlier rule looks like an arbitrary constant. It is not.

Definition 1.38 (Tukey's fences). A value is a mild outlier if it lies outside

[\,Q_1 - 1.5\,\mathrm{IQR},\; Q_3 + 1.5\,\mathrm{IQR}\,],

and an extreme outlier if it lies outside the same interval with $1.5$ replaced by $3$ .

Proposition 1.39 (What the 1.5 fence flags in normal data). If the data follow a normal distribution with mean $\mu$ and standard deviation $\sigma$ , then $Q_1 = \mu - 0.6745\sigma$ and $Q_3 = \mu + 0.6745\sigma$ , so $\mathrm{IQR} = 1.349\sigma$ and the mild-outlier fences sit at

\mu \pm (0.6745 + 1.5 \times 1.349)\sigma = \mu \pm 2.698\sigma.

The proportion of a normal population outside those fences is about $0.70\%$ .

Proof. For the standard normal, $\Phi^{-1}(0.75) = 0.6745$ to four places, and the distribution is symmetric about $0$ , so $\Phi^{-1}(0.25) = -0.6745$ . An increasing linear map preserves order, so it carries quantiles to quantiles: the $q$ -quantile of $X = \mu + \sigma Z$ is $\mu + \sigma z_q$ , where $z_q$ is the $q$ -quantile of $Z$ . Hence $\mathrm{IQR} = 2(0.6745)\sigma = 1.3490\sigma$ and the upper fence is at $0.6745\sigma + 1.5(1.3490)\sigma = 2.6980\sigma$ above the mean. The tail probability is $2\bigl(1 - \Phi(2.698)\bigr) = 2(0.003488) = 0.006976$ .∎

So $1.5$ is calibrated: on clean normal data the rule flags roughly seven points

in a thousand — rare enough that a flag means something, common enough that the

rule is not vacuous. The $3\,\mathrm{IQR}$ fence sits at $4.72\sigma$ , flagging

about two points in a million.

Pitfall. A flagged point is not an error and must not be deleted on sight. The rule is calibrated for a roughly normal, unimodal distribution; on genuinely skewed data — incomes, waiting times, insurance claims — it flags a large fraction of perfectly valid observations, because the long tail is the distribution rather than a contamination of it. Investigate a flagged point; delete it only if you find an actual reason.

Example 1.40 (Five-number summary and fences). For the twelve service times $\{2,5,6,7,8,9,10,11,12,13,15,42\}$ , give the five-number summary, the IQR, and the mild-outlier fences.

Solution.

Median: $n = 12$ is even, so $Q_2 = (9+10)/2 = 9.5$ .
Lower half $\{2,5,6,7,8,9\}$ has median $(6+7)/2 = 6.5 = Q_1$ .
Upper half $\{10,11,12,13,15,42\}$ has median $(12+13)/2 = 12.5 = Q_3$ .
Five-number summary: $\{2,\ 6.5,\ 9.5,\ 12.5,\ 42\}$ and $\mathrm{IQR} = 12.5 - 6.5 = 6$ .
Fences: $6.5 - 1.5(6) = -2.5$ and $12.5 + 1.5(6) = 21.5$ . Only $42$ falls outside, so it is the sole outlier. It is beyond $12.5 + 3(6) = 30.5$ as well, so it is an extreme outlier.
Check: $Q_1 \le Q_2 \le Q_3$ holds, and the whisker ends at the largest non-outlier, $15$ . ✓

□

Example 1.41 (Locating a percentile). A sorted list has $n = 20$ values. Where does the $90$ th percentile fall?

Solution.

The position index is $L = \frac{p}{100}n = 0.90(20) = 18$ .
$L$ is a whole number, so exactly $18$ values lie below the percentile and it is taken as the midpoint of the $18$ th and $19$ th sorted values.
Check: $90\%$ of $20$ is $18$ , matching the count below. ✓

□

Example 1.42 (IQR versus standard deviation under contamination). In the service-time data, replacing $42$ by $4200$ changes $s$ from about $10.2$ to about $1210$ . What happens to the IQR?

Solution.

The quartiles depend only on the ordered positions $6,7$ within each half. Replacing the largest value by a larger one changes no quartile.
So $\mathrm{IQR} = 6$ exactly as before, a change of zero.
Check: this is the robustness claim made concrete — $s$ grew by a factor of more than a hundred, the IQR did not move. ✓

□

Exercises for §1.6 →

1.7Standard Scores, the Empirical Rule and Chebyshev

A raw value carries units and a scale, so " $137.5$ " means nothing until you know

what is typical. Standardising strips both away.

Definition 1.43 (Standard score). The z-score of a value $x$ from a distribution with mean $\mu$ and standard deviation $\sigma > 0$ is

z = \frac{x - \mu}{\sigma},

the number of standard deviations $x$ sits above the mean. For sample data, $z_i = (x_i - \bar{x})/s$ .

Proposition 1.44 (Standardised data has mean 0 and variance 1). If $z_i = (x_i - \bar{x})/s$ with $s > 0$ , then $\bar{z} = 0$ and $s_z^{2} = 1$ .

Proof. Write $z_i = a x_i + b$ with $a = 1/s$ and $b = -\bar{x}/s$ . By Proposition Mean under a linear transformation,

\bar z = \frac{\bar x}{s} - \frac{\bar x}{s} = 0,

and by Proposition Variance under a linear transformation, $s_z^{2} = a^{2} s^{2} = s^{2}/s^{2} = 1$ .∎

Because standardisation is exactly a linear transformation, everything proved

about linear transformations applies, and the two propositions together say that

standardising puts any dataset on a common footing: mean $0$ , spread $1$ ,

dimensionless. That is what makes two scores on differently-scaled exams

comparable.

Intuition. A z-score is a universal ruler. "Twelve centimetres above average" means nothing without knowing how much heights vary; " $1.5$ standard deviations above average" means the same thing whether the subject is heights, test scores or house prices, and can be compared directly across all three.

Pitfall. A z-score is comparable across distributions but a z-score does not by itself give a percentile. Converting $z = 1.5$ into "the $93$ rd percentile" requires the normal distribution; on a skewed distribution that conversion can be badly wrong. The z-score is always a position in standard-deviation units, and only sometimes a probability statement.

The empirical rule

Proposition 1.45 (The empirical rule (68–95–99.7)). If the data are approximately normal with mean $\mu$ and standard deviation $\sigma$ , then approximately

$68.3\%$ of values lie in $\mu \pm \sigma$ ,
$95.4\%$ lie in $\mu \pm 2\sigma$ ,
$99.7\%$ lie in $\mu \pm 3\sigma$ .

These are not conventions; they are the values of

$\Phi(k) - \Phi(-k)$ at $k = 1, 2, 3$ , computed from the normal density. The

hypothesis of approximate normality is doing all the work, and the rule fails

badly without it — for a strongly right-skewed distribution, far more than

$0.15\%$ of the mass can sit above $\mu + 3\sigma$ .

Figure 1.46. The empirical rule on a normal curve with

\mu = 50

and

\sigma = 10

. The darker central band spans

\mu \pm \sigma = [40,60]

and holds about

68\%

of the area; the full shaded band spans

\mu \pm 2\sigma = [30,70]

and holds about

95\%

Example 1.47 (Applying the empirical rule). Test scores are roughly normal with $\mu = 50$ and $\sigma = 10$ . What fraction of scores lie in $[40,60]$ ? Above $70$ ?

Solution.

$[40,60] = \mu \pm \sigma$ , so about $68\%$ .
$70 = \mu + 2\sigma$ . About $95\%$ lies inside $\mu \pm 2\sigma$ , leaving about $5\%$ split between the two tails; by symmetry about $2.5\%$ lies above $70$ .
Check: the exact normal values are $68.27\%$ and $2.28\%$ , so the rule's rounding is what causes the small discrepancy. ✓

□

Chebyshev's inequality

When normality is not available, a weaker but universal bound is.

Theorem 1.48 (Chebyshev's inequality). Let $X$ be any random variable with finite mean $\mu$ and finite variance $\sigma^{2}$ . For every $k > 0$ ,

P\bigl(|X - \mu| \ge k\sigma\bigr) \le \frac{1}{k^2}, \qquad\text{equivalently}\qquad P\bigl(|X-\mu| < k\sigma\bigr) \ge 1 - \frac{1}{k^2}.

No assumption whatsoever is made about the shape of the distribution.

Proof. First, Markov's inequality: if $Y \ge 0$ and $a > 0$ , then $P(Y \ge a) \le E[Y]/a$ . To see it, note that the random variable $Y - a\,\mathbf{1}\{Y \ge a\}$ is non-negative — when $Y \ge a$ it equals $Y - a \ge 0$ , and otherwise it equals $Y \ge 0$ . Taking expectations, $E[Y] - a\,P(Y \ge a) \ge 0$ , which rearranges to Markov's inequality.

Now apply it to $Y = (X-\mu)^2$ , which is non-negative with $E[Y] = \sigma^{2}$ , and $a = k^{2}\sigma^{2} > 0$ . Since $|X-\mu| \ge k\sigma$ holds exactly when $(X-\mu)^2 \ge k^{2}\sigma^{2}$ ,

P\bigl(|X-\mu|\ge k\sigma\bigr) = P\bigl((X-\mu)^2 \ge k^2\sigma^2\bigr) \le \frac{\sigma^2}{k^2\sigma^2} = \frac{1}{k^2}. \qquad

∎

The bound is vacuous for $k \le 1$ (a probability is always at most $1$ ) and is

deliberately loose for $k > 1$ — it must hold for every distribution with the

stated variance, including the worst one. That worst case is attainable: the

variable taking values $\mu \pm k\sigma$ with probability $1/(2k^{2})$ each and

$\mu$ otherwise has variance $\sigma^{2}$ and achieves equality, so Chebyshev's

inequality cannot be improved without further assumptions.

Example 1.49 (Chebyshev versus the empirical rule). A distribution has $\mu = 100$ and $\sigma = 15$ , with unknown shape. What can be guaranteed about the interval $[70, 130]$ ? What would the empirical rule claim?

Solution.

$[70,130] = \mu \pm 2\sigma$ , so $k = 2$ and Chebyshev guarantees at least $1 - 1/4 = 75\%$ .
If the distribution were known to be normal, the empirical rule would give about $95\%$ .
Check: $75\% \le 95\%$ , as required — the universal guarantee must be weaker than the one that gets to assume a shape. ✓

□

Example 1.50 (Solving for k). For any distribution, within how many standard deviations of the mean must at least $96\%$ of the values lie?

Solution.

Set $1 - 1/k^{2} \ge 0.96$ , i.e. $1/k^{2} \le 0.04$ .
So $k^{2} \ge 25$ and $k \ge 5$ .
Check: at $k=5$ the bound is $1 - 1/25 = 0.96$ exactly. ✓

□

Exercises for §1.7 →

1.8Shape and Visualization

Numerical summaries answer questions you thought to ask. A picture answers the

ones you did not, which is why it should always come first.

Histograms bin a quantitative variable and plot the count or density in each

bin. They show shape: symmetric, skewed, unimodal, bimodal, uniform, heavy

tailed. The one choice they force on you is the bin width, and it matters — too

wide hides structure, too narrow turns the picture into noise.

Box plots draw the five-number summary. They compress a distribution into

five numbers, which makes them ideal for comparing many groups at once and

useless for seeing modality.

Scatter plots show two quantitative variables jointly, and are the tool for

relationship rather than distribution.

Definition 1.51 (Skewness). A distribution is right- (or positively) skewed when its right tail is longer, left- (negatively) skewed when its left tail is longer, and symmetric when the two tails mirror each other. Numerically, the sample skewness is

g_1 = \frac{\frac1n\sum_{i=1}^n (x_i - \bar{x})^3}{s_n^3}, \qquad s_n^2 = \frac1n\sum_{i=1}^n (x_i-\bar{x})^2,

which is positive for right skew and zero for any symmetric distribution.

The cube is what makes this work: it preserves sign, so deviations on the long

side do not cancel those on the short side, and it weights distant points

heavily, so the tail dominates.

Proposition 1.52 (Skew separates the mean from the median). For a right-skewed unimodal distribution one typically has $\text{mode} < \text{median} < \text{mean}$ , and the reverse ordering for left skew.

Pitfall. The word typically is honest, not hedging. The ordering is a reliable rule of thumb, not a theorem: discrete and multimodal counterexamples exist in which a right-skewed distribution has mean below median. Use the ordering to form a hypothesis about shape, then confirm it with a plot.

The mechanism behind the ordering is the mean's sensitivity, already proved: by

Theorem The mean minimises total squared deviation the mean chases squared

distance, so a long tail pulls it; by Theorem *The median minimises total

absolute deviation* the median only counts observations on each side, so the

tail's length is irrelevant to it.

Figure 1.53. A right-skewed distribution of

198

incomes (thousands). The long right tail drags the mean to

\approx 50

while the median stays at

\approx 41

: the gap between the two lines *is* the skew.

Centre is not the whole story

Two distributions can share a mean exactly and behave completely differently,

because the mean says nothing at all about scale. Spread is an independent

dimension of the summary, not a footnote to the centre.

Figure 1.54. Two normal distributions with the identical mean

\mu = 50

but standard deviations

\sigma = 5

and

\sigma = 15

. Every measure of centre agrees on these two datasets; only a measure of spread tells them apart.

Remark (Anscombe's quartet and the Datasaurus). Anscombe's four datasets share their means, variances, correlation and fitted regression line to two decimal places, yet one is linear, one is curved, one is a perfect line with a single outlier, and one is a vertical stack with a lone influential point. The "Datasaurus dozen" pushes the joke further: a dozen datasets with identical summary statistics, one of which is a dinosaur. The moral is not that summaries are useless — it is that summaries are answers to specific questions, and a plot is what tells you which question to ask.

Example 1.55 (Diagnosing skew from two numbers). A dataset of annual incomes has mean $52{,}000$ and median $41{,}000$ . What shape is it, and which summary should be reported?

Solution.

The mean exceeds the median by $11{,}000$ , so by Proposition Skew separates the mean from the median the distribution is right-skewed.
A few very high earners stretch the right tail and drag the mean; the median is the better summary of a typical income, paired with the IQR.
Check: this matches the known shape of income data, and the direction agrees with the histogram above. ✓

□

Example 1.56 (Reading a box plot comparison). Two groups have the same median but group $A$ 's box is twice as wide as group $B$ 's. What do you conclude?

Solution.

Equal medians mean the two groups have the same typical value.
$A$ 's IQR is twice $B$ 's, so $A$ 's middle half is twice as spread out; $A$ is the more variable group.
Check: this is exactly the situation the two-normals figure draws, where centre alone cannot distinguish the distributions. ✓

□

Exercises for §1.8 →

1.9Choosing the Right Summary

Everything above reduces to one decision, made after looking at a plot.

Method 1.57 (Choosing a centre and a spread).

Plot the data first — a histogram for one variable, box plots for several groups.
If the distribution is roughly symmetric and unimodal with no extreme values, report $\bar{x}$ and $s$ . They use all the data and feed directly into the inferential machinery of later chapters.
If it is skewed or contains outliers you cannot justify removing, report the median and the IQR, or the five-number summary.
If it is bimodal, report neither as a single figure. Say so, and summarise the groups separately.
For categorical data, report the mode and the proportions; a mean is undefined.
Always report $n$ alongside any summary. A mean of $7.2$ from $n = 4$ and one from $n = 4000$ are not the same claim, because the precision $\sigma/\sqrt{n}$ differs by a factor of more than thirty.

Summary. Notation. Parameters $\mu, \sigma^{2}, \sigma, p$ describe a population and are fixed; statistics $\bar{x}, s^{2}, s, \hat{p}$ are computed from a sample and are random.

Centres. $\bar{x} = \frac1n\sum x_i$ is the unique minimiser of $\sum (x_i-c)^2$ , with the exact identity $S(c) = S(\bar{x}) + n(c-\bar{x})^2$ ; any median minimises $\sum |x_i - c|$ , uniquely when $n$ is odd. Deviations from the mean always satisfy $\sum(x_i - \bar{x}) = 0$ . The pooled mean of two groups is $\frac{n_A\bar{x}_A + n_B\bar{x}_B}{n_A+n_B}$ , never the average of the means unless $n_A = n_B$ .

Spread. $\sigma^{2} = \frac1N\sum(x_i-\mu)^2$ for a population; $s^{2} = \frac{1}{n-1}\sum(x_i-\bar{x})^2$ for a sample. For a random variable, $\operatorname{Var}(X) = E[X^{2}]-(E[X])^2$ ; for data, $\sum(x_i-\bar{x})^2 = \sum x_i^{2} - n\bar{x}^2$ . Under $y = ax+b$ : $\bar{y} = a\bar{x}+b$ , $s_y^{2} = a^{2} s_x^{2}$ , $s_y = |a| s_x$ .

Why $n-1$ . For a random sample (independent, identically distributed, finite variance — normality not required), $E\bigl[\sum (X_i - \bar{X})^2\bigr] = (n-1)\sigma^{2}$ , hence $E[s^{2}] = \sigma^{2}$ while the $n$ -divisor version has expectation $\frac{n-1}{n}\sigma^{2}$ and bias $-\sigma^{2}/n$ . Unbiasedness does not survive the square root: $E[s] < \sigma$ .

Sampling behaviour of $\bar{X}$ . $E[\bar{X}] = \mu$ needs only a common mean and linearity of expectation; $\operatorname{Var}(\bar{X}) = \sigma^{2}/n$ needs independence and finite variance.

Position. $\mathrm{IQR} = Q_3 - Q_1$ ; Tukey's mild fences are $Q_1 - 1.5\,\mathrm{IQR}$ and $Q_3 + 1.5\,\mathrm{IQR}$ , which on normal data sit at $\mu \pm 2.698\sigma$ and flag about $0.7\%$ of values. The z-score $z = (x-\mu)/\sigma$ standardises to mean $0$ and variance $1$ .

Coverage. The empirical rule ( $68/95/99.7$ within $1/2/3$ standard deviations) requires approximate normality. Chebyshev's inequality, $P(|X-\mu| \ge k\sigma) \le 1/k^{2}$ , requires only a finite variance and is sharp, hence loose.

Exercises for §1.9 →

Common Mistakes

Reporting the mean for heavily skewed data such as incomes or house prices. A right tail drags the mean above anything typical; report the median and IQR.
Averaging two group means without weighting by group size. The pooled mean is $\frac{n_A\bar{x}_A+n_B\bar{x}_B}{n_A+n_B}$ , and the unweighted average is right only when the groups are equally large.
Using the $n$ divisor for a sample variance. It is biased downward by exactly $\sigma^{2}/n$ , which is negligible at $n = 1000$ and catastrophic at $n = 3$ .
Claiming that $n-1$ makes the *standard deviation* unbiased. It does not; Jensen's inequality gives $E[s] < \sigma$ . Only $s^{2}$ is unbiased.
Forgetting to sort the data before reading off a median, a quartile or a percentile.
Quoting the computational formula's result from floating-point code when the mean is large relative to the spread. Catastrophic cancellation can even produce a negative variance.
Treating the standard deviation as a percentage. It carries the units of the data; the dimensionless comparison is the coefficient of variation, and that one requires a meaningful zero and a positive mean.
Applying the empirical rule to non-normal data. Without approximate normality the $68/95/99.7$ figures have no standing; Chebyshev's bound is what holds universally.
Expecting Chebyshev's bound to be tight. It is attained by a specific three-point distribution, so it cannot be improved in general, and on normal data it is far weaker than the truth.
Deleting every point flagged by the $1.5\,\mathrm{IQR}$ rule. On skewed data the rule flags a large fraction of legitimate observations; a flag is a prompt to investigate, not a licence to discard.
Reading a percentile off a z-score without normality. $z = 2$ is the $97.7$ th percentile for a normal distribution and can be anywhere for another shape.
Using the range as the primary measure of spread. It is a function of the two least typical observations and grows with $n$ even when the distribution does not change.
Reporting a single centre for a bimodal distribution, which describes a value that may occur in neither subpopulation.
Quoting a summary without $n$ . Precision scales as $\sigma/\sqrt{n}$ , so the sample size is part of the claim.
Treating repeated measurements on the same subjects as independent. The $\sigma^{2}/n$ formula needs independence, and positive correlation makes the sample mean far noisier than it promises.

About the practice questions. They check that you can carry out this chapter's computations correctly, and each one is graded on a single answer. They are not proof exercises: working through them confirms the mechanics, not that you could prove the results yourself. For that, re-read the statements above and try to reconstruct their proofs with the page closed.

Sampling and Data Distributions →

Helpful?

Descriptive Statistics

Introduction

1.1Populations, Samples and Notation

1.2Mean

1.3Median and Mode

1.4Variance and Standard Deviation

1.5Why the Sample Variance Divides by n−1n-1n−1

1.6Percentiles, Quartiles and the IQR

1.7Standard Scores, the Empirical Rule and Chebyshev

1.8Shape and Visualization

1.9Choosing the Right Summary

1.5Why the Sample Variance Divides by $n-1$