Probability and Statistics: Chapter Review

Vocabulary and notation

Sample space $S$, outcome, event

The set of possible results, one result, and any subset $A \subseteq S$; "not", "or", "and" are $A^{c}$, $A \cup B$, $A \cap B$.

Equally likely

Every outcome of $S$ carries the same chance: an assumption about the world, not a theorem. One experiment has many sample spaces, usually one equally likely.

Mutually exclusive (disjoint)

$A \cap B = \emptyset$, so the events cannot both occur. A statement about sets; independence is one about numbers.

With and without replacement

Whether a drawn object is returned. With replacement the draws are independent; without, the second sees a reduced pool.

Resistant and sensitive

An outlier cannot drag a resistant statistic (median, IQR) but can drag a sensitive one (mean, range, standard deviation).

Right-skewed, left-skewed, bimodal

A long tail of large values, of small ones, or two separate peaks. Two peaks usually mean two mixed groups, which no single center describes.

Five-number summary and box plot

$\text{min}$, $Q_1$, median, $Q_3$, $\text{max}$, drawn as a box from $Q_1$ to $Q_3$, whiskers to the extremes inside the fences, outliers as points.

Density curve

The curve of a continuous quantity: probability is area under it, total area $1$. Its height is not a probability.

$\varphi$ and $\Phi$

$\varphi(z) = \frac{1}{\sqrt{2\pi}}e^{-z^2/2}$ is the standard normal height at $z$; $\Phi(z) = P(Z < z)$ is the area to its left, which a table records because no closed form exists.

Central Limit Theorem

A sum of many independent small effects tends to the normal shape, provided none dominates. Heavy-tailed quantities break that proviso.

Formulas and theorems

• Counting principles

  $$\begin{gathered} n_1 \times n_2 \times \cdots \times n_k \\ n^k \\ \text{total} = \text{case}_1 + \cdots + \text{case}_r \\ \text{total} = \text{wanted} + \text{unwanted} \end{gathered}$$

  Use when Decision $i$ offers $n_i$ options regardless of the earlier ones; the COUNT is fixed, not the options. $n^k$ is $k$ ordered slots with repeats. Add cases only if they overlap nowhere and miss nothing.

• Permutations

  $$\begin{gathered} P(n,k) = n(n-1)\cdots(n-k+1) \\ P(n,k) = \frac{n!}{(n-k)!} \end{gathered}$$

  Use when Integers $0 \le k \le n$, distinct objects, no repeats, order matters. Exactly $k$ factors, the last $n-k+1$. $P(n,n) = n!$, and $0! = 1$ is forced by making the closed form return it. Here $P$ COUNTS and is not a probability: it takes two integers and is at least $1$.

• Combinations

  $$\binom{n}{k} = \frac{P(n,k)}{k!} = \frac{n!}{k!\,(n-k)!}$$

  Use when Integers $0 \le k \le n$, distinct objects, no repeats, order irrelevant; the $k!$ forgets the order. $\binom{n}{k} = \binom{n}{n-k}$, and $\binom{n}{0} = \binom{n}{n} = 1$.

• Arrangements of objects not all distinct

  $$\frac{n!}{n_1!\,n_2!\,\cdots\,n_r!}$$

  Use when $r$ types with $n_1 + \cdots + n_r = n$, copies of one type identical; divide by the factorials of the repeat counts.

• Probability of an event

  $$P(A) = \frac{\lvert A \rvert}{\lvert S \rvert}$$

  Use when $S$ finite with equally likely outcomes; forces $0 \le P(A) \le 1$, $P(\emptyset) = 0$, $P(S) = 1$. The eleven sums of two dice are not, so use the $36$ ordered pairs.

• Complement and addition rules

  $$\begin{gathered} P(A^{c}) = 1 - P(A) \\ P(A \cup B) \\ = P(A) + P(B) - P(A \cap B) \end{gathered}$$

  Use when Both hold for any events. The complement needs no hypothesis; the addition rule subtracts the overlap that adding counts twice, and drops to $P(A) + P(B)$ only when $A \cap B = \emptyset$.

  e.g. King or heart: $\frac{4}{52} + \frac{13}{52} - \frac{1}{52} = \frac{4}{13}$.

• Conditional probability and the multiplication rule

  $$\begin{gathered} P(A \mid B) = \frac{P(A \cap B)}{P(B)} \\ P(A \mid B) = \frac{\lvert A \cap B \rvert}{\lvert B \rvert} \\ P(A \cap B) = P(B)\,P(A \mid B) \end{gathered}$$

  Use when $P(B) > 0$; the count form also needs equally likely outcomes. Conditioning shrinks $S$ to $B$, and $P(A \mid B)$ is not $P(B \mid A)$. The product form needs no independence.

• Independence

  $$P(A \cap B) = P(A)\,P(B)$$

  Use when The equation IS the definition, so compute it rather than sense it; equivalent to $P(A \mid B) = P(A)$ when $P(B) > 0$. Positive-probability events are never both independent and mutually exclusive.

• Mean and median as minimizers

  $$\begin{gathered} \bar{x} = \frac{1}{n}\sum x_i \\ \bar{x} \ \text{minimizes} \ \sum (x_i - c)^2 \\ \text{the median minimizes} \\ \sum \lvert x_i - c \rvert \end{gathered}$$

  Use when Any data set. The mean's minimizer is unique; for odd $n$ so is the median's, while for even $n$ every value between the two middle ones ties. One value moved by $d$ shifts the mean by $\frac{d}{n}$.

• Variance and standard deviation

  $$\begin{gathered} \sigma^2 = \frac{1}{N}\sum (x_i - \mu)^2 \\ s^2 = \frac{1}{n-1}\sum (x_i - \bar{x})^2 \end{gathered}$$

  Use when $N, \mu, \sigma$ for a whole population; $n, \bar{x}, s$ for a sample estimating a larger one, needing $n \ge 2$. Square-root either for the original units.

  e.g. $1, 3, 5, 7, 9$: $\sum (x_i - 5)^2 = 40$, so $\sigma^2 = 8$ but $s^2 = 10$.

• Quartiles, IQR, and Tukey's fences

  $$\begin{gathered} \text{IQR} = Q_3 - Q_1 \\ x < Q_1 - 1.5\,\text{IQR} \\ \text{or } \ x > Q_3 + 1.5\,\text{IQR} \end{gathered}$$

  Text description

  A box plot with its quartiles, median, and IQR marked, dashed fences 1.5 IQR beyond each end of the box, and one point past the upper fence flagged as an outlier.

  Use when Sort first; $Q_1, Q_3$ are the medians of the halves, an odd count's middle value in neither. The $1.5$ is a convention, and the IQR resists an outlier only once the count is big enough that the extremes sit outside the quartile CALCULATION: in $1, 2, 3, 4$, $Q_3$ averages the top two, so raising the $4$ drags the IQR.

• The 68-95-99.7 (empirical) rule

  $$\begin{gathered} P(\lvert X - \mu \rvert < \sigma) \approx 0.68 \\ P(\lvert X - \mu \rvert < 2\sigma) \approx 0.95 \\ P(\lvert X - \mu \rvert < 3\sigma) \approx 0.997 \end{gathered}$$

  Text description

  A normal curve above three nested bands centred on the mean, holding 68%, 95%, and 99.7% of the area out to one, two, and three standard deviations.

  Use when Requires an approximately NORMAL distribution; it says nothing about a skewed or heavy-tailed one. Rounded from $68.27\%$, $95.45\%$, $99.73\%$; the half-bands $34\%$, $13.5\%$, $2.35\%$ are its own bookkeeping.

• z-score and its inverse

  $$z = \frac{x - \mu}{\sigma}, \qquad x = \mu + z\sigma$$

  Use when $\sigma > 0$, and the divisor is $\sigma$, never $\sigma^2$. A $z$ counts standard deviations, carries no units, is not a probability, and is negative below the mean.

• Reading the standard normal table

  $$\begin{gathered} P(Z < z) = \Phi(z) \\ P(Z > z) = 1 - \Phi(z) \\ P(a < Z < b) = \Phi(b) - \Phi(a) \\ \Phi(-z) = 1 - \Phi(z) \end{gathered}$$

  Use when Continuous, standardized first; $\Phi$ is the area to the LEFT. An endpoint has no area, so $P(X = c) = 0$ and $P(Z < z) = P(Z \le z)$, unlike a discrete distribution.

Problem types, step by step

Exam traps

• Trap Never asking whether order matters: ten people shaking hands counted as $P(10,2) = 90$.

  Fix A handshake is a set: $\binom{10}{2} = 45$. The $90$ answers a different question, electing a president and vice-president.

• Trap Adding for "or" without checking the overlap: king or heart as $\frac{4}{52} + \frac{13}{52} = \frac{17}{52}$.

  Fix Subtract the intersection: $\frac{16}{52}$. Adding alone is legal only for disjoint events, and can otherwise exceed $1$.

• Trap Multiplying for "and" without checking independence: three hearts drawn without replacement as $\left(\frac{1}{4}\right)^3 = \frac{1}{64}$.

  Fix The pool shrinks each draw: $\frac{13}{52} \times \frac{12}{51} \times \frac{11}{50} = \frac{11}{850}$. Only $P(B)P(A \mid B)$ needs no independence.

• Trap Calling mutually exclusive events independent because neither seems to say anything about the other.

  Fix Disjoint events of positive probability are maximally dependent: $P(A \mid B) = 0$ while $P(A) > 0$. Independent ones of positive probability must overlap.

• Trap Answering $P(B \mid A)$ for $P(A \mid B)$: a $99\%$ detection rate read as a $99\%$ chance of illness after a positive test.

  Fix They divide by different denominators: $1$ case in $1000$ with a $5\%$ false-positive rate leaves a positive patient ill under $2\%$ of the time.

• Trap Reserving one in advance: at least one woman on a $3$-person committee from $4$ men and $3$ women as $3 \times \binom{6}{2} = 45$.

  Fix That builds a two-woman committee twice and a three-woman one three times, an uneven overcount no division repairs. Total minus none: $\binom{7}{3} - \binom{4}{3} = 31$.

• Trap Dividing by the repeat counts rather than their factorials: BANANA as $\frac{6!}{3 \times 2} = 120$.

  Fix Divide by the factorials: $\frac{6!}{3!\,2!} = 60$.

• Trap Reporting the mean as the typical value of skewed data.

  Fix The mean follows the tail, a reliable rule of thumb rather than a theorem: sales of $220, 245, 260, 285, 990$ average $400$, above four of five. Report the median, $260$, with the IQR.

• Trap Dividing by $n$ when the numbers are a sample.

  Fix A sample takes its deviations from the mean it fitted to itself, so they are never too large. Divide by $n-1$; reserve $N$ for a population.

• Trap Applying the 68-95-99.7 rule to right-skewed incomes.

  Fix The rule describes approximately normal data only. For skewed data report the median and IQR, and use a fence, not a band.

• Trap Reading the table entry as the tail wanted: $P(Z > 1.4)$ answered $0.9192$, or $0.4192$ by subtracting from $0.5$.

  Fix $\Phi$ is the area to the LEFT and the total is $1$: $P(Z > 1.4) = 1 - 0.9192 = 0.0808$. A negative $z$ needs $\Phi(-z) = 1 - \Phi(z)$.