The whole Probability and Statistics formula sheet, to print

Selecting and arranging r of n distinct objects. P(n,n) = n! is the number of orderings of all n, and P(n,0) = 1 because 0! = 1.

Selecting r of n distinct objects with no arrangement. Related to permutations by P(n,r) = r! times the binomial coefficient. The binomial coefficient is taken as 0 when r > n.

The sum counts all subsets of an n-element set, so there are 2^n - 1 non-empty subsets. Useful for 'at least one' style selections.

Putting x = y = 1 gives the sum 2^n. Putting x = -1, y = 1 shows that the alternating sum of the binomial coefficients is 0 for n >= 1.

For finite sets. The count of objects with NONE of the properties is the total minus the union. The same pattern for probabilities is in section 4.

The number of ways to split n distinct objects into LABELLED groups of sizes n_1, ..., n_k, and equally the number of arrangements of n objects with repeated types. If the groups are unlabelled, divide by m! for each set of m groups of the same size. For example, 6 distinct people split into three unlabelled pairs: 6!/(2! 2! 2! 3!) = 15.

Distributing n identical objects into k distinct boxes with no limit per box. For a lower bound x_i >= a, substitute y_i = x_i - a and reduce n by a. For an upper bound x_i <= b, count all solutions and subtract those with x_i >= b + 1, using inclusion-exclusion when several variables are bounded.

Also the number of ways to place m distinct objects into n distinct boxes with every box non-empty. It comes from inclusion-exclusion over the set of boxes left empty, and is 0 when m < n. Check: m = 4, n = 2 gives 16 - 2 = 14; m = 4, n = 3 gives 81 - 48 + 3 = 36.

Valid only when the sample space is finite and every outcome is equally likely. State this assumption before using it; the wording of the question (a fair die, chosen at random, uniformly) is what justifies it.

Every other rule is derived from these three. Axiom 3 applies to any finite or countably infinite sequence of events, but only if they are pairwise mutually exclusive.

Use it for 'at least one' events: compute the probability that none occurs and subtract from 1.

Holds for any two events. The subtraction removes the overlap, which was counted twice.

Holds for any three events.

Holds for any events. The signs alternate: singles plus, pairs minus, triples plus, and so on. S_k has C(n,k) terms.

The first formula holds for any A and B; the simpler form P(B) - P(A) needs A to be a subset of B. The union bound holds with no condition.

The lower bound follows from P(A union B) <= 1; the upper bound from monotonicity. It gives the possible values of the intersection when only the two marginal probabilities are known.

With the conditional probability P(A | B) = P(A and B)/P(B), defined for P(B) > 0, independence is equivalent to P(A | B) = P(A). If A and B are independent, so are A and B^c, A^c and B, and A^c and B^c.

All four equations are required; pairwise independence is only the first three. For n events, the product rule must hold for every sub-collection of two or more events, which is 2^n - n - 1 equations.

Valid when A_1, ..., A_n are mutually independent: their complements are then mutually independent too, and the probability that none occurs is the product.

An urn has R red and B blue balls, and n <= R + B balls are drawn at random without replacement. Order is ignored in both the numerator and the denominator.

Assumes N equally likely days and independent birthdays (N = 365, ignoring 29 February). For k > N the probability is 0. The probability that at least two people share a birthday is 1 minus this.

n letters are placed at random in n addressed envelopes, one per envelope. D_n counts the permutations with no fixed point: D_1 = 0, D_2 = 1, D_3 = 2, D_4 = 9, D_5 = 44. Recurrence: D_n = (n-1)(D_{n-1} + D_{n-2}) for n >= 3. As n grows, P(no match) tends to 1/e, about 0.3679.

Defined when $P(B)>0$. For fixed $B$, $P(\cdot \mid B)$ is itself a probability measure, so $P(A^c\mid B)=1-P(A\mid B)$.

The first form needs $P(B)>0$, the second needs $P(A)>0$. No independence is needed.

Used for sequential experiments such as draws without replacement. Needs $P(A_1\cap \cdots \cap A_{n-1})>0$, which makes every conditioning event positive.

Always valid. The second identity splits $A$ by whether $B$ occurs; it is the two-part case of the law of total probability below.

Always valid. Given any three of $P(A)$, $P(B)$, $P(A\cup B)$ and $P(A\cap B)$, it gives the fourth.

Always valid. Add the singles, subtract the pairs, add back the triple.

Sum over all values of the other variable. All cells of the table sum to 1.

Defined when $P(Y=y)>0$. Fix the column of the conditioning value, then divide each cell in it by that column's total. For each fixed $y$ these conditionals sum to 1 over $x$.

Each cell must equal the product of its row total and column total. One cell that fails is enough to show $X$ and $Y$ are dependent.

For jointly continuous $X,Y$ with joint density $f_{X,Y}$. Sums become integrals. The conditional density is defined where $f_Y(y)>0$. Densities are not probabilities: probabilities come from integrating them, and $P(X=x)=0$ for a continuous $X$.

The $B_i$ must be mutually exclusive and exhaustive with $P(B_i)>0$. The simplest partition is $\{B,B^c\}$: $P(A)=P(A\mid B)P(B)+P(A\mid B^c)P(B^c)$.

Follows from the multiplication rule written both ways. Needs $P(A)>0$ and $P(B)>0$.

$B_1,\dots ,B_n$ is a partition with positive probabilities, and $P(A)>0$. The denominator is the law of total probability, so the posteriors sum to 1 over $j$.

Posterior odds = likelihood ratio × prior odds. It needs $P(A\mid B^c)>0$, $0<P(B)<1$ and $P(A)>0$. Convert back with probability = odds / (1 + odds).

Valid when $E_1$ and $E_2$ are conditionally independent given $D$ and also given $D^c$. In the example above, two positive tests give $\frac{0.81\times 0.02}{0.81\times 0.02+0.0025\times 0.98}=\frac{0.0162}{0.01865}=\frac{324}{373}\approx 0.869$. Equivalently, apply Bayes once and use the first posterior as the prior for the second test.

The product form is the definition and needs no positivity condition. If $A$ and $B$ are independent, so are $A$ and $B^c$, $A^c$ and $B$, and $A^c$ and $B^c$.

$A$, $B$, $C$ are mutually independent when this holds AND every pair is independent: $P(A\cap B)=P(A)P(B)$, $P(A\cap C)=P(A)P(C)$, $P(B\cap C)=P(B)P(C)$. For $n$ events, the product rule must hold for every subcollection of two or more.

Events $A$ and $B$ are conditionally independent given $C$ when this holds, with $P(C)>0$. In general it neither implies nor is implied by ordinary independence.

Raw data. For grouped data with frequencies $f_i$ and class midpoints $x_i$, use $\bar{x}=\sum f_i{x}_i/\sum f_i$. This treats every value as sitting at its class midpoint, so it is an approximation unless the values really are the midpoints.

Here $x_{(k)}$ is the $k$-th smallest value; sort first.

The median class is the first class whose cumulative frequency is at least $N/2$. $L$ = its lower class boundary, $N=\sum f_i$, $F$ = cumulative frequency of the classes before it, $f$ = its frequency, $h$ = its width. Classes must be continuous: if the class limits leave gaps (10-19, 20-29), use the boundaries (9.5-19.5, 19.5-29.5).

$L$ = lower boundary of the modal class (highest frequency $f_1$), $f_0$ and $f_2$ = frequencies of the classes just before and after it, $h$ = class width. The modal class and its two neighbours must have the same width. If the modal class is the first or last class, take the missing neighbour's frequency as $0$.

$p$ is the pmf, $f$ the pdf and $F$ the CDF. The median condition $F(m)=\frac{1}{2}$ is for a continuous $X$ (in general a median satisfies $P(X\le m)\ge \frac{1}{2}$ and $P(X\ge m)\ge \frac{1}{2}$). The mean exists only when the sum or integral of $|x|$ converges. Example: for the exponential distribution with rate $\lambda$, $1-e^{-\lambda m}=\frac{1}{2}$ gives median $=\ln 2/\lambda$, while the mean is $1/\lambda$ and the mode is $0$.

For any constant $c$, the sum of squared deviations is therefore smallest when $c=\bar{x}$. The sum of absolute deviations $\sum |x_i-c|$ is smallest when $c$ is the median (for even $n$, any $c$ between the two middle values).

$\mu$ is the population mean. The second form is the shortcut $E[X^2]-(E[X]{)}^2$.

$s^2$ is an unbiased estimator of the population variance. For the same data it relates to the divide-by-$n$ value by $s^2=\frac{n}{n-1}{\sigma }^2$.

$\mathrm{SD}=\sqrt{\mathrm{Var}(X)}$. If $X$ picks one of the $n$ data values uniformly at random, this equals the population (divide-by-$n$) variance of the data.

Defined for data with a positive mean. It has no units, so it compares the spread of data sets measured on different scales: the one with the larger CV is relatively more variable.

This holds for the population and the sample versions alike. The median and the mode also transform as $a\cdot (\text{value})+b$.

Weighted by group size. It equals the plain average $(m_1+m_2)/2$ only when $n_1=n_2$.

Both group variances must be the divide-by-$n$ kind. If sample variances are given, first convert them with ${\sigma }_i^2=\frac{n_i-1}{n_i}{s}_i^2$.

$\mathrm{Cov}(X,X)=\mathrm{Var}(X)$, and $\mathrm{Cov}(X,c)=0$ for a constant $c$.

For the population version, divide by $n$ instead. State the convention you use.

Both variances must be positive and finite. The $n$ or $n-1$ factors cancel in $r$, so the sample value does not depend on the convention.

For $a\ne 0$ and $c\ne 0$. A positive-slope linear change of either variable leaves the correlation unchanged; a negative multiplier on exactly one variable flips its sign.

Covariance is linear in each argument separately. Note the $+\mathrm{Var}(Y)$ in $\mathrm{Var}(X-Y)$: variances never subtract.

The cross term vanishes when the covariance is zero (for example, when $X$ and $Y$ are independent).

If the $X_i$ are pairwise uncorrelated, the variance of the sum is the sum of the variances.

For $X_1,\dots ,X_n$ independent (pairwise uncorrelated is enough) with common variance ${\sigma }^2$; here $\bar{X}=\frac{1}{n}\sum X_i$.

$b_{yx}$ is the least-squares slope of the line of $y$ on $x$, and $b_{xy}$ that of $x$ on $y$. Both lines pass through $(\bar{x},\bar{y})$. The two slopes and $r$ share one sign, so $r$ is the square root of $b_{yx}{b}_{xy}$ taken with that sign. Use the same divisor in the covariance and the variances.

$d_i$ is the difference between the ranks of $x_i$ and $y_i$. This shortcut is exact only when there are no tied ranks. With ties, compute Pearson's $r$ on the ranks instead.

Use the second condition to find an unknown constant in a given PMF.

A density can exceed 1 at a point; only its integral is a probability.

The CDF is non-decreasing, right-continuous, with limits 0 at $-\infty$ and 1 at $+\infty$. For any random variable, $P(a<X\le b)=F_X(b)-F_X(a)$.

For a continuous variable differentiate the CDF where it is differentiable; for a discrete variable the PMF is the size of the jump at $x$.

Defined when the sum or integral converges absolutely.

No need to find the distribution of $g(X)$. In general $E[g(X)]\ne g(E[X])$; they are equal when $g$ is linear, $g(x)=ax+b$.

True whenever the expectations exist, with or without independence.

Standard deviation $\text{SD}=\sqrt{\mathrm{Var}(X)}$. Since $\mathrm{Var}(X)\ge 0$, $E[X^2]\ge (E[X]{)}^2$, and $\mathrm{Var}(X)=0$ only when $X$ is constant with probability 1. Convention: $\mathrm{Var}(X)$ here is the population (distribution) variance. For a list of $n$ data values, the population variance divides by $n$ and the sample variance divides by $n-1$.

A shift $b$ does not change the variance; a scale $a$ multiplies it by $a^2$ (the sign of $a$ is lost).

To find the expected number of events $A_1,\dots ,A_n$ that occur, write the count as $\sum I_{A_i}$; by linearity its mean is $\sum P(A_i)$, with no independence needed.

Sum (or integrate) out the other variable over its whole range. Mind the limits when the support is not a rectangle.

For a PMF the product rule must hold at every pair $(x,y)$, so one failing pair shows dependence. For a PDF it must hold everywhere apart from sets of zero probability. If the set where the joint PMF or PDF is positive is not a product set $A\times B$ (for example the triangle $0<x<y<1$), then $X$ and $Y$ are dependent.

$\mathrm{Cov}(X,X)=\mathrm{Var}(X)$; $\mathrm{Cov}(aX+b,cY+d)=ac\mathrm{Cov}(X,Y)$.

Defined when both variances are positive. $|\rho |=1$ exactly when $Y=aX+b$ with probability 1 for some constants $a\ne 0$ and $b$; then $\rho =1$ if $a>0$ and $\rho =-1$ if $a<0$.

For uncorrelated (in particular independent) variables the covariance term vanishes, so $\mathrm{Var}(X\pm Y)=\mathrm{Var}(X)+\mathrm{Var}(Y)$. Note the plus sign, even for the difference.

For independent identically distributed $X_i$ with variance ${\sigma }^2$, the sample mean has $\mathrm{Var}(\bar{X})={\sigma }^2/n$.

The converse is false: zero covariance (uncorrelated) does not imply independence.

Requires $p_Y(y)>0$ (or $f_Y(y)>0$). If $X,Y$ are independent the conditional equals the marginal.

For each fixed $y$ this is a number; $E[X\mid Y]$ is the random variable obtained by replacing $y$ with $Y$.

Also a random variable, a function of $Y$.

Replace the sum by an integral against $f_Y$ when $Y$ is continuous. For events $A_1,A_2,\dots$ that partition the sample space, $E[X]={\sum }_{i}E[X\mid A_i]P(A_i)$.

Average within-group variance plus variance of the group means.

Once $Y$ is given, any function of $Y$ behaves like a constant. The last property does not work in reverse: $E[X\mid Y]=E[X]$ does not imply independence.

Here $X_1,X_2,\dots$ are independent and identically distributed with mean $\mu$ and variance ${\sigma }^2$, and $N$ is a non-negative integer random variable independent of the $X_i$. It follows from the two total laws with $E[S\mid N]=N\mu$ and $\mathrm{Var}(S\mid N)=N{\sigma }^2$.

For $X\ge 0$ and $a>0$. Needs only the mean.

Holds for any distribution with mean $\mu$ and finite variance ${\sigma }^2$; $k,t>0$. Complementary form: $P(|X-\mu |<k\sigma )\ge 1-\frac{1}{k^2}$.

Applies when each of the n consecutive integers is equally likely; zero elsewhere.

Here n = b - a + 1 and the values are consecutive integers. The variance depends only on n, not on where the block starts.

Special case a = 1, b = n. For a fair die (n = 6): mean 3.5, variance 35/12.

One trial, success probability p, 0 <= p <= 1.

Because X^2 = X for a 0/1 variable, E[X^2] = p, so Var(X) = p - p^2. The variance is largest at p = 1/2, where it equals 1/4.

Requires fixed n, independent trials and the same p on every trial. The probabilities sum to 1 by the binomial theorem.

Follows by adding n independent Bernoulli(p) variables: means add, and variances add because of independence. When 0 < p < 1 the variance is always less than the mean.

E[X^2] = Var(X) + (E[X])^2. Useful when a question asks for E[X^2] or the expectation of a quadratic in X.

Use it to compute successive probabilities quickly and to locate the mode.

Valid for 0 < p < 1. At the end points the distribution is degenerate: the mode is 0 when p = 0 and n when p = 1.

Both independence and the common p are essential. With different success probabilities the sum is in general not binomial, although means still add, and variances still add because of independence.

Independent trials, constant p with 0 < p <= 1. X counts the trial on which the first success occurs, so X >= 1. Mean 1/p and variance (1-p)/p^2. If instead the count is the number of failures before the first success, the PMF is (1-p)^k p for k = 0, 1, 2, ..., the mean is (1-p)/p and the variance is still (1-p)/p^2.

Use the total area equal to 1 to find an unknown constant in a given density.

F is non-decreasing and, for a continuous X, continuous. Its limits are 0 as x goes to -infinity and 1 as x goes to +infinity. The PDF is the derivative of the CDF wherever the derivative exists.

Valid for any continuous X. Also P(X > a) = 1 - F(a).

Solve F(x) = p for x. The p-th percentile (p as a fraction) is x_p; the median is the 50th percentile.

E[X] exists only when the integral of |x| f(x) converges. Some distributions have no mean, for example t with 1 degree of freedom. Convention used throughout: E[X^2] = integral of x^2 f(x) dx, and SD = sqrt(Var(X)).

Holds for any constants a and b. Adding a constant shifts the mean and leaves the variance unchanged. SD(aX + b) = |a| SD(X).

Requires P(X > a) > 0, and the conditional PDF is zero for x <= a. In general, conditioning on an event A with P(X in A) > 0 gives f(x)/P(X in A) for x in A. The result integrates to 1 over the conditioning region.

Defined only where f_Y(y) > 0. For a fixed y it is a density in x and integrates to 1. If X and Y are independent, f_{X|Y}(x | y) = f_X(x).

Zero density outside [a, b]; F is 0 below a and 1 above b. For a sub-interval inside [a, b], probability = length / (b - a).

The median equals the mean for the uniform distribution.

Support is x >= 0; lambda is the rate. P(X > x) is the survival function.

Mean and SD are equal. The median (about 0.693/lambda) is smaller than the mean 1/lambda, and the distribution is right-skewed. ln is the natural logarithm.

Holds for all s, t >= 0. The exponential is the only continuous distribution with this property. Time already waited does not change the distribution of the remaining wait.

Requires X1 and X2 independent. Reason: P(min > x) = P(X1 > x) P(X2 > x) = e^{-(lambda1 + lambda2) x}. The same holds for more than two: the rates add.

lambda > 0 is the average number of events in the interval considered. P(X >= 1) = 1 - e^{-lambda}.

Mean equals variance. If X1 ~ Poisson(l1) and X2 ~ Poisson(l2) are independent, then X1 + X2 ~ Poisson(l1 + l2).

Events occur independently at a constant average rate. The count in a window of length t has mean lambda*t, and the gaps between successive events are exponential with mean 1/lambda.

Use when n is large and p is small, so that np is moderate. The binomial's variance np(1-p) is close to np only when p is small.

Mean = median = mode = mu; variance sigma^2; symmetric about mu.

Divide by the standard deviation sigma, not the variance. P(X > a) = 1 - Phi((a - mu)/sigma).

Phi(0) = 0.5. Use this to turn negative z into positive z when reading a table.

Applies to every normal distribution, whatever mu and sigma are.

Requires X and Y independent and each normal, with a and b not both zero. Variances add with squared coefficients, even for a difference: Var(X - Y) = sigma_X^2 + sigma_Y^2.