Probability and Distributions¶
Chapter 4: From Uncertainty to Quantified Belief¶
Why This Chapter Matters¶
Chapter 2 described data, and Chapter 3 made structure visible. Chapter 4 asks a different question: when the outcome is not yet known, how should uncertainty be represented, calculated, and updated?
Machine learning lives inside probability. A classifier score is a probability statement. A train/test split assumes future data will resemble sampled data. A loss function often comes from a probability model. A confidence interval, a p-value, a Bayesian posterior, and a calibrated risk score all depend on the same basic language: experiments, outcomes, events, random variables, distributions, and conditional belief.
The chapter begins with finite sample spaces because the simplest probability questions should feel concrete. It then moves through event rules, conditional probability, contingency tables, and Bayes' theorem. Those ideas lead naturally into probability distributions, where instead of assigning probability to one named event, we assign probability across all possible values of a random variable.
The practical distributions in this chapter are the Binomial, Poisson, and Normal distributions. They are not the only distributions in statistics, but they are the ones readers must recognize early. Binomial models repeated yes/no trials. Poisson models counts in a fixed interval. Normal models many continuous measurements and becomes the visual center of z-scores, sampling, and inference.
For technical reference, keep SciPy's stats distribution reference, Python's math.comb, NumPy's random sampling documentation, and OpenStax's open probability text at Introductory Business Statistics nearby.
Section 4.0: Setup, Imports, and Reproducibility¶
This chapter uses the same scientific Python stack as the earlier chapters, with SciPy's stats module doing most of the probability-specific work. NumPy supplies arrays and reproducible simulation. Pandas supports contingency tables. Matplotlib and Seaborn make probability mass functions, density curves, and cumulative probabilities visible.
The setup cell imports these tools and fixes plotting defaults. It also creates a NumPy random generator so examples that involve simulation can be repeated exactly. Reproducibility matters here because probability examples can otherwise appear to change every time a notebook is run, distracting the reader from the underlying rule.
Run the setup cell before executing the probability examples.
# === Chapter 4: Setup ===
# Standard imports for probability & distribution work.
import math
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import scipy
from scipy import stats
# Configure plots for clean rendering inside the notebook
plt.rcParams.update({
"figure.figsize": (8, 4), # default figure size (width, height) in inches
"axes.grid": True, # show gridlines by default
"grid.alpha": 0.3, # light gridlines (not distracting)
"axes.spines.top": False, # remove top border
"axes.spines.right": False, # remove right border
})
sns.set_theme(style="whitegrid")
# Seed for reproducibility -- a Generator instance gives the same results every run
rng = np.random.default_rng(42)
print("Chapter 4 environment ready.")
print(f"NumPy {np.__version__} | Pandas {pd.__version__} | SciPy {scipy.__version__}")
Section 4.1: Probability Foundations¶
Probability is the mathematics of uncertainty. It assigns numbers between 0 and 1 to events, where 0 means impossible and 1 means certain. The numbers are meaningful only when the experiment and the event are defined clearly.
An experiment is a repeatable process that produces an observation: a coin toss, a diagnostic test, a blood pressure measurement, a model prediction, or an inspection of a manufactured part. An outcome is one specific result of that experiment. The sample space is the set of all possible outcomes. An event is a subset of the sample space, meaning a collection of outcomes that answer a question we care about.
For three parts that can each be good or defective, the sample space has eight outcomes. The event "at least two defective parts" is not a new kind of object; it is a subset of those eight outcomes. Probability begins by counting or weighting how much of the sample space belongs to the event.
The cleanest introductory model assumes equally likely outcomes. Later, real data often breaks that assumption, but the vocabulary remains the same.
Section 4.2: The Division Rule of Probability¶
When every outcome in a finite sample space is equally likely, the probability of an event is the number of favorable outcomes divided by the number of possible outcomes.
$$P(A) = \frac{\text{number of outcomes in }A}{\text{number of outcomes in }S}$$
This rule is simple, but it is foundational. It teaches the reader that probability is not a mood or a guess. It is a ratio attached to a clearly defined event and a clearly defined universe of possible outcomes.
In machine learning, this frequency intuition appears constantly. A class proportion in a training set is an empirical probability. A validation error rate is a fraction of held-out cases. A confusion-matrix cell is a count inside a larger count. The division rule is the first bridge from raw counts to quantified uncertainty.
# === Section 4.2: Division Rule — Enumerated Example ===
# Enumerate the sample space for three machine parts (G/D)
from itertools import product
S = list(product("GD", repeat=3)) # all 2^3 outcomes
print(f"Sample space S = {[''.join(o) for o in S]}")
print(f"|S| = {len(S)}")
# Event A: at least 2 defective
A = [o for o in S if o.count('D') >= 2]
print(f"\nEvent A (≥2 defective) = {[''.join(o) for o in A]}")
print(f"|A| = {len(A)}")
P_A = len(A) / len(S)
print(f"\nP(A) = {len(A)}/{len(S)} = {P_A:.4f}")
Section 4.3: Combining Events: Independent and Dependent¶
Real questions rarely involve only one event. We often need the probability that two events occur together, or the probability that one event changes the likelihood of another.
Two events are independent when knowing that one occurred does not change the probability of the other. If A and B are independent, the probability that both occur is the product of their probabilities.
$$P(A \cap B) = P(A)P(B)$$
Dependence is different. If one event changes the probability of another, the joint probability must use conditional probability.
$$P(A \cap B) = P(A)P(B \mid A)$$
This distinction matters in data science. Features are often correlated, clinical risk factors are not independent, and repeated measurements on the same patient cannot be treated as if they came from unrelated people. Independence is powerful when it is justified and dangerous when it is assumed out of convenience.
# === Section 4.3: Multiplication Rule for Independent Events ===
P_A = 0.5
P_B = 0.7
# Independent: P(A ∩ B) = P(A) * P(B)
P_AB = P_A * P_B
print(f"P(A) = {P_A}")
print(f"P(B) = {P_B}")
print(f"P(A ∩ B) = {P_A} × {P_B} = {P_AB:.4f}")
Section 4.4: Mutually Exclusive and Non-Exclusive Events¶
Two events are mutually exclusive when they cannot happen at the same time. A single card cannot be both a king and a queen. A single binary label cannot be both positive and negative. When events are mutually exclusive, the probability of A or B is the sum of the two probabilities.
$$P(A \cup B) = P(A) + P(B)$$
Most real analytic events are not mutually exclusive. A patient can be older than 65 and hypertensive. A message can contain the word "free" and come from an unknown sender. If events can overlap, adding the two probabilities double-counts the overlap. The correction is inclusion-exclusion.
$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$
The key habit is to ask whether the overlap is possible before choosing the formula. Probability errors often come from applying a clean rule to a situation where its assumptions do not hold.
# === Section 4.4: Addition Rules — Card Deck Examples ===
# --- Example 1: Mutually exclusive (King OR Queen) ---
P_king = 4 / 52
P_queen = 4 / 52
P_king_or_queen = P_king + P_queen # no overlap
print("--- Mutually Exclusive ---")
print(f"P(King) = {P_king:.4f}")
print(f"P(Queen) = {P_queen:.4f}")
print(f"P(King ∪ Queen) = {P_king_or_queen:.4f} ({P_king_or_queen} = 8/52 = 2/13)")
# --- Example 2: Non-mutually exclusive (King OR Diamond) ---
P_diamond = 13 / 52
P_king_diamond = 1 / 52 # King of Diamonds
P_king_or_diamond = P_king + P_diamond - P_king_diamond
print("\n--- Non-Mutually Exclusive (Inclusion–Exclusion) ---")
print(f"P(King) = {P_king:.4f}")
print(f"P(Diamond) = {P_diamond:.4f}")
print(f"P(King ∩ Diamond) = {P_king_diamond:.4f}")
print(f"P(King ∪ Diamond) = {P_king_or_diamond:.4f} (= 16/52 = 4/13)")
Section 4.5: Conditional Probability¶
Conditional probability answers a restricted-universe question: if we know that B has occurred, what is the probability of A within that smaller universe?
$$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$$
The denominator matters. We are no longer asking about the whole sample space. We are asking only among cases where B is true. This is the logic behind many clinical and machine-learning questions: probability of disease given a positive test, probability of readmission given a prior admission, probability of spam given a word, probability of model failure given a subgroup.
Conditional probability also explains why direction matters. P(A | B) is usually not the same as P(B | A). Confusing the two is one of the most common errors in statistical reasoning.
# === Section 4.5: Conditional Probability — Cards Without Replacement ===
# Two spades in a row (without replacement)
P_first_spade = 13 / 52
P_second_given = 12 / 51 # one spade already removed
P_both_spades = P_first_spade * P_second_given
print("Two cards without replacement — both spades:")
print(f"P(1st spade) = 13/52 = {P_first_spade:.4f}")
print(f"P(2nd spade | 1st) = 12/51 = {P_second_given:.4f}")
print(f"P(both spades) = {P_first_spade:.4f} × {P_second_given:.4f} = {P_both_spades:.4f}")
Section 4.6: Contingency Tables and Marginal Probability¶
A contingency table turns counts for two categorical variables into a visible probability structure. The inner cells are joint counts. Row totals and column totals are marginal counts. Dividing by the grand total turns counts into probabilities.
This is where probability becomes concrete for tabular data. If a table crosses disease status with test result, then the positive-test column, the disease row, and the diseased-and-positive cell each answer a different question. Marginal probability looks at one variable without conditioning on the other. Joint probability looks at both together. Conditional probability restricts the table to a row or column before dividing.
Contingency tables are also the foundation for confusion matrices, sensitivity, specificity, precision, recall, and many fairness diagnostics. A model report is often a probability table wearing different names.
# === Section 4.6: Contingency Table — Car Purchase Example ===
# Build the table as a DataFrame
data = {
"Income < 20K": [38, 82],
"Income >= 20K": [42, 38],
}
ct = pd.DataFrame(data, index=["Bought car", "Did not buy"])
ct["Total"] = ct.sum(axis=1)
ct.loc["Total"] = ct.sum(axis=0)
print("Contingency Table of Counts")
print(ct)
N = 200 # grand total
print("\n--- Marginal Probabilities ---")
print(f"P(Bought car) = {80/N:.2f}")
print(f"P(Did not buy) = {120/N:.2f}")
print(f"P(Income < 20K) = {120/N:.2f}")
print(f"P(Income >= 20K) = {80/N:.2f}")
print("\n--- Joint Probabilities ---")
print(f"P(Buy ∩ ≥20K) = {42/N:.2f}")
print(f"P(Buy ∩ <20K) = {38/N:.2f}")
print(f"P(NotBuy ∩ ≥20K) = {38/N:.2f}")
print(f"P(NotBuy ∩ <20K) = {82/N:.2f}")
print("\n--- Conditional Probability ---")
print(f"P(Buy | Income ≥ 20K) = 42/80 = {42/80:.4f}")
Section 4.7: Bayes' Theorem¶
Bayes' theorem is the rule for updating belief when evidence arrives. It connects a prior probability, a likelihood, and the total probability of the evidence to produce a posterior probability.
$$P(H \mid E) = \frac{P(E \mid H)P(H)}{P(E)}$$
The prior P(H) is what we believed about the hypothesis before seeing the evidence. The likelihood P(E | H) is how expected the evidence would be if the hypothesis were true. The denominator P(E) normalizes across all ways the evidence could have appeared. The posterior P(H | E) is the updated probability after evidence.
Bayes' theorem is important because it reverses the direction of conditioning correctly. Diagnostic tests, spam filters, risk scores, and clinical reasoning all ask questions of the form: given what I observed, how should I revise the probability of an underlying state?
The example below uses spam filtering, but the same structure applies to disease testing and many model-updating problems.
# === Section 4.7: Bayes' Theorem — Spam Filter Illustration ===
# A simple Bayesian spam detector
P_spam = 0.40 # prior: 40 % of email is spam
P_ham = 0.60 # 60 % is legitimate
P_word_spam = 0.30 # P(word | spam) — the word appears in 30 % of spam
P_word_ham = 0.02 # P(word | ham) — the word appears in 2 % of ham
# Total probability of the word (law of total probability)
P_word = P_word_spam * P_spam + P_word_ham * P_ham
# Posterior: P(spam | word)
P_spam_given_word = (P_word_spam * P_spam) / P_word
print("Bayes' Theorem — Spam Filtering")
print(f"P(spam) = {P_spam}")
print(f"P(word | spam) = {P_word_spam}")
print(f"P(word | ham) = {P_word_ham}")
print(f"P(word) = {P_word:.4f}")
print(f"P(spam | word) = {P_spam_given_word:.4f}")
print(f"\nSeeing this word raises spam probability from {P_spam:.0%} to {P_spam_given_word:.1%}.")
Section 4.8: Probability Distributions Overview¶
A probability distribution assigns probability across the possible values of a random variable. A random variable is a numerical summary of an uncertain process: number of heads, number of arrivals, systolic blood pressure, tumor size, model score, or time until an event.
Discrete random variables take countable values. Their probabilities are described by a probability mass function, or PMF. Continuous random variables take values along an interval. Their density is described by a probability density function, or PDF, and probabilities are areas under the curve rather than heights at one point.
The cumulative distribution function, or CDF, works for both discrete and continuous variables. It answers P(X <= x). The inverse CDF, often called the percent point function or PPF in SciPy, answers the reverse question: what value cuts off a chosen cumulative probability?
The rest of the chapter focuses on Binomial, Poisson, and Normal distributions because they are the first durable patterns readers need.
Section 4.9: The Binomial Distribution¶
The Binomial distribution models the number of successes in a fixed number of independent yes/no trials. It has two parameters: n, the number of trials, and p, the probability of success on each trial.
The assumptions are strict. The number of trials is fixed. Each trial has only two outcomes. The success probability is constant. The trials are independent. When those assumptions hold, the probability of exactly k successes is:
$$P(X = k) = \binom{n}{k}p^k(1-p)^{n-k}$$
The mean is np, and the variance is np(1-p). In practice, SciPy's stats.binom handles the arithmetic and avoids hand-computing combinations. The examples below use quality control, traffic lights, and order accuracy to show how the same distribution pattern appears in different domains.
# === Section 4.9: Binomial Distribution — Coin Toss (from AIML_Book Ex. 4) ===
# 10 fair coin tosses: what is the probability of 8 or more heads?
n, p = 10, 0.5
k = np.arange(0, n + 1)
pmf = stats.binom.pmf(k, n, p)
print(f"Binomial PMF for n={n}, p={p}:")
for xi, pi in zip(k, pmf):
print(f" P(X={xi:2d}) = {pi:.6f}")
# P(X >= 8)
P_ge_8 = pmf[8:].sum()
print(f"\nP(X ≥ 8) = {P_ge_8:.6f}")
print(f"Mean μ = n·p = {n*p:.1f}")
print(f"Std σ = √(n·p·q) = {np.sqrt(n*p*(1-p)):.4f}")
# === Binomial Distribution — Bank Credit Card (from AIML_Book Ex. 3) ===
# A bank issues credit cards; 60 % of customers pay on time.
# 7 accounts are sampled. What is the chance that exactly 2 pay on time?
n, p = 7, 0.6
k = np.arange(0, n + 1)
pmf = stats.binom.pmf(k, n, p)
print(f"P(X = 2) = {pmf[2]:.6f} (2 out of 7 pay on time)")
print(f"Mean μ = {n * p:.1f}")
print(f"Std σ = {np.sqrt(n * p * (1 - p)):.4f}")
# Plot the PMF
fig, ax = plt.subplots(figsize=(8, 4))
ax.bar(k, pmf, color="steelblue", edgecolor="white", alpha=0.85)
ax.set_xlabel("Number of Successes (k)")
ax.set_ylabel("P(X = k)")
ax.set_title(f"Binomial Distribution: n = {n}, p = {p}")
ax.set_xticks(k)
plt.tight_layout()
plt.show()
# === Binomial — LED Bulb Quality Check (from AIML_Book Ex. 10) ===
# Failure rate = 5 %, sample of 10 bulbs.
# a) None defective b) Exactly 1 defective
# c) ≤ 2 defective d) ≥ 3 defective
p_fail = 0.05
n = 10
k = np.arange(0, n + 1)
pmf = stats.binom.pmf(k, n, p_fail)
cdf = stats.binom.cdf(k, n, p_fail)
print(f"a) P(X = 0) = {pmf[0]:.4f} — none defective")
print(f"b) P(X = 1) = {pmf[1]:.4f} — exactly one defective")
print(f"c) P(X ≤ 2) = {cdf[2]:.4f} — two or fewer defective")
print(f"d) P(X ≥ 3) = {1 - cdf[2]:.4f} — three or more defective")
# Plot
fig, ax = plt.subplots(figsize=(8, 4))
colors = ["#e74c3c" if xi <= 2 else "#95a5a6" for xi in k]
ax.bar(k, pmf, color=colors, edgecolor="white", alpha=0.85)
ax.set_xlabel("Number of Defective LED Bulbs")
ax.set_ylabel("Probability")
ax.set_title(f"Binomial: n = {n}, p(defect) = {p_fail}")
ax.set_xticks(k)
plt.tight_layout()
plt.show()
# === Binomial — Traffic Lights (from AIML_Book Ex. 8) ===
# 15 traffic lights, P(stop at any one) = 0.3
# a) P(stop at exactly 2) b) P(stop at ≥ 1)
n, p = 15, 0.3
k = np.arange(0, n + 1)
pmf = stats.binom.pmf(k, n, p)
print(f"a) P(X = 2) = {pmf[2]:.6f}")
print(f"b) P(X ≥ 1) = {pmf[1:].sum():.6f}")
print(f" (alt: 1 - P(X=0) = {1 - pmf[0]:.6f})")
# === Binomial — Wendy's Order Accuracy (from AIML_Book Ex. 3 alt) ===
# 86.8 % orders filled correctly. Three independent orders.
# a) All 3 correct b) None correct c) ≥ 2 correct
# d) Mean and Std
n, p = 3, 0.868
k = np.arange(0, n + 1)
pmf = stats.binom.pmf(k, n, p)
mean_x = n * p
std_x = np.sqrt(n * p * (1 - p))
print(f"PMF: {pmf}")
print(f"a) P(all 3 correct) = {pmf[3]:.4f}")
print(f"b) P(none correct) = {pmf[0]:.4f}")
print(f"c) P(≥ 2 correct) = {pmf[2:].sum():.4f}")
print(f"d) Mean = {mean_x:.4f}, Std = {std_x:.4f}")
# Plot
fig, ax = plt.subplots(figsize=(7, 4))
ax.bar(k, pmf, color="steelblue", edgecolor="white", alpha=0.85)
ax.set_xlabel("Number of Orders Filled Correctly")
ax.set_ylabel("Probability")
ax.set_title(f"Binomial: n = {n}, p = {p}")
ax.set_xticks(k)
plt.tight_layout()
plt.show()
Section 4.10: The Poisson Distribution¶
The Poisson distribution models the number of events occurring in a fixed interval of time, space, or exposure when events occur independently at a known average rate. Its parameter is lambda, the expected number of events per interval.
$$P(X = x) = \frac{\lambda^x e^{-\lambda}}{x!}$$
The Poisson distribution is useful when there is no natural fixed number of trials. We may know the average number of customers per minute, injuries per month, calls per hour, mutations per genomic region, or events per patient-year, without observing a fixed set of yes/no trials.
For a Poisson random variable, the mean and variance are both lambda, and the standard deviation is the square root of lambda. Use the Binomial when the number of trials and success probability are known. Use the Poisson when the average count per interval is the natural information.
# === Section 4.10: Poisson — Bank Footfall (from AIML_Book Ex. 1) ===
# Average footfall: λ = 6 customers per minute.
# Compute the PMF for 0–19 and P(X ≥ 4).
rate = 6
x = np.arange(0, 20)
pmf = stats.poisson.pmf(x, rate)
print(f"Poisson PMF (λ = {rate}):")
for xi, pi in zip(x[:10], pmf[:10]):
print(f" P(X={xi:2d}) = {pi:.6f}")
print(" ...")
P_ge_4 = 1 - pmf[:4].sum() # 1 - P(X ≤ 3)
print(f"\nP(X ≥ 4) = {P_ge_4:.6f}")
# Plot
fig, ax = plt.subplots(figsize=(8, 4))
ax.bar(x, pmf, color="darkorange", edgecolor="white", alpha=0.85)
ax.set_xlabel("Number of Customers per Minute")
ax.set_ylabel("Probability")
ax.set_title(f"Poisson Distribution: $\\lambda$ = {rate}")
ax.set_xticks(x)
plt.tight_layout()
plt.show()
# === Poisson — Work Injuries (from AIML_Book Ex. 4) ===
# Manufacturing plant: λ = 2.5 injuries/month.
# a) P(no injuries) b) P(≥ 1 injury)
rate = 2.5
P_zero = stats.poisson.pmf(0, rate)
P_ge_1 = 1 - P_zero
print(f"λ = {rate} injuries/month")
print(f"a) P(X = 0) = {P_zero:.6f} — no injuries")
print(f"b) P(X ≥ 1) = {P_ge_1:.6f} — at least one injury")
# === Poisson — Mosquito Larvae (from AIML_Book Ex. 5) ===
# 5-litre bucket with 75 larvae. A 200 mL flask is drawn.
# Expected larvae? P(≥ 1 larva)?
expected_larvae = 75 / 5000 * 200 # proportional scaling
print(f"Expected larvae in flask = {expected_larvae:.1f}")
P_ge_1 = 1 - stats.poisson.pmf(0, expected_larvae)
print(f"P(≥ 1 larva) = {P_ge_1:.6f}")
# === Poisson — Insurance Agent (from AIML_Book Ex. 6) ===
# Agent sells on average 3 life-insurance policies per week.
# a) P(some policies, i.e., ≥ 1) b) P(2 ≤ X < 5)
rate = 3
x = np.arange(0, 16)
cdf = stats.poisson.cdf(x, rate)
# a) P(X >= 1)
P_some = 1 - stats.poisson.pmf(0, rate)
print(f"a) P(sells some policies) = {P_some:.4f}")
# b) P(2 ≤ X < 5) = P(X ≤ 4) - P(X ≤ 1)
P_2to4 = cdf[4] - cdf[1]
print(f"b) P(2 ≤ X < 5) = {P_2to4:.4f}")
Section 4.11: The Normal Distribution¶
The Normal distribution is the familiar bell-shaped continuous distribution. It is defined by a mean mu, which sets the center, and a standard deviation sigma, which sets the spread.
Many measurements are not exactly Normal, but the Normal distribution is central because measurement error, averages, and many biological and physical quantities are often close enough for useful approximation. It also becomes fundamental in inference because sampling distributions often become approximately Normal under broad conditions.
For a continuous distribution, the PDF height is not itself a probability. Probability is area under the curve. The CDF gives area to the left of a value. The famous 68-95-99.7 rule says that about 68 percent of a Normal distribution lies within one standard deviation of the mean, about 95 percent within two, and about 99.7 percent within three.
# === Section 4.11: Normal Distribution — Visualizing the Bell Curve ===
mu, sigma = 0, 1 # standard normal
x = np.linspace(mu - 4*sigma, mu + 4*sigma, 400)
y = stats.norm.pdf(x, mu, sigma)
fig, ax = plt.subplots(figsize=(9, 4))
ax.plot(x, y, "k-", lw=2)
# Shade the 68-95-99.7 bands
for n_sigma, alpha, label in [(1, 0.35, "68.27 %"), (2, 0.20, "95.45 %"), (3, 0.10, "99.73 %")]:
lo, hi = mu - n_sigma * sigma, mu + n_sigma * sigma
mask = (x >= lo) & (x <= hi)
ax.fill_between(x[mask], y[mask], alpha=alpha, color="steelblue", label=f"μ ± {n_sigma}σ ({label})")
ax.set_title("Standard Normal Distribution — Empirical Rule (68–95–99.7)")
ax.set_xlabel("z")
ax.set_ylabel("f(z)")
ax.legend(loc="upper right", fontsize=9)
plt.tight_layout()
plt.show()
# === Verify the 68-95-99.7 rule with scipy ===
for n_sigma in [1, 2, 3]:
area = stats.norm.cdf(n_sigma) - stats.norm.cdf(-n_sigma)
print(f"P(μ - {n_sigma}σ ≤ X ≤ μ + {n_sigma}σ) = {area:.6f} ({area:.2%})")
Section 4.12: z-Scores and the Standard Normal¶
A z-score converts a raw value into the number of standard deviations it lies above or below the mean.
$$z = \frac{x - \mu}{\sigma}$$
Standardization turns any Normal distribution into the standard Normal distribution with mean 0 and standard deviation 1. Once a value is expressed as a z-score, stats.norm.cdf can compute left-tail probabilities, right-tail probabilities, central intervals, and cutoffs.
The z-score is more than a formula. It is a scale conversion that lets measurements with different units be compared on the same probability scale.
# === Section 4.12: z-Scores — Cereal Pack Weight (from AIML_Book Ex. 1) ===
# Breakfast cereal: μ = 295 g, σ = 25 g.
# a) P(weight < 280 g)
# b) P(weight > 350 g)
# c) P(260 g ≤ weight ≤ 340 g)
mu, sigma = 295, 25
# a) P(X < 280)
z_a = (280 - mu) / sigma
P_a = stats.norm.cdf(280, loc=mu, scale=sigma)
print(f"a) z = (280 - {mu}) / {sigma} = {z_a:.2f}")
print(f" P(X < 280) = {P_a:.4f}")
# b) P(X > 350)
P_b = 1 - stats.norm.cdf(350, loc=mu, scale=sigma)
print(f"\nb) P(X > 350) = {P_b:.4f}")
# c) P(260 ≤ X ≤ 340)
P_c = stats.norm.cdf(340, loc=mu, scale=sigma) - stats.norm.cdf(260, loc=mu, scale=sigma)
print(f"\nc) P(260 ≤ X ≤ 340) = {P_c:.4f}")
# === Normal — Car Speeds on Mumbai-Pune Highway (from AIML_Book Ex. 2) ===
# μ = 70 km/h, σ = 10 km/h
# a) P(speed > 100 km/h) b) P(80 ≤ speed ≤ 100)
mu, sigma = 70, 10
z_100 = (100 - mu) / sigma
z_80 = (80 - mu) / sigma
P_gt_100 = 1 - stats.norm.cdf(z_100)
P_80_to_100 = stats.norm.cdf(z_100) - stats.norm.cdf(z_80)
print(f"μ = {mu} km/h, σ = {sigma} km/h")
print(f"z(100) = {z_100:.1f}, z(80) = {z_80:.1f}")
print(f"a) P(speed > 100) = {P_gt_100:.6f}")
print(f"b) P(80 ≤ speed ≤ 100) = {P_80_to_100:.6f}")
# === Normal — Exam Scores with Distinction Cutoff (from AIML_Book Ex. 4) ===
# μ = 60, σ = 12, n = 300 students.
# a) % scoring > 80
# b) % scoring < 40
# c) Distinction mark if top 10 % are awarded distinction
mu, sigma, n_students = 60, 12, 300
P_gt_80 = 1 - stats.norm.cdf(80, loc=mu, scale=sigma)
P_lt_40 = stats.norm.cdf(40, loc=mu, scale=sigma)
# Top 10 % → find score at 90th percentile
distinction = stats.norm.ppf(0.90, loc=mu, scale=sigma)
print(f"μ = {mu}, σ = {sigma}, n = {n_students} students")
print(f"a) P(score > 80) = {P_gt_80:.4f} → ~{P_gt_80 * n_students:.0f} students")
print(f"b) P(score < 40) = {P_lt_40:.4f} → ~{P_lt_40 * n_students:.0f} students")
print(f"c) Distinction cutoff (top 10 %) ≈ {distinction:.1f} marks")
Section 4.13: Inferential Statistics as the Next Step¶
Probability and inference move in opposite directions. Probability starts with a known model or parameter and asks what data we might see. Inference starts with observed data and asks what values of an unknown parameter are plausible.
This chapter has mostly worked in the probability direction: choose n and p, then compute Binomial probabilities; choose lambda, then compute Poisson counts; choose mu and sigma, then compute Normal areas. Chapter 5 turns the arrow around. It asks what the sample says about the population.
That reversal is the foundation of estimation, confidence intervals, hypothesis testing, p-values, and model comparison. The distribution vocabulary from this chapter becomes the machinery for deciding whether observed data are ordinary under a proposed explanation or surprising enough to question it.
# === Section 4.13: Central Limit Theorem — Visual Demonstration ===
# Draw repeated samples from a non-normal (exponential) distribution
# and show that the distribution of sample means becomes Normal.
rng = np.random.default_rng(42)
pop = rng.exponential(scale=5.0, size=100_000) # highly right-skewed
fig, axes = plt.subplots(1, 4, figsize=(16, 3.5))
# Panel 1: the population
axes[0].hist(pop, bins=80, density=True, color="gray", edgecolor="white", alpha=0.7)
axes[0].set_title("Population\n(Exponential, λ = 5)")
# Panels 2-4: sampling distributions of the mean for increasing n
for ax, n_sample in zip(axes[1:], [5, 30, 100]):
means = [rng.choice(pop, size=n_sample).mean() for _ in range(2000)]
ax.hist(means, bins=50, density=True, color="steelblue", edgecolor="white", alpha=0.7)
ax.set_title(f"Sample means\nn = {n_sample}")
fig.suptitle("Central Limit Theorem: Sample Means → Normal", fontsize=13, y=1.02)
plt.tight_layout()
plt.show()
Section 4.14: Chapter Summary and Forward Link¶
This chapter gave uncertainty a working language. It began with experiments, outcomes, sample spaces, and events, then used the division rule to turn counts into probabilities. It separated independent from dependent events, mutually exclusive from overlapping events, and unconditional from conditional probability. It then used contingency tables and Bayes' theorem to show how probabilities are read from data and revised by evidence.
The second half of the chapter moved from individual events to distributions. A distribution describes how probability is arranged across the possible values of a random variable. The Binomial distribution models successes in fixed independent trials. The Poisson distribution models counts in fixed intervals. The Normal distribution models continuous measurements and provides the standard Normal scale through z-scores.
The most important habit is assumption checking. A formula is useful only when its conditions match the problem. Are the outcomes equally likely? Are the trials independent? Is the event overlap possible? Is a count better modeled by fixed trials or by a rate? Is a Normal approximation reasonable? These questions protect probability from becoming mechanical arithmetic.
Chapter 5 uses these ideas to begin formal inference. Once we know how probability behaves under an assumed model, we can ask whether observed data support or challenge that model.
Quick Reference¶
| Question | Tool |
|---|---|
| Probability of an event in an equally likely sample space | m / n |
| Probability of both independent events | P(A) * P(B) |
| Probability of either overlapping event | P(A) + P(B) - P(A and B) |
| Probability after conditioning on evidence | P(A and B) / P(B) |
| Updating a hypothesis with evidence | Bayes' theorem |
| Fixed independent yes/no trials | Binomial distribution |
| Counts per interval at an average rate | Poisson distribution |
| Continuous bell-shaped measurements | Normal distribution |
| Convert a value to standard-deviation units | z-score |
| Compute PMF, CDF, PPF, and random samples | scipy.stats |
References¶
SciPy Development Team. Statistical functions and probability distributions.
Python Software Foundation.
math.combandmath.factorial.NumPy Developers. Random sampling.
Blitzstein, J. K., and Hwang, J. (2019). Introduction to Probability (2nd ed.). CRC Press.
OpenStax. Introductory Business Statistics.
DeGroot, M. H., and Schervish, M. J. (2012). Probability and Statistics (4th ed.). Addison-Wesley.