Key Concepts
σ-algebras, probability measures, and random variables as measurable functions.
This section introduces the core building blocks of measure-theoretic probability: σ-algebras, probability measures, and the formal definition of random variables. These concepts formalize the intuitions you already have about probability.
The σ-Algebra: Measurable Events
Not every conceivable collection of outcomes can be assigned a probability. A σ-algebra (sigma-algebra) specifies which events are "measurable" — the events to which we can assign probabilities.
σ-Algebra (σ-Field)
A σ-algebra on a sample space is a collection of subsets of that is:
- Contains the whole space:
- Closed under complements: If , then
- Closed under countable unions: If , then
Read: “F is a sigma-algebra on Omega if it contains Omega and is closed under complements and countable unions”
A collection of events that is 'complete' — if you can measure an event, you can also measure its opposite and any countable combination
The Borel σ-Algebra
For the real line , the most commonly used σ-algebra is the Borel σ-algebra . It contains:
- All intervals:
- All open and closed sets
- Any countable union or intersection of these
Almost any set you can describe explicitly is Borel-measurable. Non-measurable sets exist but require the Axiom of Choice to construct and never arise in practice.
Why σ-Algebras Matter
The σ-algebra defines what questions we can ask about our random experiment. If an event isn't in the σ-algebra, we cannot assign it a probability. For most practical purposes, the Borel σ-algebra contains every event you'll ever care about.
Visualizing σ-Algebra Properties
The entire sample space is always measurable.
If you can measure an event, you can measure its opposite.
Countably many measurable events can be combined.
Derived Properties:
Why does this matter for fat tails? The σ-algebra determines what questions we can ask about our random variable. For continuous distributions, we can ask about intervals (like P(X > 5)), but not about non-measurable sets.
For practical purposes, the Borel σ-algebra on ℝ contains every set you'll ever need — all intervals, their unions, intersections, and limits. This is the standard setup for defining continuous random variables.
Probability Measures
A probability measure assigns numerical probabilities to events in a way that is consistent with our intuitions about probability.
Probability Measure
A probability measure on is a function satisfying:
- Normalization: (something happens)
- Non-negativity: for all
- Countable additivity: For disjoint events :
These axioms (Kolmogorov's axioms) imply many familiar properties:
- (the impossible event has probability 0)
- (complement rule)
- (inclusion-exclusion)
- If , then (monotonicity)
Uniform Distribution on [0, 1]
The uniform distribution on [0, 1] is defined by setting for any interval . This assigns probabilities proportional to length.
For individual points: . Points have zero length, hence zero probability.
The Probability Space
Probability Space
A probability space is a triple where:
- is the sample space (all possible outcomes)
- is a σ-algebra on (measurable events)
- is a probability measure on
The probability space is the complete mathematical description of a random experiment. Every statement in probability theory is made with respect to some (often implicit) probability space.
Random Variables as Measurable Functions
In measure theory, a random variable is defined as a functionthat maps outcomes to numbers in a "nice" way.
Random Variable (Formal)
A random variable is a measurable function . "Measurable" means that for any Borel set :
Read: “X inverse of B is the set of all omega in Omega such that X of omega is in B”
The pre-image of any Borel set must be a measurable event — we can ask about the probability that X lands in any reasonable set
In simpler terms: if we want to compute for some set of real numbers, the set of outcomes that produce values in must be an event in our σ-algebra.
Why Measurability Matters
Measurability ensures we can ask questions like "what is ?" The condition guarantees that is an event we can assign probability to. Without measurability, probability questions might be undefined.
Expected Value as a Lebesgue Integral
The expected value is defined using the Lebesgue integral, which is more general than the Riemann integral from calculus.
Expected Value (Lebesgue)
For a random variable on probability space , the expected value is:
This integral is well-defined when .
For a continuous random variable with PDF , this becomes the familiar formula:
When the Integral Diverges
For the Pareto distribution with :
The integral diverges, so the expected value doesn't exist. This is a precise statement: the Lebesgue integral is not finite, so we cannot define as a real number.
Existence vs. Convergence
In measure theory, we distinguish between an integral existing (being well-defined and finite) and a sequence of approximations converging. For fat-tailed distributions, the expected value integral may not exist — this is why sample means behave erratically.
Key Takeaways
- A σ-algebra specifies which events can be assigned probabilities — it's closed under complements and countable unions
- A probability measure assigns probabilities satisfying:, non-negativity, and countable additivity
- A probability space is the complete mathematical model of a random experiment
- A random variable is a measurable function
- Expected value is defined as a Lebesgue integral:
- For fat-tailed distributions, this integral may diverge, meaning the expected value doesn't exist as a finite number