The Exponential Distribution
The memoryless distribution that marks the boundary between thin and fat tails.
The exponential distribution models waiting times for random events and is characterized by its unique “memoryless” property. In the landscape of tail behavior, it serves as the critical boundary between thin-tailed and fat-tailed distributions.
The Exponential Function
Before diving into the exponential distribution, let's understand the mathematical function at its core: and its decaying counterpart .
Euler's Number
The constant e (Euler's number) is approximately 2.71828... It's defined as:
The function is the unique function that equals its own derivative:.
Read: “e to the x”
The exponential function — growth or decay at a rate proportional to the current value
When the exponent is negative, we get exponential decay:
Read: “e to the minus lambda x”
Exponential decay — starts at 1 when x = 0 and smoothly approaches 0 as x grows
This decay function is the heart of the exponential distribution. The parameter λ controls how quickly the function decays: larger λ means faster decay.
Explore: The Exponential Function
Experiment with different bases to see why e is special, then explore how the rate parameter λ controls exponential decay.
Exponential Growth: y = bx
Compare how different bases grow. Euler's number e ≈ 2.718 is special: the function y = ex is its own derivative.
Key properties of ex:
- e0 = 1 (any number to the power 0 is 1)
- ex > 0 for all x (always positive)
- As x → ∞, ex → ∞ (explosive growth)
- As x → −∞, ex → 0 (approaches but never reaches 0)
Exponential Decay: y = e−λx
When the exponent is negative, the function decays toward zero. The rate λ controls how fast it decays — larger λ means faster decay.
Why e−λx models “forgetting”:
- Starts at 1 when x = 0
- Smoothly decreases toward 0 as x increases
- The rate of decay at any point is proportional to the current value
- This “constant proportional decay” is what makes it memoryless
Why This Matters for Probability
The exponential decay is the perfect model for “forgetting” — at any point, the rate of decrease is proportional to the current value. This property translates directly into the memoryless property of the exponential distribution: the past provides no information about the future.
The Exponential PDF
Exponential Distribution
A random variable follows an exponentialdistribution with rate parameter λ () if its PDF is:
We write .
Key quantities:
- Mean:
- Variance:
- Standard deviation: (equals the mean!)
The Survival Function
The survival function of the exponential has a beautifully simple form:
Read: “S of x equals e to the minus lambda x”
The probability of surviving beyond time x decays exponentially
And the CDF is:
The Memoryless Property
Memoryless Property
The exponential distribution satisfies:
Read: “The probability that X exceeds s plus t, given that X exceeds s, equals the probability that X exceeds t”
If you've already waited s units of time, your expected additional wait is the same as if you'd just started
Waiting for a Bus
If buses arrive according to a Poisson process with rate λ = 0.1 per minute (average wait: 10 minutes), and you've already waited 5 minutes, the distribution of your remaining wait is still Exp(0.1).
Your expected additional wait is still 10 minutes — the past 5 minutes of waiting provide no information. The exponential distribution “forgets” the past.
Uniqueness
The exponential is the only continuous distribution with the memoryless property. (For discrete distributions, it's the geometric.) This makes it natural for modeling radioactive decay, service times, and other “forgetful” processes.
Explore: The Memoryless Property
Adjust the rate λ and observe the survival function. See how the conditional distribution after waiting s units looks identical to the original distribution.
The Memoryless Property in Action
Increase “time already waited” (s) to see how the conditional distribution compares to the original. They're identical (just shifted)!
Probability Density Function
Connection to the Poisson Process
The exponential distribution arises naturally from the Poisson process:
- If events occur according to a Poisson process with rate λ, the time between consecutive events follows Exp(λ).
- The number of events in time t follows Poisson(λt).
This connection makes the exponential distribution fundamental to queuing theory, reliability engineering, and any domain where we model random arrivals.
The Boundary Between Thin and Fat Tails
The exponential distribution occupies a special place in the taxonomy of tail behavior:
Comparing decay rates for large x:
- Gaussian: — much faster than exponential
- Exponential: — the “baseline”
- Pareto: — much slower than exponential
Defining Fat Tails
A distribution is called fat-tailed (or heavy-tailed) if its tails decay slower than exponential. This is formalized through the concept of “subexponential” distributions, which we'll explore in Module 3.
The exponential sits exactly at the boundary: all its moments exist, but just barely. Any slower decay (like power law) and moments start to blow up.
All Moments Exist
The n-th moment of the exponential distribution is:
All moments exist and are finite because the exponential decay dominates any polynomial for large x.
Computing Moments
For :
All finite, though they grow rapidly with n.
Where the Exponential Applies
- Radioactive decay: Time until an atom decays (original application)
- Service times: Time to complete a service in queuing systems
- Inter-arrival times: Time between Poisson events
- Lifetimes: Time until failure for components without “wear”
However, the memoryless property often doesn't hold in reality:
- Car parts wear out — failure probability increases with age
- Human lifetimes have “aging” — not memoryless
- Financial crises cluster — past matters
Key Takeaways
- The exponential has PDF and survival
- It's the unique continuous memoryless distribution
- All moments exist:
- It defines the boundary between thin and fat tails
- Distributions with slower-than-exponential tail decay are called fat-tailed
- Naturally arises from Poisson processes