25 Lecture 24, March 08, 2024

25.1 Proof of the moments of exponential distribution

Recall that we can have many ways to show the expectation and the variance of \(X\sim Exp(\theta)\) using the Gamma function. Then we have \[ \mathbb{E}(X)=\theta\] and \[\mathbb{V}ar(X)=\theta^2.\]

Let \(X\sim Exp(\theta)\). We use the change of variable \(y=x\theta\) with \(dx =\theta dy\) \[\begin{align*} E[X]&= \int_0^\infty x \cdot \frac{1}{\theta} e^{-\frac{x}{\theta}}dx \overset{y=x/\theta}{=} \int_0^\infty y e^{-y} \theta dy\\ &= \theta \underbrace{\int_0^\infty y e^{-y} dy}_{=\Gamma(2)}= \theta \Gamma(2) = \theta \cdot (1!) =\theta \end{align*}\] and similarly \[\begin{align*} E[X^2]&= \int_0^\infty x^2 \cdot \frac{1}{\theta} e^{-\frac{x}{\theta}}dx\overset{y=x/\theta}{=} \int_0^\infty \theta y^2 e^{-y} \theta dy\\ &= \theta^2 \underbrace{\int_0^\infty y^{3-1} e^{-y} dy}_{=\Gamma(3)}= \theta^2 \Gamma(3) = \theta \cdot (2!) =2\theta^2 \end{align*}\] so that \[ Var(X) = E[X^2]-E[X]^2=2\theta^2-\theta^2 =\theta^2\]

25.1.1 Memoryless property of exponential distribution

Theorem 25.1 (Memoryless property) If \(X \sim Exp(\theta)\), then \[ P(X > s + t | X > s) = P(X > t). \]

• We’ve seen the memoryless property for the \(Geo(p)\) earlier (and the geometric distribution is the only discrete distribution with this property)
• If a continuous random variable has memoryless property, it must follow exponential distribution.
• Intuitively, both the geometric and exponential distributions measure waiting time until first success

Proof. Recall the cdf of \(X\sim Exp(\theta)\) is \[ F(x)=P(X\leq x) =\int_{-\infty}^x f(t)dt = \int_0^x \theta^{-1} e^{-t/\theta}dt = 1-\exp(-x/\theta)\] for \(x>0\) and 0 otherwise. Hence, \[\begin{align*} P(X > s + t | X > s) &= \frac{P( X>s+t \text{ and }X>s)}{P(X>s)}\\ &= \frac{P(X>s+t)}{P(X>s)} = \frac{e^{-(s+t)/\theta}}{e^{-s/\theta}}\\ &= e^{-t/\theta} = 1-F(t) = P(X>t) \end{align*}\] as desired.