The Poisson distribution is a discrete probability distribution with identical mean and variance that expresses the probability of a given number of events occurring in a fixed interval of time. In this post, we derive the Poisson probability mass function beginning with the Binomial PDF.
The binomial distribution represents the number of successes, \(k\) in \(n\) independent Bernoulli trials, where the probability of success \(p\) for any given trial is \(0 \leq p \leq 1\). Formally, the binomial PDF is given by
\[ f(k; p, n) = \binom{n}{k}p^{k}(1-p)^{n-k}, \]
where \(\binom{n}{k}\) represents the binomial coefficient, which equates to
\[ \binom{n}{k} = \frac{n!}{n!(n-k)!}. \]
The expanded binomial PDF then appears as
\[ f(k; p, n) = \frac{n!}{n!(n-k)!}p^{k}(1-p)^{n-k}. \]
The mean of the binomial distribution is
\[ \mu = \lambda = np, \]
which can be rearranged in terms of \(p\) yielding
\[ p = \frac{\lambda}{n}. \]
Substituting this expression for \(p\) into the binomial PDF results in
\[ f(k; p, n) = \frac{n!}{n!(n-k)!}\Big(\frac{\lambda}{n}\Big)^{k}\Big(1-\frac{\lambda}{n}\Big)^{n-k} \]
If we let \(n\) grow very large and \(p\) grow very small, after a bit of re-expression we can formulate the binomial distribution as
\[ \lim_{n\to\infty , p \to 0} f(k; p, n) = \frac{n(n-1) \cdots (n-k+1)}{k!} \frac{\lambda^{k}}{n^{k}} \Big(1-\frac{\lambda}{n}\Big)^{n} \Big(1-\frac{\lambda}{n}\Big)^{-k}. \]
Swapping denominator terms yields
\[ = \frac{n(n-1) \cdots (n-k+1)}{n^{k}} \frac{\lambda^{k}}{k!} \Big(1-\frac{\lambda}{n}\Big)^{n} \Big(1-\frac{\lambda}{n}\Big)^{-k} \]
Recall from calculus that \(e = \lim_{k\to\infty} (1 + 1/k)^{k}\) and similarly \(e^{x} = \lim_{k\to\infty} (1 + x/k)^{k}\). Leveraging these identities, \((1 - \frac{\lambda}{n})^{n}\) becomes \(e^{-\lambda}\) in the limit. With very large \(n\), the expression for the PDF reduces to
\[ \lim_{n\to\infty} f(k; p, n) = (1) \Big( \frac{\lambda^{k}}{k!}\Big)(e^{-\lambda})(1). \]
After a little cleanup, we obtain the familiar expression for the Poisson density with mean \(\lambda\):
\[ f(k; \lambda) = \frac{\lambda^{k}}{k!}e^{-\lambda} \hspace{1cm} k \in 0, 1, 2, \cdots. \]
An interesting property of the Poisson distribution (as well as the binomial and negative binomial distributions) is that successive probabilities can be found recursively. The recurrence relation is given by
\[ \frac{p_{k}}{p_{k-1}} = a +\frac{b}{k}, \hspace{2mm} k = 2,3,4, \cdots, \]
where in the case of the Poisson distribution, \(a=0\) and \(b=\lambda\), resulting in
\[ p_{k} = \frac{\lambda}{k}p_{k-1}, \hspace{2mm} k = 1,2,3, \cdots. \]
To demonstrate, for a system approximated by a Poisson distribution with \(\lambda = 3\), the probabilities for \(0 \leq k \leq 6\) are:
\[ \begin{align*} p_{0} &= .049787068 \\ p_{1} &= .149361205 \\ p_{2} &= .224041808 \\ p_{3} &= .224041808 \\ p_{4} &= .168031356 \\ p_{5} &= .100818813 \\ p_{6} &= .050409407. \end{align*} \]
To use the recurrence relation, we first calculate \(p_0\) as above. Then:
\[ \begin{align*} p_{0} &=.049787068 \\ p_{1} &=p_{0}\frac{\lambda}{k} =.049787068*\frac{3}{1}=.149361205 \\ p_{2} &=p_{1}\frac{\lambda}{k} =.149361205*\frac{3}{2}=.224041808 \\ p_{3} &=p_{2}\frac{\lambda}{k} =.224041808*\frac{3}{3}=.224041808 \\ p_{4} &=p_{3}\frac{\lambda}{k} =.224041808*\frac{3}{4}=.168031356 \\ p_{5} &=p_{4}\frac{\lambda}{k} =.168031356*\frac{3}{5}=.100818813 \\ p_{6} &=p_{5}\frac{\lambda}{k} =.100818813*\frac{3}{6}=.050409407, \end{align*} \]
which are identical to the probabilities obtained from evaluating the Poisson PDF directly.