17.7: Independence

Last updated
Save as PDF

Page ID: 48428

Eric Lehman, F. Thomson Leighton, & Alberty R. Meyer
Google and Massachusetts Institute of Technology via MIT OpenCourseWare

\( \newcommand{\vecs}[1]{\overset { \scriptstyle \rightharpoonup} {\mathbf{#1}} } \) \( \newcommand{\vecd}[1]{\overset{-\!-\!\rightharpoonup}{\vphantom{a}\smash {#1}}} \)\(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\) \(\newcommand{\id}{\mathrm{id}}\) \( \newcommand{\Span}{\mathrm{span}}\) \( \newcommand{\kernel}{\mathrm{null}\,}\) \( \newcommand{\range}{\mathrm{range}\,}\) \( \newcommand{\RealPart}{\mathrm{Re}}\) \( \newcommand{\ImaginaryPart}{\mathrm{Im}}\) \( \newcommand{\Argument}{\mathrm{Arg}}\) \( \newcommand{\norm}[1]{\| #1 \|}\) \( \newcommand{\inner}[2]{\langle #1, #2 \rangle}\) \( \newcommand{\Span}{\mathrm{span}}\)\(\newcommand{\AA}{\unicode[.8,0]{x212B}}\)

Suppose that we flip two fair coins simultaneously on opposite sides of a room. Intuitively, the way one coin lands does not affect the way the other coin lands. The mathematical concept that captures this intuition is called independence.

Definition \(\PageIndex{1}\)

An event with probability 0 is defined to be independent of every event (including itself). If \(\text{Pr}[B] \neq 0\), then event \(A\) is independent of event \(B\) iff

\[\label{17.7.1} \text{Pr}[A \mid B] = \text{Pr}[A].\]

In other words, \(A\) and \(B\) are independent if knowing that \(B\) happens does not alter the probability that \(A\) happens, as is the case with flipping two coins on opposite sides of a room.

Potential Pitfall

Students sometimes get the idea that disjoint events are independent. The opposite is true: if \(A \cap B = \emptyset\), then knowing that \(A\) happens means you know that \(B\) does not happen. Disjoint events are never independent—unless one of them has probability zero.

Alternative Formulation

Sometimes it is useful to express independence in an alternate form which follows immediately from Definition 17.7.1:

Theorem \(\PageIndex{2}\)

\(A\) is independent of \(B\) if and only if

\[\label{17.7.2} \text{Pr}[A \cap B] = \text{Pr}[A] \cdot \text{Pr}[B].\]

Notice that Theorem 17.7.2 makes apparent the symmetry between \(A\) being independent of \(B\) and \(B\) being independent of \(A\):

Corollary 17.7.3. \(A\) is independent of \(B\) iff \(B\) is independent of \(A\).

Independence Is an Assumption

Generally, independence is something that you assume in modeling a phenomenon. For example, consider the experiment of flipping two fair coins. Let \(A\) be the event that the first coin comes up heads, and let \(B\) be the event that the second coin is heads. If we assume that \(A\) and \(B\) are independent, then the probability that both coins come up heads is:

\[\nonumber \text{Pr}[A \cap B] = \text{Pr}[A] \cdot \text{Pr}[B] = \frac{1}{2} \cdot \frac{1}{2} = \frac{1}{4}.\]

In this example, the assumption of independence is reasonable. The result of one coin toss should have negligible impact on the outcome of the other coin toss. And if we were to repeat the experiment many times, we would be likely to have \(A \cap B\) about \(1/4\) of the time.

On the other hand, there are many examples of events where assuming independence isn’t justified. For example, an hourly weather forecast for a clear day might list a 10% chance of rain every hour from noon to midnight, meaning each hour has a 90% chance of being dry. But that does not imply that the odds of a rainless day are a mere \(0.9^{12} \approx 0.28\). In reality, if it doesn’t rain as of 5pm, the odds are higher than 90% that it will stay dry at 6pm as well—and if it starts pouring at 5pm, the chances are much higher than 10% that it will still be rainy an hour later.

Deciding when to assume that events are independent is a tricky business. In practice, there are strong motivations to assume independence since many useful formulas (such as equation (\ref{17.7.2})) only hold if the events are independent. But you need to be careful: we’ll describe several famous examples where (false) assumptions of independence led to trouble. This problem gets even trickier when there are more than two events in play.