4.6. Independence of Events

In the previous sections, we explored how knowledge about an event influences the probabilities involving another event. But do events always influence each other? The answer is no, and we say that such events are idnependent. In this section, we’ll dive deeper into the concept of independence.

Road map 🧭

  • Understand what it means for events to be independent.

  • Distinguish between independence and mutually exclusive events.

  • See how independence simplifies probability calculations.

  • Explore different types of independence: pairwise vs. mutual.

  • Connect independence to the concepts covered in previous chapters.

4.6.1. Independence

We say two events A and B are independent if the occurrence of one event does not affect the probability of the other. More formally, events A and B are independent if:

\[P(A|B) = P(A) \text{ and } P(B|A) = P(B)\]

The first equation means that knowing B has occurred provides no additional information about A. Likewise, the second equation says that knowing A has occurred does not provide any update on the probability of B.

The two conditions are equivalent—one implies the other. Independence is a symmetric relationship between events.

Additional equivalent expressions

Independence of A and B implies the pairwise independence of A and B’, A’ and B, and A’ and B’.

Special Multiplication Rule

When two events are independent, the general multiplication rule simplifies to:

\[P(A \cap B) = P(A) P(B).\]

‼️ Avoid the common mistake ‼️

This is a special-case rule which can only be used when the independence of A and B has been mathematically shown. When unsure, always begin with the general version (Section 4.3).

4.6.2. Independent vs. Mutually Exclusive Events

It’s important to distinguish between independence and mutual exclusivity, as these concepts are often confused but are fundamentally different. Recall their definitions:

  • Mutually exclusive (or disjoint) events cannot occur simultaneously. Their intersection is empty.

  • Independent events provide no information about each other. Knowing one occurred doesn’t change the probability of the other.

For events with positive probability, these concepts are actually incompatible. Specifically:

  1. If two events A and B are mutually exclusive, then they cannot be independent.

    If A and B are mutually exclusive, then P(A ∩ B) = 0. Using the conditional probability formula,

    \[P(A|B) = \frac{P(A \cap B)}{P(B)} = \frac{0}{P(B)} = 0.\]

    But if P(A) > 0, then P(A|B) = 0 ≠ P(A), which means A and B are not independent.

  2. If two events A and B are independent, then they cannot be mutually exclusive.

    If A and B are independent with P(A) > 0 and P(B) > 0, then

    \[P(A \cap B) = P(A) \times P(B) > 0.\]

    This means A and B have a non-empty intersection, so they are not mutually exclusive.

4.6.3. Pairwise Independence vs. Mutual Independence

When dealing with more than two events, we can define different levels of independence:

Pairwise Independence

A collection of events \(\{A_1, A_2, \cdots, A_n\}\) is pairwise independent if all pairs of events are independent.

For example, four events \(\{A_1, A_2, A_3, A_4\}\) are pairwise independence if

\[\begin{split}P(A_1|A_2) = P(A_1) \quad &\text{and} \quad P(A_1 \cap A_2) = P(A_1)P(A_2)\\ P(A_1|A_3) = P(A_1) \quad &\text{and} \quad P(A_1 \cap A_3) = P(A_1)P(A_3)\\ & \vdots\end{split}\]

for all pairs.

Mutual Independence

Mutual indepdendence of events is a stronger condition where the special multiplication rule holds for all combinations of events, not just pairs. For four events to be mutually independent, they must be pairwise independent. In addition, they must satisfy

\[\begin{split}P(A_1 \cap A_2 \cap A_3) &= P(A_1)P(A_2)P(A_3)\\ P(A_1 \cap A_2 \cap A_4) &= P(A_1)P(A_2)P(A_4)\\ &\vdots\end{split}\]

for all triplets, and

\[P(A_1 \cap A_2 \cap A_3 \cap A_4) = P(A_1)P(A_2)P(A_3)P(A_4).\]

There exist cases where events are pairwise independent but not mutually independent.

Example💡: Circuit Reliability

Consider two electrical systems with different circuit configurations.

Diagram of two circuits

Fig. 4.19 Diagram of two circuit systems

  • System 1: Four parallel paths (labeled 1, 2, 3, 4) connect points A and B.

  • System 2: Three paths (labeled 1, 2, 3) connect points A and B, but with more complex connectivity.

Each path contains mutually independent switches that are each activated with probability 0.3. The system functions if current can flow from point A to point B. We want to calculate the probability that each system will function.

System 1

  • Let \(L_i\) denote the event that the \(i\) th line is on, for each \(i=1,2,3,4\).

  • We are given that \(P(L_i)=0.3\) for all \(i\). This also implies \(P(L_i')=0.7\) for all \(i\).

  • Let \(F_1^+\) denote the event that system 1 is functioning.

For the system to function, at least one path must be on. In other words, Line 1 or Line 2 or Line 3 or Line 4 must be on. This gives:

\[P(F_1^+) = P(L_1 \cup L_2 \cup L_3 \cup L_4)\]

Using the complement rule and De Morgan’s law:

\[\begin{split}P(F_1^+) &= 1 - P((L_1 \cup L_2 \cup L_3 \cup L_4)')\\ &= 1 - P(L_1' \cap L_2' \cap L_3' \cap L_4')\end{split}\]

Since the lines operate independently, we can use the special multiplication rule:

\[\begin{split}P(F_1^+) &= 1 - P(l_1') P(l_2') P(l_3') P(l_4')\\ &= 1 - (0.7)^4 = 1 - 0.2401 = 0.7599.\end{split}\]

System 2

  • Let \(L_i\) now denote the event that the \(i\) th line is on in the second system, for each \(i=1,2,3\).

  • Let \(F_2^+\) denote the event that system 1 is functioning.

The first steps are identical to the first system.

\[\begin{split}P(F_2^+) &= P(L_1 \cup L_2 \cup L_3) = 1 - P((L_1 \cup L_2 \cup L_3)')\\ &= 1 - P(L_1' \cap L_2' \cap L_3') = 1- P(L_1')P(L_2')P(L_2')\end{split}\]

Now we need to calculate the probability that each line is not functioning.

  • \(P(L_1') = 1 - P(\text{both switches are on}) = 1-(0.3)^2 = 0.91\)

  • \(P(L_2') = 1 - 0.3 = 0.7\)

  • \(P(L_3') = 1 - P(\text{all three switches are on}) = 1-(0.3)^3 = 0.973\)

Putting it all together,

\[P(F_2^+) = 1 - (0.91 \times 0.7 \times 0.973) \approx 0.3802.\]

The difference in reliability between the two systems (76% vs. 38%) highlights how parallel paths increase reliability compared to series connections.

4.6.4. Bringing It All Together

Key Takeaways 📝

  1. Independence means that knowing one event occurs doesn’t change the probability of another event.

  2. Independence allows us to use the special multiplication rule: P(A ∩ B) = P(A) × P(B).

  3. Mutually exclusivity and independence generally do not occur simultaneously.

  4. Pairwise independence means all pairs of events are independent, while mutual independence requires that all combinations of events follow the special multiplication rule.

  5. Independence often leads to powerful simplifications in computation, but its should use should always follow mathematical justification.

Exercises

  1. Basic Independence: A standard deck of 52 cards has 4 suits (hearts, diamonds, clubs, spades) and 13 ranks (Ace, 2-10, Jack, Queen, King). You draw one card randomly. Let A be the event “the card is a heart” and B be the event “the card is a King.” Are events A and B independent? Mathematically justify your answer.

  2. Independence vs. Mutual Exclusivity: Give an example from everyday life of two events that are

    1. both independent and not mutually exclusive,

    2. mutually exclusive but not independent, and

    3. neither independent nor mutually exclusive.

  3. Circuit Reliability: Consider a circuit with three components in series. Each component works independently with probability 0.9. Calculate the probability that the circuit functions (all components must work for the circuit to function).

  4. Weather Prediction: Suppose there’s a 30% chance of rain on Saturday and a 40% chance of rain on Sunday. If the weather on these two days is independent, what is the probability that

    1. it rains on both days,

    2. it rains on at least one day,

    3. it doesn’t rain at all over the weekend?

  5. Pairwise vs. Mutual Independence: Three fair coins are tossed. Let A = “first coin shows heads,” B = “second coin shows heads,” and C = “an odd number of heads appears.” Show that these events are pairwise independent but not mutually independent.