4.3. Conditional Probability

With the foundations of probability established, we can now extend our toolset to explore how events influence each other. In real-world scenarios, we often have partial information about an experiment’s outcome, and need to update our probability assessments as more information is gathered. Conditional probability provides the mathematical framework for incorporating this new information and analyzing relationships between events.

Road Map 🧭

  • Define conditional probability and understand its interpretation.

  • Derive the conditional probability formula and the general multiplication rule.

  • Understand how tree diagrams represent conditional probabilities graphically.

4.3.1. Understanding Conditional Probability

Conditional probability addresses how the probability of one event changes when we know that another event has occurred. It represents the revised probability assessment for an event based on new information.

For example, if an AI model is trying to detect wolves in an image, knowing that the image contains large carnivores increases the probability that the image contains wolves. The additional information about large carnivores changes, or conditions, the probability assessment for wolves.

Definition and Notation

The conditional probability of event A given that event B has occurred is denoted by P(A|B), read as “the probability of A given B.”

The conditional probability is computed as

\[P(A|B) = \frac{P(A \cap B)}{P(B)}.\]

This formula is valid only when P(B) > 0 (we cannot condition on impossible events).

What is happening in the conditional probability formula?

A probability, whether conditional or not, represents the relative size of a part out of a whole.

In non-conditional situations, the relative size of event A is assessed out of the whole sample space.

Venn diagram illustration of non-conditional probability

Fig. 4.11 Non-conditional probability

However, by knowing that event B occurs, our sample space becomes restricted to only the outcomes in B -anything that does not belong to B is not a potential outcome anymore. Within this reduced sample space, we’re interested in the relative size of the regoin which belongs to event A.

Venn diagram illustration of conditional probability

Fig. 4.12 Conditional probability

Example: Cards from a Deck

Suppse we are drawing a card from a standard deck with 52 cards. Define H as the event of drawing a heart and Q as the event of drawing a queen.

A standard 52-card deck

Fig. 4.13 A standard 52-card deck

  1. A card has been drawn. If it is known that the card is a heart, what is the probability that it is a queen?

    • \(P(Q ∩ H) = 1/52\) (only one card is both a queen and a heart)

    • \(P(H) = 13/52 = 1/4\) (there are thirteen hearts)

    \[P(Q|H) = \frac{P(Q ∩ H)}{P(H)} = \frac{1/52}{13/52} = \frac{1}{13}.\]

  2. If it is known that the card is a queen, what is the probability that it is a heart?

    \(P(Q) = 4/52 = 1/13\) (there are four queens). Therefore,

    \[P(H|Q) = \frac{P(Q ∩ H)}{P(Q)} = \frac{1/52}{1/13} = \frac{1}{4}.\]

4.3.2. The General Multiplication Rule

The conditional probability formula can be rearranged to give us the general multiplication rule:

\[P(A \cap B) = P(A|B) \times P(B)\]

This rule allows us to find the probability of the intersection of two events when we know a conditional probability and an unconditional probability.

The formula can also be written as:

\[P(A \cap B) = P(B|A) \times P(A)\]

Both forms are valid, and which one to use depends on what information is available in a given problem.

Extending the general multiplication rule to multiple events

  1. Begin by taking an unconditoined probability of any one event.

  2. Multiply the probabiity of a new event conditional on the event used for step 1.

  3. Multiply the probability of another new event conditional on the intersection of all previously used events.

  4. Continue in a similar manner.

For three events A, B, and C, there are six different ways to apply the multiplication rule:

\[\begin{split}P(A \cap B \cap C) &= P(A) \times P(B|A) \times P(C|A \cap B) \\ P(A \cap B \cap C) &= P(A) \times P(C|A) \times P(B|A \cap C) \\ P(A \cap B \cap C) &= P(B) \times P(A|B) \times P(C|A \cap B) \\ &\vdots\end{split}\]

4.3.3. Tree Diagrams

Tree diagrams provide a visual tool for organizing and calculating probabilities in multi-stage experiments, especially when conditional probabilities are involved.

Constructing a Tree Diagram

A general tree diagram

Fig. 4.14 A general tree diagram; the superscript \(^c\) is another notation for complement.

A tree diagram is constructed using the following rules:

A. Layers

The different stages are organized into vertical layers. The first vertical layer describes all events that happen during the first stage. The second layer describe all events that happen in the second stage, and the process is repeated if there are more than two stages.

B. Nodes

Each node represents an event conditional on all previous nodes. Since the first layer does not have any previous node, this is the only layer which involves unconditional events.

C. Branches

The set of branches stemming from a single node must cover all scenarios that can happen given the previous path of nodes. On each branch leading to a node, we wirte the conditional probability of the node.

Why is a tree diagram convenient?

  • It provides a comprehensive picture of an experiment with many components.

  • It allows us to view the general multiplication rule graphically, since the probability of a path can be computed by multiplying all the conditional probabilities along the path.

    In a two-layer tree diagram, for example, \(P(A' \cap B')\) can be computed by

    Computing a path probability using a tree diagram

    Fig. 4.15 Computing a path probability using a tree diagram

Tree diagrams will also play a crucial role in illustrating important concepts such as the law of total probability in the following sections.

Example :Drawing a tree diagram

Glen and Jia are going to Indianapolis for one day this weekend. They are twice as likely to go on Sunday as they are on Friday, and three times as likely to go on Saturday as they are on Friday. There is a 45% chance of rain on Friday, a 25% chance of rain on Saturday, and a 30% chance of rain on Sunday.

  1. Draw a complete tree diagram to represent this situation.

    Let’s define our notation:

    • Fri, Sat, Sun = events representing which day they go to Indianapolis

    • R = “it rains during their trip”

    • R’ = “it does not rain during their trip”

    The tree diagram for the Indianapolis problem

    Step 1: First, we need to find the unknown probabilities.

    Given that they are twice as likely to go on Sunday as on Friday, and three times as likely to go on Saturday as on Friday, we can write:

    \[P(Sun) = 2P(Fri) \hspace{0.2cm}\text{ and }\hspace{0.2cm} P(Sat) = 3P(Fri).\]

    Since they will go on exactly one of the three days:

    \[\begin{split}1 &= P(Fri) + P(Sat) + P(Sun) \\ &= P(Fri) + 3P(Fri) + 2P(Fri) = 6P(Fri)\\\end{split}\]

    Therefore, \(P(Fri) = 1/6\), and

    \[P(Sat) = 3P(Fri) = 1/2 \hspace{0.2cm}\text{ and }\hspace{0.2cm} P(Sun) = 2P(Fri) = 1/3.\]

    Step 2: We know the conditional probabilities from the weather forecasts.

    \[\begin{split}P(R|Fri) &= 0.45 \\ P(R|Sat) &= 0.25 \\ P(R|Sun) &= 0.30\end{split}\]

    Using the complement rule, we can also find:

    \[\begin{split}P(R'|Fri) = 1 - P(R|Fri)&= 0.55 \\ P(R'|Sat) = 1 - P(R|Sat)&= 0.75 \\ P(R'|Sun) = 1 - P(R|Sun)&= 0.70\end{split}\]

  2. Find the probability that they go to Indianapolis on Sunday and it does not rain.

    \[P(R' \cap Sun) = P(Sun)P(R'|Sun) = (1/3)(0.70) = 7/30.\]

4.3.4. Bringing It All Together

Key Takeaways 📝

  1. Conditional probability P(A|B) represents the probability of event A occurring, given that event B occurs, and is calculated as P(A ∩ B)/P(B).

  2. When we condition on an event B, we effectively restrict our sample space to only the outcomes in B, creating a new probability measure within this reduced space.

  3. The general multiplication rule P(A ∩ B) = P(A|B) × P(B) allows us to find the probability of intersections using conditional probabilities.

  4. Tree diagrams provide a systematic way to organize and calculate probabilities in multi-stage experiments, with the probability of each path found by multiplying conditional probabilities along branches.

  5. When solving conditional probability problems, clearly define the events, identify what is known, and determine what needs to be calculated before applying the appropriate formulas.

Exercises

  1. Basic Calculation: A survey found that 70% of students attend morning classes, 80% attend afternoon classes, and 60% attend both morning and afternoon classes. If a student is selected at random:

    1. What is the probability that the student attends afternoon classes given that they attend morning classes?

    2. What is the probability that the student attends morning classes given that they attend afternoon classes?

  2. Card Problems: A card is drawn from a standard 52-card deck. Calculate:

    1. The probability that the card is a king, given that it is a face card.

    2. The probability that the card is a heart, given that it is red.

    3. The probability that the card is a red king, given that it is a king.

  3. Tree Diagram Application: A bag contains 4 red, 3 blue, and 2 green marbles. Two marbles are drawn without replacement. a) Draw a tree diagram representing this experiment. b) What is the probability that both marbles are the same color?