Worksheet 4: Independence and Random Variables

Learning Objectives 🎯

  • Understand independence and mutual independence of events

  • Apply De Morgan’s Law and the Inclusion-Exclusion Principle

  • Distinguish between independent and mutually exclusive events

  • Define random variables as mappings from outcomes to numbers

  • Work with probability mass functions (PMFs)

  • Calculate marginal distributions from joint PMFs

Part 1: Independence Property

The independence property states that if two events are known to be independent then the occurrence of one event does not affect the probability of the other. Similarly, two events are dependent if the occurrence of one event changes the probability of the other event occurring.

For two non-empty events \(A\) and \(B\) belonging to the same sample space \(\Omega\):

If \(A\) is independent of \(B\), then it implies that:

  • \(P(A|B) = P(A)\)

  • \(P(B|A) = P(B)\)

  • The special multiplication rule holds: \(P(A \cap B) = P(A)P(B)\)

  • All complementary events are also independent

    Example: Suppose \(A\) is independent of \(B\)

    \[P(A'|B) = 1 - P(A|B) = 1 - P(A) = P(A')\]

    Therefore, \(A'\) is also independent of \(B\).

If \(A\) and \(B\) are dependent, then:

  • \(P(A|B) \neq P(A)\) and \(P(B|A) \neq P(B)\)

  • Cannot use special multiplication rule! Must resort to general multiplication rule \(P(A \cap B) = P(B|A)P(A) = P(A|B)P(B)\) or other methods.

Warning

Always check what properties the sets you are working with have before trying to calculate any probabilities. Do not make up your own rules!

Question 1: Let \(A\), \(B\), and \(C\) be three events belonging to the same sample space \(\Omega\) with the following probabilities:

\[ \begin{align}\begin{aligned}P(A) = 0.6 \quad P(B) = 0.7 \quad P(C) = 0.2\\P(A \cap B) = 0.6 \quad P(A \cap C) = 0.12 \quad P(B \cap C) = 0.14\\P(A \cap B \cap C) = 0.12\end{aligned}\end{align} \]

  1. Are the three events mutually independent? Either mathematically show they are mutually independent or provide a counter example to show they are not.

    Note

    For mutual independence, we need:

    • Pairwise independence: \(P(A \cap B) = P(A)P(B)\), \(P(A \cap C) = P(A)P(C)\), \(P(B \cap C) = P(B)P(C)\)

    • Three-way independence: \(P(A \cap B \cap C) = P(A)P(B)P(C)\)

  2. Compute \(P(A' \cap B' \cap C')\) by combining De Morgan’s Law, Complement Rule, and Inclusion-Exclusion Principle.

    Hint

    De Morgan’s Law: \((A \cup B \cup C)' = A' \cap B' \cap C'\)

  3. If you determined the sets are mutually independent, recalculate \(P(A' \cap B' \cap C')\) utilizing the property of mutual independence. If you determine they were not mutually independent, still calculate \(P(A' \cap B' \cap C')\) as if they were and compare with your answer in part b.

R Code for Verification:

# Given probabilities
p_A <- 0.6
p_B <- 0.7
p_C <- 0.2
p_AB <- 0.6
p_AC <- 0.12
p_BC <- 0.14
p_ABC <- 0.12

# Check pairwise independence
cat("Checking pairwise independence:\n")
cat("P(A)P(B) =", p_A * p_B, "vs P(A∩B) =", p_AB, "\n")
cat("P(A)P(C) =", p_A * p_C, "vs P(A∩C) =", p_AC, "\n")
cat("P(B)P(C) =", p_B * p_C, "vs P(B∩C) =", p_BC, "\n")

# Check three-way independence
cat("\nChecking three-way independence:\n")
cat("P(A)P(B)P(C) =", p_A * p_B * p_C, "vs P(A∩B∩C) =", p_ABC, "\n")

# Calculate P(A' ∩ B' ∩ C') using inclusion-exclusion
p_union <- p_A + p_B + p_C - p_AB - p_AC - p_BC + p_ABC
p_complement <- 1 - p_union
cat("\nP(A' ∩ B' ∩ C') using inclusion-exclusion:", p_complement, "\n")

# If mutually independent
p_complement_indep <- (1 - p_A) * (1 - p_B) * (1 - p_C)
cat("P(A' ∩ B' ∩ C') if mutually independent:", p_complement_indep, "\n")

Question 2: Repeat the analysis with the following probabilities:

\[ \begin{align}\begin{aligned}P(A) = 0.6 \quad P(B) = 0.7 \quad P(C) = 0.2\\P(A \cap B) = 0.42 \quad P(A \cap C) = 0.12 \quad P(B \cap C) = 0.14\\P(A \cap B \cap C) = 0.084\end{aligned}\end{align} \]

  1. Are the three events mutually independent?

  2. Compute \(P(A' \cap B' \cap C')\) by combining De Morgan’s Law, Complement Rule, and Inclusion-Exclusion Principle.

  3. Recalculate \(P(A' \cap B' \cap C')\) utilizing the property of mutual independence (if applicable) and compare results.

Part 2: Independent vs. Mutually Exclusive Events

Critical Distinction 🔍

It is important to understand the difference between independent events and mutually exclusive (disjoint) events—these concepts may sound similar but are fundamentally different!

Independent events have no influence on each other. The outcome of one event does not affect the probability of the other event occurring. For example, flipping a coin and rolling a die are independent events because knowing the coin’s result doesn’t tell you anything about the die roll’s outcome. Independent events can both happen together. In fact, their probability of occurring together is given by \(P(A \cap B) = P(A)P(B)\).

Mutually exclusive events, on the other hand, cannot occur at the same time. If one event happens, the other is guaranteed not to happen. For instance, rolling a 6 and rolling an odd number on a single die are mutually exclusive because a single die roll cannot satisfy both conditions at once. Mutually exclusive events can never happen together. If events A and B are mutually exclusive, \(P(A \cap B) = 0\).

Question 3: In a distant galaxy, treasure hunters search for magical chests containing two gems: one from the Red Nebula and one from the Blue Comet. Each gem is assigned an integer value from 1 to 100, chosen randomly with equally likely outcomes.

Consider these events for a given chest:

  • Event R: The Red Nebula gem’s value is a prime number. Recall a prime number is a positive integer greater than 1 that has exactly two distinct positive divisors: 1 and itself.

  • Event B: The Blue Comet gem’s value is a perfect square. A perfect square is a number that can be expressed as the square of an integer. In other words, if \(n\) is a perfect square, there exists some integer \(k\) such that: \(n = k^2\).

  • Event T: The total value (Red + Blue) is greater than 120.

  • Event U: The total value (Red + Blue) is less than or equal to 120.

  1. Are the events \(R\) and \(B\) independent or mutually exclusive? Justify your answer.

  2. Are the events \(T\) and \(U\) independent or mutually exclusive? Justify your answer.

R Code for Investigation:

# Define prime numbers from 1 to 100
is_prime <- function(n) {
  if (n <= 1) return(FALSE)
  if (n == 2) return(TRUE)
  if (n %% 2 == 0) return(FALSE)
  for (i in seq(3, sqrt(n), by = 2)) {
    if (n %% i == 0) return(FALSE)
  }
  return(TRUE)
}

primes <- sapply(1:100, is_prime)
prime_values <- which(primes)

# Define perfect squares from 1 to 100
perfect_squares <- (1:10)^2

cat("Prime numbers (1-100):", length(prime_values), "values\n")
cat("Perfect squares (1-100):", length(perfect_squares), "values\n")

# Check for overlap
overlap <- intersect(prime_values, perfect_squares)
cat("Numbers that are both prime AND perfect square:", overlap, "\n")

# Probabilities
p_R <- length(prime_values) / 100
p_B <- length(perfect_squares) / 100
p_RB <- length(overlap) / 100

cat("\nP(R) =", p_R, "\n")
cat("P(B) =", p_B, "\n")
cat("P(R ∩ B) =", p_RB, "\n")
cat("P(R) × P(B) =", p_R * p_B, "\n")

# Check independence
if (abs(p_RB - p_R * p_B) < 1e-10) {
  cat("R and B are independent\n")
} else {
  cat("R and B are NOT independent\n")
}

# Check mutual exclusivity
if (p_RB == 0) {
  cat("R and B are mutually exclusive\n")
} else {
  cat("R and B are NOT mutually exclusive\n")
}

Part 3: Introduction to Random Variables

A random variable (r.v.) is a function that maps each outcome \(\omega\) in a sample space \(\Omega\) to a unique numerical value. That is, for any outcome \(\omega \in \Omega\), the random variable produces a value \(X(\omega)\). Despite its name, a random variable is not truly a “variable” in the traditional sense, nor is it inherently random. Instead, it is a deterministic function that maps random outcomes to numerical values. By translating random outcomes into numbers, random variables provide a framework to analyze and understand random processes systematically.

Probabilities are assigned to events through the inverse mapping of the random variable. The probability that a random variable \(X\) takes on a specific value \(x\), denoted as \(P(X = x)\) corresponds to the probability of the set of outcomes \(\omega \in \Omega\) for which \(X(\omega) = x\). Similarly, for an interval, \(P(a \leq X \leq b)\) is the probability of the set of outcomes for which \(a \leq X(\omega) \leq b\).

Question 4: In a secured vault, a sealed treasure chest is known to contain exactly three coins. Each coin is selected independently from a set of two denominations with the following values and probabilities:

  • Gold Coin: 100 units (probability 0.2)

  • Silver Coin: 20 units (probability 0.8)

Let \(X\) be the random variable representing the total monetary value of the three coins drawn from the chest. Although \(X\) deterministically maps each outcome (i.e., a specific sequence of three coins) to a numerical total, the probability associated with any total value is determined by the inverse mapping from that value back to the outcomes in the sample space.

  1. What are the possible values that \(X\) can take on with positive probability? This collection of values is called the support of the random variable.

    \[\text{Supp}(X) = \{x \in \mathbb{R} | P(X = x) > 0\} = \{ \quad \}\]

  2. For each value of \(x\) in the support of \(X\), list the specific outcomes (coin sequences) in the sample space that map to \(x\) (i.e., describe the inverse image \(X^{-1}(\{x\})\)). Explain how these outcomes collectively determine the probability \(P(X = x)\).

  3. Calculate the following probability \(P(X = 300)\) by summing the probabilities of all outcomes for which \(X(\omega) = 300\). Hint: Each coin is selected independently.

R Code for Analysis:

# Define coin values and probabilities
gold_value <- 100
silver_value <- 20
p_gold <- 0.2
p_silver <- 0.8

# Calculate all possible outcomes for 3 coins
outcomes <- expand.grid(
  coin1 = c("G", "S"),
  coin2 = c("G", "S"),
  coin3 = c("G", "S")
)

# Calculate total value for each outcome
outcomes$value <- apply(outcomes, 1, function(row) {
  sum(ifelse(row == "G", gold_value, silver_value))
})

# Calculate probability for each outcome
outcomes$prob <- apply(outcomes, 1, function(row) {
  prod(ifelse(row == "G", p_gold, p_silver))
})

# Find support and PMF
pmf <- aggregate(prob ~ value, data = outcomes, sum)
names(pmf) <- c("x", "p_X(x)")

print("Probability Mass Function:")
print(pmf)

# Verify probabilities sum to 1
cat("\nSum of probabilities:", sum(pmf$`p_X(x)`), "\n")

# Calculate P(X = 300)
p_300 <- pmf$`p_X(x)`[pmf$x == 300]
cat("P(X = 300) =", p_300, "\n")

Part 4: Probability Mass Functions

Note

For the rest of the semester, we will not use functional notation \(X(\omega)\) or explicitly refer to the inverse mapping \(X^{-1}(\{x\})\), as these details are cumbersome and unnecessary. However, it is important to understand that a random variable assigns a numerical value to each outcome (the thing that is random) in the sample space.

A probability distribution for a random variable specifies the probabilities associated with all its potential values. When a random variable is discrete, we refer to the probability distribution as a probability mass function (pmf).

Symbolic Representation: \(p_X(x) = P(X = x)\)

The support of a discrete random variable (r.v.) is the set of all possible values that have a strictly positive probability with respect to the probability mass function.

\[\text{Supp}(X) = \{x \in \mathbb{R} | p_X(x) > 0\}\]

For probability mass functions to be valid, they must satisfy the following axioms:

  1. Probabilities must be between 0 and 1 inclusive: \(0 \leq p_X(x) \leq 1\) for all \(x \in \mathbb{R}\)

  2. The probabilities must sum to 1: \(\sum_{x \in \text{Supp}(X)} p_X(x) = 1\)

Question 5: The table below defines a possible probability mass function.

Table 6 PMF Table

x

-3

-2

-1

0

1

2

3

pₓ(x)

k

k

2k

8k

2k

k

k

  1. Find a value \(k\) such that makes it a valid pmf.

  2. Determine the probability that \(X\) takes on non-negative values.

  3. Determine the following probability \(P(\{X = -1\} \cup \{X = -2\} \cup \{X = 1\} \cup \{X = 2\})\).

  4. Given that \(X\) takes on non-negative values, determine the probability that it takes strictly positive values.

Part 5: Joint Probability Mass Functions

The probability of two discrete random variables \(X\) and \(Y\) is defined jointly and called the joint probability mass function.

Symbolic Representation: \(p_{X,Y}(x, y) = P(\{X = x\} \cap \{Y = y\})\)

Question 6: The following is a joint probability mass function for two random variables \(X\) and \(Y\).

Table 7 Joint PMF Table

x, y

1

2

3

4

1

pₓ,ᵧ(1,1) = 1/144

pₓ,ᵧ(1,2) = 1/72

pₓ,ᵧ(1,3) = 1/288

pₓ,ᵧ(1,4) = 41/288

2

pₓ,ᵧ(2,1) = 1/144

pₓ,ᵧ(2,2) = 93/144

pₓ,ᵧ(2,3) = 1/144

pₓ,ᵧ(2,4) = 1/144

3

pₓ,ᵧ(3,1) = 1/144

pₓ,ᵧ(3,2) = 1/72

pₓ,ᵧ(3,3) = 1/288

pₓ,ᵧ(3,4) = 41/288

  1. Determine the marginal distribution for \(X\).

  2. Determine the marginal distribution for \(Y\).

  3. If a random trial is performed, find the probability that \(Y\) takes an even value given that \(X\) takes an odd value.

R Code for Joint PMF Analysis:

# Create joint PMF matrix
joint_pmf <- matrix(c(
  1/144, 1/72, 1/288, 41/288,
  1/144, 93/144, 1/144, 1/144,
  1/144, 1/72, 1/288, 41/288
), nrow = 3, byrow = TRUE)

rownames(joint_pmf) <- 1:3
colnames(joint_pmf) <- 1:4

# Display joint PMF
cat("Joint PMF:\n")
print(round(joint_pmf, 6))

# Calculate marginal distributions
marginal_X <- rowSums(joint_pmf)
marginal_Y <- colSums(joint_pmf)

cat("\nMarginal distribution of X:\n")
for (i in 1:3) {
  cat("P(X =", i, ") =", marginal_X[i], "=",
      fractions::as.fractions(marginal_X[i]), "\n")
}

cat("\nMarginal distribution of Y:\n")
for (j in 1:4) {
  cat("P(Y =", j, ") =", marginal_Y[j], "=",
      fractions::as.fractions(marginal_Y[j]), "\n")
}

# Calculate P(Y even | X odd)
# X odd means X = 1 or X = 3
# Y even means Y = 2 or Y = 4

p_X_odd <- marginal_X[1] + marginal_X[3]
p_Y_even_and_X_odd <- sum(joint_pmf[c(1,3), c(2,4)])
p_Y_even_given_X_odd <- p_Y_even_and_X_odd / p_X_odd

cat("\nP(Y even | X odd) =", p_Y_even_given_X_odd, "=",
    fractions::as.fractions(p_Y_even_given_X_odd), "\n")

Key Takeaways

Summary 📝

  • Independence means \(P(A|B) = P(A)\) and allows the special multiplication rule \(P(A \cap B) = P(A)P(B)\)

  • Mutual independence requires both pairwise and multi-way independence conditions

  • Independent events can occur together; mutually exclusive events cannot

  • A random variable maps outcomes to numbers, providing a framework for quantitative analysis

  • Probability mass functions assign probabilities to discrete values and must sum to 1

  • Marginal distributions are obtained by summing over the other variable in a joint PMF