\( \definecolor{colordef}{RGB}{249,49,84} \definecolor{colorprop}{RGB}{18,102,241} \)

Probability

Ever wondered if it will rain tomorrow or if you will win a game? That’s probability! It is a mathematical way to measure how likely an event is to happen.

Algebra of Events

Sample Spaces

Definition Outcome
An outcome is one possible result of a random experiment.
Example
What are all the possible outcomes when you flip a coin?

The outcomes are Heads (H) = and Tails (T) = .

Example
What are the outcomes when you roll a six-sided die?

The outcomes are\(1 = \),\(2 = \),\(3 = \),\(4 = \),\(5 = \)and \(6 = \).

Definition Sample Space
The sample space is the set of all possible outcomes of a random experiment.
Example
What’s the sample space when you flip a coin?

The sample space is \(\{\text{Heads}, \text{Tails}\} = \{\),\(\}\), or just \(\{\text{H}, \text{T}\}\) for short.

Example
What’s the sample space when you roll a six-sided die?

The sample space is \(\{1, 2, 3, 4, 5, 6\} = \{\),,,,,\(\}\).

Events

Once we know all the possible outcomes of an experiment (the sample space), we can focus on specific outcomes we are interested in. An event is a set of outcomes (it may contain one outcome, several outcomes, or even none at all).
Definition Event
An event (often denoted by a capital letter like \(E\)) is a subset of the sample space.
Example
For the experiment of rolling a die, list the outcomes in the event \(E\): “rolling an even number”.

Among the outcomes of the sample space \(\{1, 2, 3, 4, 5, 6\} = \{\),,,,,\(\}\), the event of rolling an even number is\(E = \{2, 4, 6\} = \{\),,\(\}\).

Complementary Events

In probability, it is often useful to consider the outcomes that do not belong to a specific event. This set of “other” outcomes is known as the complementary event. It represents everything in the sample space that is outside the original event. The complement of an event \(E\) is denoted by \(E'\).
Definition Complementary Event
The complementary event of an event \(E\), denoted \(E'\), \(E^c\), or \(\overline{E}\), is the set of all outcomes in the sample space that are not in \(E\).
Example
In the experiment of rolling a fair six-sided die, let \(E\) be the event “rolling an even number”. Find the complementary event, \(E'\).

The sample space is \(\{1, 2, 3, 4, 5, 6\} = \{\),,,,,\(\}\).
The event is \(E = \{2, 4, 6\} = \{\),,\(\}\).
The complementary event \(E'\) contains all outcomes in the sample space that are not in \(E\).
Therefore, \(E' = \{1, 3, 5\} = \{\),,\(\}\). This is the event “rolling an odd number”.

Multi-Step Random Experiments

A multi-step experiment is a random experiment made up of a sequence of two or more simple steps. For example, flipping two coins is a multi-step experiment composed of two actions: flipping the first coin and then flipping the second.
In many multi-step experiments, we can find the total number of possible outcomes by multiplying the number of outcomes at each step. To display all the combined outcomes, we can use tools like lists, tables, or tree diagrams.
Method Representing Sample Spaces for Multi-Step Experiments
When an experiment has more than one step, the sample space can be represented in several ways:
  • by listing all possible ordered outcomes;
  • using a table (best for two-step experiments);
  • using a tree diagram (useful for any number of steps).
Example
For the experiment of tossing two coins, represent the sample space by:
  1. listing all possible outcomes;
  2. using a table;
  3. using a tree diagram.

  1. List:
    Each outcome records the result of coin 1 first, then coin 2:$$\{\textcolor{colordef}{H}\textcolor{colorprop}{H}, \textcolor{colordef}{H}\textcolor{colorprop}{T}, \textcolor{colordef}{T}\textcolor{colorprop}{H}, \textcolor{colordef}{T}\textcolor{colorprop}{T}\}$$
  2. Table:
    \(\begin{aligned} & \textcolor{colorprop}{\text{coin 2}} \\ \textcolor{colordef}{\text{coin 1}} \end{aligned} \) \(\textcolor{colorprop}{H}\) \(\textcolor{colorprop}{T}\)
    \(\textcolor{colordef}{H}\) \(\textcolor{colordef}{H}\textcolor{colorprop}{H}\) \(\textcolor{colordef}{H}\textcolor{colorprop}{T}\)
    \(\textcolor{colordef}{T}\) \(\textcolor{colordef}{T}\textcolor{colorprop}{H}\) \(\textcolor{colordef}{T}\textcolor{colorprop}{T}\)
  3. Tree diagram:

\(E\) or \(F\)

Definition \(E\) or \(F\)
The union of two events \(E\) and \(F\), denoted \(E \cup F\), is the event that occurs if \(E\) occurs, or \(F\) occurs, or both occur. We read this as "E or F". It includes all outcomes that are in at least one of the events.
Example
A standard six-sided die is rolled. Let event \(E\) be "rolling an even number" and event \(F\) be "rolling a number less than 4". Find the event \(E \cup F\).

The events are \(E = \{2, 4, 6\}\) and \(F = \{1, 2, 3\}\).
The union \(E \cup F\) contains all outcomes that appear in either set, without repetition:$$ E \cup F = \{1, 2, 3, 4, 6\}. $$

\(E\) and \(F\)

Definition \(E\) and \(F\)
The intersection of two events \(E\) and \(F\), denoted \(E \cap F\), is the event that occurs if both \(E\) and \(F\) occur simultaneously. We read this as "E and F". It includes all outcomes that are common to both events.
Example
A standard six-sided die is rolled. Let event \(E\) be "rolling an odd number" and event \(F\) be "rolling a number less than 4". Find the event \(E \cap F\).

The events are \(E = \{1, 3, 5\}\) and \(F = \{1, 2, 3\}\).
The intersection \(E \cap F\) contains only the outcomes that are in both sets:$$ E \cap F = \{1, 3\}. $$

Mutually Exclusive

Definition Mutually Exclusive
Two events \(E\) and \(F\) are mutually exclusive (or disjoint) if they have no outcomes in common. This means they cannot occur at the same time. Their intersection is the empty set (\(\emptyset\)):$$E \cap F = \emptyset.$$
Example
When rolling a die, let \(E\) be the event "rolling an odd number" and \(F\) be the event "rolling an even number". Show that \(E\) and \(F\) are mutually exclusive.

The event sets are \(E = \{1, 3, 5\}\) and \(F = \{2, 4, 6\}\).
We find their intersection:$$ E \cap F = \emptyset. $$Since there are no outcomes common to both events, they are mutually exclusive.

Venn Diagram

Definition Set Theory and Probability Vocabulary
Notation Set Vocabulary Probability Vocabulary Venn Diagram
\(U\) Universal set Sample space
\(x\) Element of \(U\) Outcome
\(\emptyset\) Empty set Impossible event
\(E\) Subset of \(U\) Event
\(x \in E\) \(x\) is an element of \(E\) \(x\) is an outcome of \(E\)
\(E'\) Complement of \(E\) in \(U\) Complement of \(E\) in \(U\)
\(E \text{ or } F\) Union of \(E\) and \(F\): \(E \cup F\) \(E\) or \(F\)
\(E \text{ and } F\) Intersection of \(E\) and \(F\): \(E \cap F\) \(E\) and \(F\)
\(E \cap F = \emptyset\) \(E\) and \(F\) are disjoint \(E\) and \(F\) are mutually exclusive

Axioms and Rules of Probability

Axioms of Probability

When you flip a coin, there are two possible outcomes: heads or tails. The chance of getting heads is 1 out of 2. We can write this as a fraction:
Definition Probability
The probability of an event \(E\), written \(P(E)\), is a number that tells us how likely the event is to happen. It is always between \(0\) (impossible) and \(1\) (certain). In other words, for any event \(E\), we have \(0 \leq P(E) \leq 1\).
All of probability is built upon three fundamental rules called axioms. These are statements we accept as true and from which all other rules can be derived. The probability of an event \(E\), denoted \(P(E)\), is a number that quantifies its likelihood.
Definition Probability Axioms
A function \(P\) is a probability measure if it satisfies the following three axioms for any events \(E\) and \(F\) in a sample space \(U\):
  1. Axiom 1 (Non-negativity): The probability of any event is a non-negative number, between 0 and 1 inclusive. $$0 \leqslant P(E) \leqslant 1$$
  2. Axiom 2 (Total Probability): The probability of the entire sample space is 1. $$P(U) = 1$$
  3. Axiom 3 (Additivity for Mutually Exclusive Events): If two events \(E\) and \(F\) are mutually exclusive (\(E \cap F = \emptyset\)), then the probability of their union is the sum of their individual probabilities. $$P(E \cup F) = P(E) + P(F)$$
Visualizing the Axioms with Venn Diagrams
Venn diagrams can help us understand the probability axioms. In this context, the entire area of the universal set \(U\) is considered to have a total probability of 1. The probability of any event \(E\) is represented by the proportion of the total area that the event covers.
  • Axiom 1: \(0 \leqslant P(E) \leqslant 1\)
    The area representing event \(E\) cannot be smaller than nothing (0) and cannot be larger than the entire sample space (1).
  • Axiom 2: \(P(U) = 1\)
    It is certain that some outcome in the sample space occurs. Therefore, the probability of the entire sample space is 1 (or 100\(\pourcent\)).
  • Axiom 3: \(P(E \cup F) = P(E) + P(F)\) for Mutually Exclusive Events
    If two events \(E\) and \(F\) are mutually exclusive, they do not overlap in the Venn diagram. The total area covered by their union (\(E \cup F\)) is simply the sum of their individual areas.
    $$ P(E \cup F) = \textcolor{colordef}{P(E)} + \textcolor{colorprop}{P(F)} $$

Fundamental Probability Rules

If there is a \(40\pourcent\) chance of rain tomorrow, what is the chance that it will not rain?
\(100\pourcent - 40\pourcent = 60\pourcent\)
This calculation is an application of the complement rule. It is a shortcut to find the probability that an event does not happen.
Proposition Complement Rule
For any event \(E\) and its complementary event \(E'\),$$\textcolor{colorprop}{P(E') = 1 - P(E)}$$

  • Algebraic Proof
    By definition, an event \(E\) and its complement \(E'\) are mutually exclusive (\(E \cap E' = \emptyset\)) and their union is the entire sample space (\(E \cup E' = U\)).
    Using Axiom 3: \(P(E \cup E') = P(E) + P(E')\).
    Using Axiom 2: \(P(U) = 1\).
    Since \(E \cup E' = U\), we can equate their probabilities:$$ P(E) + P(E') = P(U) = 1. $$Rearranging the formula gives the complement rule:$$P(E') = 1 - P(E).$$
  • Geometric Proof
    The total area of the sample space, \(\textcolor{olive}{P(U)}\),,is the sum of the area of the event, \textcolor{colordef}{\(P(E)\)}, and the area of its complement, \textcolor{colorprop}{\(P(E')\)}.
    So,$$\textcolor{colordef}{P(E)} + \textcolor{colorprop}{P(E')} = \textcolor{olive}{P(U)}.$$Since \(\textcolor{olive}{P(U) = 1}\) (from Axiom 2), we have:$$\textcolor{colordef}{P(E)} + \textcolor{colorprop}{P(E')} = 1.$$

Proposition Addition Law of Probability
For any two events \(E\) and \(F\),$$\textcolor{colorprop}{P(E \cup F) = P(E) + P(F) - P(E \cap F)}$$This formula holds whether or not \(E\) and \(F\) are mutually exclusive. If they are mutually exclusive, then \(P(E \cap F) = 0\) and the formula reduces to Axiom 3.

The Venn diagram below shows the sample space \(U\) with two intersecting events, \(E\) and \(F\).
If we simply add \(P(E)\) and \(P(F)\), we are double-counting the intersection region, \(P(E \cap F)\). To correct for this, we must subtract the intersection once.
Therefore, the total probability of the union is$$P(E \cup F) = P(E) + P(F) - P(E \cap F).$$

Example
A local high school is holding a talent show. The probability that a randomly selected student participates in singing is \(0.4\), the probability that a student participates in dancing is \(0.3\), and the probability that a student participates in both singing and dancing is \(0.1\). Find the probability that a randomly selected student participates in either singing or dancing.

Let \(S\) be the event "participates in singing" and \(D\) be the event "participates in dancing". We are given:
  • \(P(S) = 0.4\)
  • \(P(D) = 0.3\)
  • \(P(S \cap D) = 0.1\)
The probability that a student participates in singing or dancing is \(P(S \cup D)\). We use the Addition Law of Probability:$$\begin{aligned}P(S \cup D) &= P(S) + P(D) - P(S \cap D) \\ &= 0.4 + 0.3 - 0.1 \\ &= 0.6.\end{aligned}$$So there is a \(0.6\) (or \(60\pourcent\)) chance that a randomly selected student participates in at least one of the two activities.

Equally Likely

Have you ever flipped a fair coin or rolled a fair die? In these experiments, each outcome is just as likely as the others. We call these equally likely outcomes.
Definition Equally Likely
When all outcomes are equally likely, the probability of an event \(E\) is:$$\textcolor{colordef}{\begin{aligned}P(E) &= \frac{\text{number of outcomes in the event}}{\text{number of outcomes in the sample space}}\\ &=\dfrac{\Card{E}}{\Card{U}}\\ \end{aligned}}$$
Example
What’s the probability of rolling an even number with a fair six-sided die?

  • Sample space \(= \{1, 2, 3, 4, 5, 6\}\) (6 outcomes).
  • \(E = \{2, 4, 6\}\) (3 outcomes).
  • $$\begin{aligned}P(E) &= \frac{3}{6} \\ &= \frac{1}{2} \end{aligned}$$
So, there’s a \(\dfrac{1}{2}\) chance (or \(50\pourcent\)) of rolling an even number!

Probability of Independent Events

Independent events are events where knowing that one event has happened does not change the probability that the other event happens. For example, when rolling two fair dice at the same time, the result of the first die does not change the chances for the second die — they are independent events.
Definition Independent Events
If two events, \(A\) and \(B\), are independent, the probability that both events happen (that is, \(P(A\cap B)\) or \(P(A \text{ and } B)\)) is the product of their individual probabilities. This is called the multiplication rule for independent events:$$P(A \text{ and } B) = P(A) \times P(B)$$
Example
An experiment consists of the following two independent actions:
  1. Tossing a fair coin.
  2. Rolling a fair six-sided die.
What is the probability of getting tails and rolling a number greater than 4?

Let \(T\) be the event “getting tails” and \(N\) be the event “rolling a number greater than 4”.
  • The events are independent, so we can use the multiplication rule.
  • The probability of getting tails is \(P(T) = \dfrac{1}{2}\).
  • The outcomes for a number greater than 4 are \(\{5, 6\}\). There are 2 favourable outcomes out of 6 total outcomes. So, \(P(N) = \dfrac{2}{6} = \dfrac{1}{3}\).
  • Now, we multiply the probabilities to find the probability of both events happening:$$\begin{aligned}P(T \text{ and } N) &= P(T) \times P(N) \\ &= \dfrac{1}{2} \times \dfrac{1}{3} \\ &= \dfrac{1}{6}\end{aligned}$$
So, the probability of getting tails and rolling a number greater than 4 is \(\dfrac{1}{6}\).

Method Using a Probability Tree Diagram
  1. Draw branches for each step: Draw branches for the first event (coin toss) and then, from the end of each of those branches, draw the branches for the second event (die roll).
  2. Write probabilities on each branch: The probabilities on the branches from a single point must add up to 1. Because the events are independent, the probabilities on the die-roll branches are the same after “Tails” and after “Heads”.
  3. Multiply along the path: To find the probability of a combined event, multiply the probabilities along the path from start to finish.
    $$\textcolor{colorprop}{P(\text{"Tails" and "Number > 4"})=\frac{1}{2}\times \frac{1}{3}}$$

Experimental Probability

So far, we have calculated theoretical probability. This is what we expect to happen based on logic. For example, we expect a coin to land on heads half the time, so we say \(P(\text{Heads}) = \frac{1}{2}\).
But what if we can’t use logic? What if the outcomes are not equally likely? In these cases, we need to do an experiment to estimate the probability.

Isaac wants to find the probability that a cone he drops will land on its base. The possible outcomes are “base down” or “on its side”.
Because of the cone’s shape, these outcomes are not equally likely. A simple theoretical calculation isn’t feasible; the outcomes aren’t symmetric, so Isaac estimates the probability by experiment. He tosses the cone 50 times (50 trials) and records the results:
  • Base down: 15 times.
  • On its side: 35 times.
Based on his experiment, Isaac can estimate the probability. The cone landed base down 15 times out of 50:$$ P(\text{base down}) \approx \frac{15}{50} = \frac{3}{10} $$This result from an experiment is called the experimental probability or relative frequency.

Definition Experimental Probability (Relative Frequency)
The experimental probability of an event is an estimate found by repeating an experiment many times. It is calculated with the formula:$$ \text{Experimental Probability} = \frac{\text{Number of times an event occurs}}{\text{Total number of trials}} $$The more trials we do, the better our estimate of the true probability will be.