3 Lecture 2, January 10, 2024

In this lecture, we went over some basic definitions from the set theory, and using them as the building block for the rest of the course. We started Chapter 2 today, with many definitions.

As for the set operations, $\cup,\cap,A^c,...$, the Venn diagrams help to visual the meaning behind. Here is a good reference HERE.

Definition 3.1 (sample space) A sample space $S$ is a set of distinct outcomes of an experiment with the property that in a single trial of the experiment only one of these outcomes occurs.

Definition 3.2 (Discrete and non-discrete sample space) A sample space $S$ is said to be discrete if it is finite, or ``countably infinite” (i.e.,there is a one-to-one correspondence with the natural numbers). Otherwise a sample space is said to be non-discrete.

Definition 3.3 (Event) An event is a subset of the sample space that can be assigned probability.

Definition 3.4 (Simple and Compound event) Let $S$ be discrete and $A\subset S$ an event. If $A$ is indivisible so it contains only one point, we call it a simple event, otherwise compound event.

Definition 3.5 (Probability distribution) Let $S=\{a_1,a_2,\dots\}$ be discrete. Assign numbers $P(\{a_i\})$ (or short: $P(a_i)$), $i=1,2,\dots$, so that

1.$0\leq P(a_i)\leq 1,\quad i=1,2,\dots$ 2. $\sum_{\text{all }i}P(a_i)=1$.

We then call the set of probabilities $\{P(a_i):i=1,2,\dots\}$ a probability distribution.

Definition 3.6 Let $S=\{a_1,a_2,\dots\}$ discrete. From any prob. distribution $P$ on $S$ we can define a prob. measure on $ {S} = 2^S$ (set of all subsets of $S$) by \[\forall A \subseteq S \qquad P(A)=\sum_{a_i\in A}P(a_i).\]

Definition 3.7 (Equally likely) We say a sample space $S$ with a finite number of outcomes is equally likely if the probability of every individual outcome in $S$ is the same.

Observe that

If $|A|$ denote the number of outcomes in an event $A$. In case of an equally likely sample space, \[ 1=P(S)=\sum_{i=1}^{|S|}P(a_{i})= P(a_i)|S|. \] \[ P(a_i)=\frac{1}{|S|}. \]
Hence, \[P(A) = \sum_{i:\;a_i \in A} P(a_i) = \sum_{i:\;a_i \in A} \frac{1}{|S|} =|A|\cdot \frac{1}{|S|}\]

Conclusion: In a finite, equally likely sample space, the probability of an event $A$ can be computed as \[ P(A) = \sum_{i:\;a_i \in A} P(a_i) = \frac{|A|}{|S|}. \]

3.1 Questions from the class

What is the difference between “countably infinite” v.s. “infinite”?

Ans: A set is countably infinite if its elements can be put in one-to-one correspondence with the set of natural numbers $\mathbb{N}$. Alternatively, you can think that a set is countably infinite if you can count off all elements in the set in such a way that, even though the counting will take forever, you will get to any particular element in a finite amount of time. [A good reference page to read]. If a set is not countable or countably infinite, it is infinite.

Why do we have something such as $2^\mathcal{S}$ in the lecture?

Ans: It is related to something called the power set. The power set consists all the possible subset of a set $\mathcal{S}$. In a subset of $S$, (i.e. $A \subseteq \mathcal{S}$, every element in $\mathcal{S}$ may be either in $A$ or not in $A$. Which means, each element has two possibilities, in $A$ or not in $A$. Hence, the cardinality (i.e. the size) of the power set is $2^\mathcal{S}$.

What did we mean by “order does not matter” and “order matters” during the lecture.

Order does not matter: I said when you write out the element of a set, the order does not matter. For instance, in the rolling a six-sided dice, which side would be faced on the top example, we can write $\mathcal{S} = \{1,2,3,4,5,6\}$, or $\mathcal{S}^\prime=\{6,5,4,3,2,1\}$, and those two sets are essentially equal to each other. To represent a set, the order does not matter, but we tend to write in a way that is intuitive and easy to understand.
Order does matter: In rolling two dices example, the dots show on each of the dice is an ordered pair, denoted by $(x,y)$. Hence, for instance, $(1,2)$ and $(2,1)$ are different. It is problem-dependent so be careful.