Discrete Random Variables

Definition of Random variable

Decide an experiment random with S as sample space. A function X which assigns to every outcomes $s \in S$, a real number X(s) = x is called a random variable.

Example:

When we toss a coin. S the sample space {H, T}. Defining a random variable X as:

• X(H) = 1 (if the result is heads)
• X(T) = 0 (if the result is tails)

Definition: Discrete random variable and probability mass function.

A random variable called X, which is able to take on a countable number of values is a discrete random variable, which probability mass function, also called pmf: p(a) = P(X = a)

Example:

Rolling a six-sided die, and random variable X represents the roll outcome, so pmf is: p(a) = P(X = a) = $\frac{1}{6}$ for x = 1, 2, 3, 4, 5, 6

Non-negativity

$p(x_i) \ge 0$for all $x_i$: The probability of each outcome is always non-negative

Example:

For a coin toss, P(X = 1) and P(X = 0) same = 0.5, both $\ge 0$

Example when 0 otherwise:

Define a random variable X, which represents number of heads when 2 coins are tossed.

1. Sample Space

Possible outcomes for the experiment: $S=\{HH, HT, TH, TT\}$

1. Random Variable X:

• X = 0 (no head, outcome is TT)
• X = 1 (one head, outcome are HT, TH)
• X = 2 (two heads, outcome is HH)

1. Probability Mass Function (pmf)

The pmf of X is:

$$ p(x) = \begin{cases} \frac{1}{4} & \text{if } x = 0, \\ \frac{2}{4} & \text{if } x = 1, \\ \frac{1}{4} & \text{if } x = 2, \\ 0 & \text{otherwise.} \end{cases} $$

Combine with 2. and 3. into the table, we have:

1. What happen when 0 otherwise

pmf assigns a probability of 0 of any value of X that is not in the set {0, 1, 2}

• p(3) = 0 and it is impossible to get 3 heads when tossing two coins
• p(-1) = 0 and negative number are not valid outcomes for the number of heads.

Normalisation

$$ \sum_{i=1}^{\infty} p(x_i) =1 $$

• The sum of probabilities over all possible outcomes must equals 1

Example:

Rolling a dice, $\sum_{x = 1}^6 p(x)= \frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} + \frac{1}{6} = 1$

Heads & Tails: $\sum_{x = 0}^2 p(x)= \frac{1}{4} + \frac{2}{4} + \frac{1}{4} = 1$

Exercise

Suppose X is a discrete random variable: P(X=x) = $\frac{k}{x}$, for x = 1,2,3,4

For some unknown k. What k value makes this a valid probability distribution?

$$ \sum_{i=1}^{4} p(X = x) = 1 \iff \sum_{i=1}^{4} \frac{k}{x} = 1 $$ $$ \sum_{i=1}^{4} \frac{k}{x} = \frac{k}{1} + \frac{k}{2} + \frac{k}{3} + \frac{k}{4} = 1 $$ $$ \iff \frac{12k}{12} + \frac{6k}{12} + \frac{4k}{12} + \frac{3k}{12} = \frac{25k}{12} = 1 \iff k = \frac{12}{25} $$

Definition and Variance

The expected value or an expectation discrete random variable X is: $E[X] = \sum_i x_ip(x_i)$

Example: Rolling a 6-sided dice, all are fair

• Random variable X: Outcome of rolling a die ($X \in \{1,2,3,4,5,6\}$)
• Probability of each outcome: $p(x_i) = \frac{1}{6}$(since the dice is fair)

$$ E[X] = \sum_{i=1}^6 x_ip(x_i) = (1 \cdot \frac{1}{6}) + (2 \cdot \frac{1}{6}) + (3 \cdot \frac{1}{6}) + (4 \cdot \frac{1}{6}) + (5 \cdot \frac{1}{6}) + (6 \cdot \frac{1}{6}) $$ $$ E[X] = \frac{1}{6} \cdot (1 + 2 + 3 + 4 + 5 + 6) = \frac{21}{6} = 3.5 $$

The variance is:

$$ Var(X) = E[(X - E[X])^2] = E[X^2] - (E[X])^2 $$

Same Example: follow up the above example on 6-sided dice

Calculate $E[X^2]$: $\frac{1}{6} \cdot (1^2 + 2^2 + 3^2 + 4^2 + 5^2 + 6^2) = \frac{91}{6} \approx 15.17 $

Calculate Var(X):

$$ Var(X) = E[X^2] - (E[X])^2 = \frac{91}{6} - (\frac{21}{6})^2 = \frac{105}{36} \approx 2.92 $$

Exercise

Calculate the following probability distribution variance

Way 1:

Use normalisation, we have

$$ E[X] = \sum_{i=0}^2 p(X = x) = (0 \cdot 0.4) + (1 \cdot 0.4) + (2 \cdot 0.2) = 0.8 $$

Calculate $E[X^2]$: $(0^2 \cdot 0.4) + (1^2 \cdot 0.4) + (2^2 \cdot 0.2) = 1.2$

Calculate Var(X):

$$ Var(X) = E[X^2] - (E[X])^2 = 1.2 - (0.8)^2 = 0.56 $$

Way 2:

Calculate X - E[X]: For each $x_i$, subtract E[X] = 0.8

• For x = 0: X - E[X] = 0 - 0.8 = -0.8
• For x = 1: X - E[X] = 1 - 0.8 = 0.2
• For x = 2: X - E[X] = 2 - 0.8 = 1.2

Calculate $(X - E[X])^2$

$$ 0.8^2 = 0.64, 0.2^2 = 0.04, 1.2^2 = 1.44 $$

Calculate each square deviation by its probability $P(X = x_i)$, use the probabilities on the table:

• For x = 0, P(X = 0) = 0.4, so $0.4 \cdot 0.64 = 0.256$
• For x = 0, P(X = 0) = 0.4, so $0.4 \cdot 0.04 = 0.016$
• For x = 0, P(X = 0) = 0.2, so $0.2 \cdot 1.44 = 0.288$

Calculate Var(X):

$$ Var(X) = E[(X - E[X])^2] = 0.256 + 0.016 + 0.288 = 0.56 $$