Summation

The mean of a set of $n$ numbers $x$:

$$ \bar{x} = \frac{x_1 + x_2 + ... + x_n}{n} $$

Example:

Having a set of numbers: $x = \{2, 5, 7, 13, 23, 32\}$

• $n = 6$ (number of elements)
• Sum of numbers: 2 + 5 + 7 + 13 + 23 + 32 ‎ = 82
• Mean: $\bar{x} = \frac{82}{6} = \sim 13.667$

To guess a response of $y_i$, use Linear regression based on the predictors $x_i$:

$$ y_i = \beta_0 + \beta_1 x_{1i} + \beta_2 x_{2i} + ... + \beta_n x_{ni} + \epsilon_i $$

Example:

Predicting a a student’s test score ($y_i$) based on hours studied ($x_1$) and hours slept ($x_2)$

• $y_i$= 50 + 5$x_1$+ 2$x_2$+ $\epsilon_i$

Explain the terms:

$y_i$: The dependent variable which you are trying to predict, this represent student’s test score

$\beta_0 = 50$:

• The intercept of the regression equation, and the baseline value of $y_i$when all predictors ($x_1$and $x_2$) are 0.
• In this case, it means if a student does not study ($x_1 = 0$) and does not sleep ($x_2 = 0$), their default predicted test score is 50.

$\beta_1 = 5$:

• The coefficient for $x_1$(hours studied)
• Representing the change in $y_1$(test score) for each additional hour of studying to assume $x_2$(hours slept) remains constant
• In this case, every additional hour studied, increasing 5 points at the test score at $y_i$.

$\beta_2 = 2$:

• The coefficient for $x_2$(hours slept)
• Representing the change in $y_i$(test score) for each additional hour of sleep to assume $x_1$ (hours slept) remains constant.
• In this situation, with every additional hour of sleep, the test score rises by 2 points at $y_i$

$\epsilon_i$:

• The error term
• This makes random noise or unexplained variation in the data that the predictors $x_1$and $x_2$cannot capture.
• In reality, $\epsilon_i$makes sure that the model is always not perfect in order to reflect the variability data in the real-world.

Calculate for a student who studied 4 hours and slept 6 hours (when rule out $\epsilon_i$: $y_i$= 50 + 5$x_1$+ 2$x_2$+ $\epsilon_i$, and replace $x_1$and $x_2$with 4 and 6, respectively:

$$ y_i = 50+5(4)+2(6)=50+20+12=82 $$

Overall, this equation shows the relationship between test scores and two key factors: hours studied and hours slept. The $y_i$is addictive and constant (e.g adding extra hour of studying increase 5 points to the test score). The intercept (50) gives the baseline the prediction alongside the coefficients (5 and 2) quantify the influence of each predictor.

The (angular) distance $\theta$between two vectors ($x_1, x_2, ..., x_n$) and ($y_1, y_2, ..., y_n$)

(This measure of distance is used in word embeddings

(1)

$$ cos(\theta) = \frac{x_1y_1+x_2y_2+...+x_ny_n}{\sqrt{x_1^2+x_2^2+...+x_n^2}\sqrt{y_1^2+y_2^2+...+y_n^2}} $$

(2)

$$ \sum_{k=1}^{n} a_k = a_1 + a_2 + ... + a_n $$

Example $cos(\theta)$at (1):

Two vectors $x = (1,2,3)$and $y = (4,5,6)$

• Dot product: $1 \cdot 4 + 2 \cdot 5 + 3 \cdot 6 = 4 + 10 + 18 = 32$
• Magnitudes: $||x|| = \sqrt{1^2 + 2^2 + 3^2} = \sqrt{14} $, $||y|| = \sqrt{4^2 + 5^2 + 6^2} = \sqrt{77} $
• Cosine similarity:

$$ cos(\theta) = \frac{32}{\sqrt{14} \cdot \sqrt{77}} \approx 0.9746 $$

Example $\sum$ at (2):

Supposing provide $a_k = k^2$ and sum k start at 1 (can be change to other N number) and ends at 5

Apart from $a_k = k^2$, we can provide other else like linear growth $a_k = k$ or $a_k = k^3$ or $a_k = 2^k$

$$ \sum_{k=1}^{5} k^2 = 1^2 + 2^2 + 3^2 + 4^2 + 5^2 = 1 + 4 + 9 + 16 + 25 = 55 $$

Other examples to show

$$ \sum_{k=3}^{6} k = 3 + 4 + 5 + 6 = 18 $$ $$ \sum_{k=1}^{4} k^3 = 1^3 + 2^3 + 3^3 + 4^3 ‎= 100 $$ $$ \sum_{k=0}^{5} 2^k = 2^0 + 2^1 + 2^2 + 2^3 + 2^4 + 2^5 = 63 $$ $$ \sum_{k=2}^{3} \frac{1}{k} = \frac{1}{2} + \frac{1}{3} = \frac{3}{6} + \frac{2}{6} = \frac{5}{6} $$

Write $log(2) + log(3) + ... + log(n)$ in sigma notation $\sum_{k=2}^{n} log(k)$

Rules of summation

$$ \begin{aligned} &1. \ \sum_{i=m}^{n} ca_i = c \sum_{i=m}^{n} a_i, \ \text{and variable c is a constant} \\ &2. \ \sum_{i=m}^{n} (a_i + b_i) = \sum_{i=m}^{n} a_i + \sum_{i=m}^{n} b_i \\ &3. \ \sum_{i=m}^{n} (a_i - b_i) = \sum_{i=m}^{n} a_i - \sum_{i=m}^{n} b_i \\ &4. \ \text{But } \sum_{i=m}^{n} (a_i \cdot b_i) \neq \left( \sum_{i=m}^{n} a_i \right) \cdot \left( \sum_{i=m}^{n} b_i \right) \\ &5. \ \sum_{i=m}^{n} \frac{a_i}{b_i} \neq \frac{\sum_{i=m}^{n} a_i}{\sum_{i=m}^{n} b_i} \end{aligned} $$

Example:

Property 1: Let $a_i = i, c = 2, m = 1$, and $n = 3$

$$ \sum_{i=1}^3 2i = 2 \sum_{i=1}^3 i = 2 (1 + 2 + 3) = 2\cdot6 = 12 $$

From Property 2 to 4: use same topic: Let $a_i = i, b = 2i, m = 1$, and $n = 3$

Property 2: : Let $a_i = i, b = 2i, m = 1$, and $n = 3$

$$ \sum_{i=1}^{3} (i + 2i) = \sum_{i=1}^{3} i + \sum_{i=1}^{3} 2i $$

Left side:

$$ \sum_{i=1}^{3} (i + 2i) = (1 + 2 \cdot 1) + (2 + 2 \cdot 2) + (3 + 2 \cdot 3) = 18 $$

equals to

Right side:

$$ \sum_{i=1}^{3} i + \sum_{i=1}^{3} 2i = (1 + 2 + 3) + (2 \cdot 1 + 2 \cdot 2 + 2 \cdot 3) = 6 + 12 = 18 $$

Property 3: Let $a_i = i, b = 2i, m = 1$, and $n = 3$

$$ \sum_{i=1}^{3} (i - 2i) = \sum_{i=1}^{3} i - \sum_{i=1}^{3} 2i $$

Left side:

$$ \sum_{i=1}^{3} (i - 2i) = (1 - 2 \cdot 1) + (2 - 2 \cdot 2) + (3 - 2 \cdot 3) = -6 $$

equals to

Right side:

$$ \sum_{i=1}^{3} i - \sum_{i=1}^{3} 2i = (i + i + i) - (2i + 2i + 2i) $$ $$ = (1+2+3) - (2 \cdot 1 + 2 \cdot 2 + 2 \cdot 3) = -6 $$

Property 4: Let $a_i = i, b = 2i, m = 1$, and $n = 3$

$$ \sum_{i=m}^{n} (a_i \cdot b_i) \neq \left( \sum_{i=m}^{n} a_i \right) \cdot \left( \sum_{i=m}^{n} b_i \right) $$

Left side:

$$ \sum_{i=1}^{3} (i \cdot 2i) = (1 \cdot 2 \cdot 1) + (2 \cdot 2 \cdot 2) + (3 \cdot 2 \cdot 3) = 28 $$

Not equals to

Right side:

$$ \left( \sum_{i=1}^{3} i \right) \cdot \left( \sum_{i=1}^{3} 2i \right) = (1 + 2 + 3) \cdot (2 \cdot 1 + 2 \cdot 2 + 2 \cdot 3) = 6 + 12 = 18 $$

Property 5: Let $a_i = i, b = 2i, m = 1$, and $n = 3$

$$ \sum_{i=m}^{n} \frac{a_i}{b_i} \neq \frac{\sum_{i=m}^{n} a_i}{\sum_{i=m}^{n} b_i} $$

Left side:

$$ \sum_{i=1}^{3} \frac{i}{2i} = \frac{1}{2 \cdot 1} + \frac{2}{2 \cdot 2} + \frac{3}{2 \cdot 3} = \frac{1}{2} + \frac{2}{4} + \frac{3}{6} = 3/2 $$

not equals to

Right side:

$$ \frac{\sum_{i=1}^{3} i}{\sum_{i=1}^{3} 2i} = \frac{1 + 2 + 3}{21 + 22 + 23} = \frac{6}{66} = \frac{1}{11} $$

Definition: Linear operator
An operator L is linear if for all functions f and g, and every scaler $c \in \mathbb{R}$
$$ L(cf) = cL(f) $$ $$ L(f+g) = L(f) + L(g) $$

Example

Let $L(f) = \int f(x)dx$

c is a constant number

$$ L(cf) = \int c f(x)dx = c\int f(x)dx = cL(f) $$ $$ L(f+g) = \int (f(x)+g(x))dx = \int f(x)dx + \int g(x)dx = L(f) + L(g) $$

As a result, the integration is a linear operator.

Series Formalas

$$ \begin{align*} &1. \ \sum_{i=1}^{n} 1 = \overbrace{1 + 1 + \dots + 1}^{n \text{ times}} = n \\ &2. \ \sum_{j=0}^{n} ar^j = a + ar + ar^2 + \dots + ar^n = a \frac{1-r^{n+1}}{1-r} \ \text{is the geometric sum.} \\ &3. \ \sum_{i=1}^{n} i = 1 + 2 + 3 + \dots + n = \frac{n(n+1)}{2} \\ &4. \ \sum_{i=1}^{n} i^2 = \frac{n^3}{3} + \frac{n^2}{2} + \frac{n}{6} = \frac{n}{6}(2n^2 + 3n + 1) = \frac{n}{6}(2n+1)(n+1) \end{align*} $$

Example

1. Sum of constants ($\sum_{i=1}^{n} 1$), if n = 4, simply adds 1 ’n’ times

$$ \sum_{i=1}^{4} i = 1 + 1 + 1 + 1 = 4 $$

1. Geometric sum ($\sum_{j=0}^{n} ar^j$), if $a = 2, r = 3, n = 3$, each term is multiplied by a constant ratio r$\sum_{j=0}^{3} 2 \cdot (3^j) = (2 \cdot (3^1)) + (2 \cdot (3^2)) + (2 \cdot (3^3)) = 78$
2. Sum of Natural Numbers $\sum_{i=1}^{n} i$, if n = 4, adds consecutive integers from 1 to n

$$ \sum_{i=1}^{4} i = 1 + 2 + 3 + 4 = 10 $$

1. Sum of Squares $\sum_{i=1}^{n} i^2$, if n = 4, add squared of consecutive integers

$$ \sum_{i=1}^{4} i^2 = 1^2 + 2^2 + 3^2 + 4^2 = 1 + 4 + 9 + 16 = 30 $$

Another example

$$ \sum_{i=3}^{10} (i+2)^2 $$ $$ (i+2)^2= i^2 + 4i + 4 $$ $$ \sum_{i=3}^{10} i^2 + 4i + 4 = \overbrace{(i^2 + 4i + 4) + (i^2 + 4i + 4) + \dots + (i^2 + 4i + 4)}^{7 \space \text{times}} = 620 $$

More example

$$ S = \sum_{i=1}^w i = 1 + 2 + \dots + (w -1) + w $$

Also

$$ \space S = w + (i -1) + \dots + 2 + 1 $$

Combine that, we have

$$ 2S = \overbrace{(w+1) + (w-1 + 2) + \dots + (2 + w-1) + (1+w)}^{n \space \text{times}} $$ $$ 2S = n(n+1) \leftrightarrow S = \frac{n(n+1)}{2} $$

Multiple Summation

$$ \sum_{i, j=1}^{3} a_i \cdot b_j $$ $$ a_1b_1 + a_1b_2 + a_1b_3 + a_2b_1 + a_2b_2 + a_2b_3 + a_3b_1 + a_3b_2 + a_3b_3 $$ $$ \sum_{i=1}^3 \sum_{j=1}^3 a_j \cdot b_j $$

Definitions
– Associativity:$$ \sum_{j \in J} \sum_{k \in K} a_{j,k} = \sum_{k \in K} \sum_{j \in J} a{j,k} $$ – Distributivity:$$ \sum_{j \in J, k \in K} a_j b_k = \left( \sum_{j \in J} a_j \right) \left( \sum_{k \in K} b_k \right) $$

Example

Associativity

Let’s make a table values j belongs to column, k belongs to row

How to verify on j and k

1. At row-first summation ($\sum_{j \in J} \sum_{k \in K} a_{j, k}$)

• Calculate the row sums: 2 + 4 + 6 = 12, 3 + 5 + 7 = 15, 1 + 8 + 9 = 18
• Sum: 12 + 15 + 18 = 45

1. At column-first summation ($\sum_{k \in K} \sum_{j \in J} a_{j, k}$)

• Compute the column sums: 2 + 3 + 1 = 6, 4 + 5 + 8 = 17, 6 + 7 + 9 = 22
• Sum: 6 + 17 + 22 = 45

Distributivity

Given: $\sum_{j \in J, k \in K} a_jb_k = (\sum_{j \in J} a_j)(\sum_{k \in K} b_k)$

Here is the given values:

• J = {1,2}, K = {1,2}
• a₁ = 2, a₂ = 3
• b₁ = 4, b₂ = 5

Left Side: $\sum_{j \in J, k \in K} a_jb_k$

1. List all terms: $a_1b_1 + a_1b_2 + a_2b_1 + a_2b_2$
2. Substitute values: $(2 \cdot 4) + (2 \cdot 5) + (3 \cdot 4) + (3 \cdot 5) = 8 + 10 + 12 + 15 = 45$

Right Side: $(\sum_{j \in J} a_j)(\sum_{k \in K} b_k)$

1. Calculate $\sum_{j \in J} a_j$:

$$ a_1 + a_2 = 2 + 3 = 5 $$

1. Calculate $\sum_{k \in K} b_k$:

$$ b_1 + b_2 = 4 + 5 = 9 $$

1. Multiply the sums:

$$ 5 \times 9 = 45 $$

From Left and Right side, as a result:

$$ \sum_{j \in J, k \in K} a_jb_k = (\sum_{j \in J} a_j)(\sum_{k \in K} b_k) $$ $$ \text{Left side} \space 45 = 45 \space \text{Right side} $$