Summation and Product NotationMath for LLMs

Summation and Product Notation

Mathematical Foundations

Private notes
0/8000

Notes stay private to your browser until account sync is configured.

Summation and Product Notation

Exercises Notebook

Converted from exercises.ipynb for web reading.

Summation and Product Notation - Exercises

This notebook contains 10 progressive exercises for 04-Summation-and-Product-Notation. Each exercise has a learner workspace followed by a complete reference solution. The goal is fluent foundational math for later linear algebra, calculus, probability, and ML sections.

Code cell 2

import numpy as np
import matplotlib.pyplot as plt
import matplotlib as mpl

try:
    import seaborn as sns
    sns.set_theme(style="whitegrid", palette="colorblind")
    HAS_SNS = True
except ImportError:
    plt.style.use("seaborn-v0_8-whitegrid")
    HAS_SNS = False

mpl.rcParams.update({
    "figure.figsize":    (10, 6),
    "figure.dpi":         120,
    "font.size":           13,
    "axes.titlesize":      15,
    "axes.labelsize":      13,
    "xtick.labelsize":     11,
    "ytick.labelsize":     11,
    "legend.fontsize":     11,
    "legend.framealpha":   0.85,
    "lines.linewidth":      2.0,
    "axes.spines.top":     False,
    "axes.spines.right":   False,
    "savefig.bbox":       "tight",
    "savefig.dpi":         150,
})
np.random.seed(42)
print("Plot setup complete.")

Code cell 3

import numpy as np
import numpy.linalg as la
from decimal import Decimal, getcontext
from itertools import product

np.set_printoptions(precision=8, suppress=True)
np.random.seed(42)

def header(title):
    print("\n" + "=" * len(title))
    print(title)
    print("=" * len(title))

def check_true(name, cond):
    ok=bool(cond)
    print(f"{'PASS' if ok else 'FAIL'} - {name}")
    return ok

def check_close(name, got, expected, tol=1e-8):
    ok=np.allclose(got, expected, atol=tol, rtol=tol)
    print(f"{'PASS' if ok else 'FAIL'} - {name}")
    if not ok:
        print('  got     =', got)
        print('  expected=', expected)
    return ok

def powerset(s):
    items=list(s)
    return [set(items[i] for i in range(len(items)) if mask & (1 << i)) for mask in range(1 << len(items))]

print("Chapter 01 helper setup complete.")

Exercise 1: Arithmetic Sum

Verify ∑i=1ni=n(n+1)/2\sum_{i=1}^n i=n(n+1)/2.

Code cell 5

# Your Solution
# Exercise 1 - learner workspace
# Write your solution here, then run the reference solution below to compare.
print("Learner workspace ready for Exercise 1.")

Code cell 6

# Solution
# Exercise 1 - Arithmetic Sum
header("Exercise 1: arithmetic sum")
for n in [5,10,100]: check_close(f"n={n}", sum(range(1,n+1)), n*(n+1)/2)

Exercise 2: Geometric Sum

Verify ∑i=0n−1ri=(1−rn)/(1−r)\sum_{i=0}^{n-1} r^i=(1-r^n)/(1-r).

Code cell 8

# Your Solution
# Exercise 2 - learner workspace
# Write your solution here, then run the reference solution below to compare.
print("Learner workspace ready for Exercise 2.")

Code cell 9

# Solution
# Exercise 2 - Geometric Sum
header("Exercise 2: geometric sum")
r=0.7; n=12
s=sum(r**i for i in range(n)); formula=(1-r**n)/(1-r)
check_close("geometric", s, formula)

Exercise 3: Swap Double Sums

Check ∑i∑jAij=∑j∑iAij\sum_i\sum_j A_{ij}=\sum_j\sum_i A_{ij}.

Code cell 11

# Your Solution
# Exercise 3 - learner workspace
# Write your solution here, then run the reference solution below to compare.
print("Learner workspace ready for Exercise 3.")

Code cell 12

# Solution
# Exercise 3 - Swap Double Sums
header("Exercise 3: double sum")
A=np.arange(12).reshape(3,4)
check_close("swap", sum(sum(row) for row in A), sum(sum(A[:,j]) for j in range(A.shape[1])))

Exercise 4: Matrix Multiplication as Sums

Compute Cij=∑kAikBkjC_{ij}=\sum_k A_{ik}B_{kj} manually.

Code cell 14

# Your Solution
# Exercise 4 - learner workspace
# Write your solution here, then run the reference solution below to compare.
print("Learner workspace ready for Exercise 4.")

Code cell 15

# Solution
# Exercise 4 - Matrix Multiplication as Sums
header("Exercise 4: matmul sum")
A=np.array([[1.,2.],[3.,4.]]); B=np.array([[0.,1.],[2.,3.]])
C=np.zeros((2,2))
for i in range(2):
    for j in range(2):
        C[i,j]=sum(A[i,k]*B[k,j] for k in range(2))
check_close("manual matmul", C, A@B)

Exercise 5: Products in Log Space

Turn a product into a sum of logs.

Code cell 17

# Your Solution
# Exercise 5 - learner workspace
# Write your solution here, then run the reference solution below to compare.
print("Learner workspace ready for Exercise 5.")

Code cell 18

# Solution
# Exercise 5 - Products in Log Space
header("Exercise 5: log product")
p=np.array([0.9,0.8,0.7,0.6])
check_close("prod", np.prod(p), np.exp(np.sum(np.log(p))))

Exercise 6: Loss Averaging

Compute empirical risk as an average of per-example losses.

Code cell 20

# Your Solution
# Exercise 6 - learner workspace
# Write your solution here, then run the reference solution below to compare.
print("Learner workspace ready for Exercise 6.")

Code cell 21

# Solution
# Exercise 6 - Loss Averaging
header("Exercise 6: empirical risk")
y=np.array([1.,0.,1.]); p=np.array([0.8,0.3,0.6])
loss=-(y*np.log(p)+(1-y)*np.log(1-p))
risk=np.mean(loss)
print("losses", loss, "risk", risk)
check_close("risk", risk, np.sum(loss)/len(loss))

Exercise 7: Gradient of a Sum

Differentiate ∑i(wxi−yi)2\sum_i (wx_i-y_i)^2 numerically and analytically.

Code cell 23

# Your Solution
# Exercise 7 - learner workspace
# Write your solution here, then run the reference solution below to compare.
print("Learner workspace ready for Exercise 7.")

Code cell 24

# Solution
# Exercise 7 - Gradient of a Sum
header("Exercise 7: gradient sum")
x=np.array([1.,2.,3.]); y=np.array([2.,4.,5.]); w=1.5
g=2*np.sum((w*x-y)*x)
eps=1e-6
f=lambda a:np.sum((a*x-y)**2)
fd=(f(w+eps)-f(w-eps))/(2*eps)
check_close("gradient", g, fd, tol=1e-6)

Exercise 8: Attention Weighted Sum

Compute oi=∑jaijvjo_i=\sum_j a_{ij}v_j.

Code cell 26

# Your Solution
# Exercise 8 - learner workspace
# Write your solution here, then run the reference solution below to compare.
print("Learner workspace ready for Exercise 8.")

Code cell 27

# Solution
# Exercise 8 - Attention Weighted Sum
header("Exercise 8: attention sum")
A=np.array([[0.7,0.3],[0.2,0.8]]); V=np.array([[1.,0.],[0.,2.]])
O=A@V
manual=np.vstack([sum(A[i,j]*V[j] for j in range(2)) for i in range(2)])
check_close("weighted sum", O, manual)

Exercise 9: Telescoping Sum

Show ∑i=1n(ai−ai−1)=an−a0\sum_{i=1}^n (a_i-a_{i-1})=a_n-a_0.

Code cell 29

# Your Solution
# Exercise 9 - learner workspace
# Write your solution here, then run the reference solution below to compare.
print("Learner workspace ready for Exercise 9.")

Code cell 30

# Solution
# Exercise 9 - Telescoping Sum
header("Exercise 9: telescoping")
a=np.array([1.,1.5,3.,2.,5.])
check_close("telescopes", np.sum(a[1:]-a[:-1]), a[-1]-a[0])

Exercise 10: Batch Product Notation

Compute sequence likelihood as a product of conditional probabilities.

Code cell 32

# Your Solution
# Exercise 10 - learner workspace
# Write your solution here, then run the reference solution below to compare.
print("Learner workspace ready for Exercise 10.")

Code cell 33

# Solution
# Exercise 10 - Batch Product Notation
header("Exercise 10: sequence likelihood")
probs=np.array([0.9,0.8,0.75,0.6])
loglik=np.sum(np.log(probs)); likelihood=np.exp(loglik)
check_close("likelihood", likelihood, np.prod(probs))
PreviousNext