Radon-Nikodym Theorem

"A density is a derivative of one measure with respect to another."

Overview

The Radon-Nikodym theorem explains when densities, likelihood ratios, importance weights, KL divergence, and change-of-measure formulas exist.

Measure theory is the grammar behind rigorous probability. Earlier probability chapters taught how to compute with random variables and distributions. This chapter explains what those objects are when sample spaces are infinite, events are generated by observations, and densities depend on a base measure.

This section uses LaTeX Markdown throughout. Inline mathematics uses $...$, and display mathematics uses `

`. The focus is the foundation needed for ML: expected loss, pushforward distributions, convergence of estimators, likelihood ratios, importance sampling, KL divergence, and support mismatch.

Prerequisites

• Probability Measure Spaces
• KL Divergence
• Sampling Methods
• Evaluation Robustness and Distribution Shift

Companion Notebooks

Learning Objectives

After completing this section, you will be able to:

• Define absolute continuity, singularity, and equivalence of measures
• State the Radon-Nikodym theorem and its assumptions
• Interpret $dP/dQ$ as a generalized density and likelihood ratio
• Use change-of-measure formulas for expectations
• Apply the chain rule for Radon-Nikodym derivatives
• Explain uniqueness up to almost-everywhere equality
• Connect KL divergence to Radon-Nikodym derivatives
• Compute finite-space importance weights
• Diagnose support mismatch in importance sampling and off-policy evaluation
• Bridge density-ratio methods to variational inference and policy learning

Study Flow

1. Read the pages in order and pause after each page to restate the main definition or theorem.
2. Run theory.ipynb when you want to check the formulas numerically.
3. Use exercises.ipynb after the reading path, not before it.
4. Return to this overview page when you need the chapter-level navigation.

Runnable Companions

• Theory notebook
• Exercises notebook