Multi-Agent Systems

"When many learners share an environment, every policy becomes part of someone else's data distribution."

Overview

Multi-agent systems study strategic learning when multiple agents act, adapt, communicate, and optimize in a shared environment.

Game theory is the part of the curriculum that studies adaptive decision makers. It asks what happens when each model, user, attacker, defender, or agent optimizes while anticipating the choices of others.

This section is written in LaTeX Markdown. Inline mathematics uses $...$, and display equations use `

`. The notes emphasize strategy, payoff, best response, equilibrium, exploitability, and adversarial adaptation.

Prerequisites

• Markov Chains
• Bellman Equations
• Nash Equilibria
• Causal Inference

Companion Notebooks

Learning Objectives

After completing this section, you will be able to:

• Define Markov games using states, joint actions, transition kernels, rewards, and discounting
• Compute simple joint-action transitions and agent-specific value functions
• Explain why independent learning creates nonstationary data for every other learner
• Relate Nash policies to equilibrium concepts in stochastic games
• Simulate fictitious play and interpret empirical strategy trajectories
• Compare cooperative, competitive, and mixed-motive multi-agent settings
• Analyze communication, conventions, and credit assignment in team games
• Connect multi-agent learning dynamics to self-play and LLM-agent orchestration
• Use welfare and fairness criteria without confusing them with equilibrium
• Identify when partial observability changes the mathematical model

Study Flow

1. Read the pages in order and pause after each page to restate the main definition or theorem.
2. Run theory.ipynb when you want to check the formulas numerically.
3. Use exercises.ipynb after the reading path, not before it.
4. Return to this overview page when you need the chapter-level navigation.

Runnable Companions

• Theory notebook
• Exercises notebook