Lesson overview | Previous part | Next part

Manifolds: Part 2: Formal Definitions

2. Formal Definitions

Formal Definitions develops the part of manifolds specified by the approved Chapter 25 table of contents. The treatment is geometry-first and AI-facing.

2.1 Topological manifold

Topological manifold belongs to the canonical scope of Manifolds. The goal is to make curved-space reasoning concrete enough for ML practice without turning the section into a pure topology course.

Working scope for this subsection: smooth manifolds, charts, atlases, tangent spaces, differentials, tangent bundles, embedded submanifolds, and ML manifold intuition. The recurring pattern is localize, linearize, measure, move, and return to the manifold.

$$\varphi_\beta\circ\varphi_\alpha^{-1}:\varphi_\alpha(U_\alpha\cap U_\beta)\to\varphi_\beta(U_\alpha\cap U_\beta).$$

Operational definition.

A chart turns a neighborhood of a curved space into coordinates in Euclidean space; an atlas is a compatible collection of such charts.

Worked reading.

On the unit circle, angle $t$ gives local coordinates except where a single global angle chart breaks. Multiple patches avoid artificial singularities.

Three examples of topological manifold:

1. Latitude-longitude patches on a sphere.
2. Angle coordinates on a circle.
3. Local PCA coordinates on a data cloud.

Two non-examples clarify the boundary:

1. One global coordinate map with a seam treated as smooth.
2. A point cloud with no neighborhood structure.

Proof or verification habit for topological manifold:

The proof habit is to check transition maps between overlapping charts, not only individual parameterizations.

global object      -> curved manifold or constraint set
local object       -> chart, tangent space, or coordinate patch
linear operation   -> derivative, gradient, velocity, Hessian approximation
geometric measure  -> metric, length, distance, curvature
algorithmic move   -> tangent step followed by geodesic or retraction

In AI systems, topological manifold matters because learned representations and constrained parameter spaces are rarely globally flat. A local linear approximation may be useful, but it must be attached to the point where it is valid.

Local coordinates are the mathematical version of local representation learning: simple coordinates may exist nearby even when no global linear model exists.

Mini derivation lens.

1. Choose a point $p$ on the manifold $M$ and name the local representation used near $p$.
2. Move the question into a chart, tangent space, or embedded constraint where first-order calculus is available.
3. Compute the local object: derivative, tangent projection, metric-weighted gradient, path velocity, or retraction step.
4. Translate the result back into coordinate-free language so the answer is not tied to one chart by accident.
5. Check the invariant: the point remains on $M$, the direction remains in $T_pM$, or the distance/gradient uses the stated metric.

Implementation lens.

A practical ML implementation should store both the ambient array representation and the geometric contract attached to it. For example, a normalized embedding is not just a vector; it is a point on a sphere. An orthogonal weight matrix is not just a matrix; it is a point on a Stiefel-type constraint. A covariance matrix is not just a symmetric array; it must stay positive definite.

The clean computational pattern is: encode the state, compute an ambient derivative if needed, convert it into a tangent or metric-aware object, take a small local step, and then return to the manifold with a geodesic formula or retraction. This is the same pattern used in the companion notebooks, just scaled down to visible two- and three-dimensional examples.

The important warning is that coordinate code can pass shape checks while still violating geometry. Differential geometry adds checks that are semantic: tangentness, smooth compatibility, metric choice, path validity, and constraint preservation.

Practical checklist:

• State the manifold and whether it is abstract, embedded, or quotient-like.
• State the local coordinates or tangent representation being used.
• Separate ambient vectors from tangent vectors.
• Name the metric before computing distances, angles, or gradients.
• Use geodesics or retractions when moving on the manifold.
• For ML claims, identify whether geometry is data geometry, parameter geometry, or statistical geometry.

Local diagnostic: Name the patch, coordinates, and transition map.

The companion notebook uses low-dimensional synthetic examples: circles, spheres, tangent projections, spherical interpolation, SPD matrices, and orthogonality constraints. These examples keep geometry visible while preserving the same update logic used in higher-dimensional ML systems.

A useful learning move is to compute everything first on a sphere. The sphere has visible curvature, simple tangent spaces, closed-form geodesics, and practical retractions. Once those are clear, Stiefel, Grassmann, SPD, and information-geometric examples become less mysterious.

For implementation, the main discipline is to avoid leaving the manifold silently. If a gradient step violates a constraint, either project the gradient into the tangent space before stepping or use a method whose update is intrinsic by design.

The final question for this subsection is whether a Euclidean formula is being used as an approximation, a coordinate expression, or a mistaken replacement for geometry. Differential geometry is the habit of telling those cases apart.

2.2 Smooth atlas and smooth compatibility

Smooth atlas and smooth compatibility belongs to the canonical scope of Manifolds. The goal is to make curved-space reasoning concrete enough for ML practice without turning the section into a pure topology course.

Working scope for this subsection: smooth manifolds, charts, atlases, tangent spaces, differentials, tangent bundles, embedded submanifolds, and ML manifold intuition. The recurring pattern is localize, linearize, measure, move, and return to the manifold.

$$T_pM=\{\dot{\gamma}(0):\gamma(0)=p,\ \gamma \text{ smooth curve in }M\}.$$

Operational definition.

A chart turns a neighborhood of a curved space into coordinates in Euclidean space; an atlas is a compatible collection of such charts.

Worked reading.

On the unit circle, angle $t$ gives local coordinates except where a single global angle chart breaks. Multiple patches avoid artificial singularities.

Three examples of smooth atlas and smooth compatibility:

1. Latitude-longitude patches on a sphere.
2. Angle coordinates on a circle.
3. Local PCA coordinates on a data cloud.

Two non-examples clarify the boundary:

1. One global coordinate map with a seam treated as smooth.
2. A point cloud with no neighborhood structure.

Proof or verification habit for smooth atlas and smooth compatibility:

The proof habit is to check transition maps between overlapping charts, not only individual parameterizations.

global object      -> curved manifold or constraint set
local object       -> chart, tangent space, or coordinate patch
linear operation   -> derivative, gradient, velocity, Hessian approximation
geometric measure  -> metric, length, distance, curvature
algorithmic move   -> tangent step followed by geodesic or retraction

In AI systems, smooth atlas and smooth compatibility matters because learned representations and constrained parameter spaces are rarely globally flat. A local linear approximation may be useful, but it must be attached to the point where it is valid.

Local coordinates are the mathematical version of local representation learning: simple coordinates may exist nearby even when no global linear model exists.

Mini derivation lens.

1. Choose a point $p$ on the manifold $M$ and name the local representation used near $p$.
2. Move the question into a chart, tangent space, or embedded constraint where first-order calculus is available.
3. Compute the local object: derivative, tangent projection, metric-weighted gradient, path velocity, or retraction step.
4. Translate the result back into coordinate-free language so the answer is not tied to one chart by accident.
5. Check the invariant: the point remains on $M$, the direction remains in $T_pM$, or the distance/gradient uses the stated metric.

Implementation lens.

A practical ML implementation should store both the ambient array representation and the geometric contract attached to it. For example, a normalized embedding is not just a vector; it is a point on a sphere. An orthogonal weight matrix is not just a matrix; it is a point on a Stiefel-type constraint. A covariance matrix is not just a symmetric array; it must stay positive definite.

The clean computational pattern is: encode the state, compute an ambient derivative if needed, convert it into a tangent or metric-aware object, take a small local step, and then return to the manifold with a geodesic formula or retraction. This is the same pattern used in the companion notebooks, just scaled down to visible two- and three-dimensional examples.

The important warning is that coordinate code can pass shape checks while still violating geometry. Differential geometry adds checks that are semantic: tangentness, smooth compatibility, metric choice, path validity, and constraint preservation.

Practical checklist:

• State the manifold and whether it is abstract, embedded, or quotient-like.
• State the local coordinates or tangent representation being used.
• Separate ambient vectors from tangent vectors.
• Name the metric before computing distances, angles, or gradients.
• Use geodesics or retractions when moving on the manifold.
• For ML claims, identify whether geometry is data geometry, parameter geometry, or statistical geometry.

Local diagnostic: Name the patch, coordinates, and transition map.

The companion notebook uses low-dimensional synthetic examples: circles, spheres, tangent projections, spherical interpolation, SPD matrices, and orthogonality constraints. These examples keep geometry visible while preserving the same update logic used in higher-dimensional ML systems.

A useful learning move is to compute everything first on a sphere. The sphere has visible curvature, simple tangent spaces, closed-form geodesics, and practical retractions. Once those are clear, Stiefel, Grassmann, SPD, and information-geometric examples become less mysterious.

For implementation, the main discipline is to avoid leaving the manifold silently. If a gradient step violates a constraint, either project the gradient into the tangent space before stepping or use a method whose update is intrinsic by design.

The final question for this subsection is whether a Euclidean formula is being used as an approximation, a coordinate expression, or a mistaken replacement for geometry. Differential geometry is the habit of telling those cases apart.

2.3 Smooth manifold $M$

Smooth manifold $M$ belongs to the canonical scope of Manifolds. The goal is to make curved-space reasoning concrete enough for ML practice without turning the section into a pure topology course.

Working scope for this subsection: smooth manifolds, charts, atlases, tangent spaces, differentials, tangent bundles, embedded submanifolds, and ML manifold intuition. The recurring pattern is localize, linearize, measure, move, and return to the manifold.

$$dF_p(\mathbf{v})=\frac{d}{dt}\bigg|_{t=0}F(\gamma(t)),\qquad \dot{\gamma}(0)=\mathbf{v}.$$

Operational definition.

A chart turns a neighborhood of a curved space into coordinates in Euclidean space; an atlas is a compatible collection of such charts.

Worked reading.

On the unit circle, angle $t$ gives local coordinates except where a single global angle chart breaks. Multiple patches avoid artificial singularities.

Three examples of smooth manifold $m$:

1. Latitude-longitude patches on a sphere.
2. Angle coordinates on a circle.
3. Local PCA coordinates on a data cloud.

Two non-examples clarify the boundary:

1. One global coordinate map with a seam treated as smooth.
2. A point cloud with no neighborhood structure.

Proof or verification habit for smooth manifold $m$:

The proof habit is to check transition maps between overlapping charts, not only individual parameterizations.

global object      -> curved manifold or constraint set
local object       -> chart, tangent space, or coordinate patch
linear operation   -> derivative, gradient, velocity, Hessian approximation
geometric measure  -> metric, length, distance, curvature
algorithmic move   -> tangent step followed by geodesic or retraction

In AI systems, smooth manifold $m$ matters because learned representations and constrained parameter spaces are rarely globally flat. A local linear approximation may be useful, but it must be attached to the point where it is valid.

Local coordinates are the mathematical version of local representation learning: simple coordinates may exist nearby even when no global linear model exists.

Mini derivation lens.

1. Choose a point $p$ on the manifold $M$ and name the local representation used near $p$.
2. Move the question into a chart, tangent space, or embedded constraint where first-order calculus is available.
3. Compute the local object: derivative, tangent projection, metric-weighted gradient, path velocity, or retraction step.
4. Translate the result back into coordinate-free language so the answer is not tied to one chart by accident.
5. Check the invariant: the point remains on $M$, the direction remains in $T_pM$, or the distance/gradient uses the stated metric.

Implementation lens.

A practical ML implementation should store both the ambient array representation and the geometric contract attached to it. For example, a normalized embedding is not just a vector; it is a point on a sphere. An orthogonal weight matrix is not just a matrix; it is a point on a Stiefel-type constraint. A covariance matrix is not just a symmetric array; it must stay positive definite.

The clean computational pattern is: encode the state, compute an ambient derivative if needed, convert it into a tangent or metric-aware object, take a small local step, and then return to the manifold with a geodesic formula or retraction. This is the same pattern used in the companion notebooks, just scaled down to visible two- and three-dimensional examples.

The important warning is that coordinate code can pass shape checks while still violating geometry. Differential geometry adds checks that are semantic: tangentness, smooth compatibility, metric choice, path validity, and constraint preservation.

Practical checklist:

• State the manifold and whether it is abstract, embedded, or quotient-like.
• State the local coordinates or tangent representation being used.
• Separate ambient vectors from tangent vectors.
• Name the metric before computing distances, angles, or gradients.
• Use geodesics or retractions when moving on the manifold.
• For ML claims, identify whether geometry is data geometry, parameter geometry, or statistical geometry.

Local diagnostic: Name the patch, coordinates, and transition map.

The companion notebook uses low-dimensional synthetic examples: circles, spheres, tangent projections, spherical interpolation, SPD matrices, and orthogonality constraints. These examples keep geometry visible while preserving the same update logic used in higher-dimensional ML systems.

A useful learning move is to compute everything first on a sphere. The sphere has visible curvature, simple tangent spaces, closed-form geodesics, and practical retractions. Once those are clear, Stiefel, Grassmann, SPD, and information-geometric examples become less mysterious.

For implementation, the main discipline is to avoid leaving the manifold silently. If a gradient step violates a constraint, either project the gradient into the tangent space before stepping or use a method whose update is intrinsic by design.

The final question for this subsection is whether a Euclidean formula is being used as an approximation, a coordinate expression, or a mistaken replacement for geometry. Differential geometry is the habit of telling those cases apart.

2.4 Smooth maps between manifolds

Smooth maps between manifolds belongs to the canonical scope of Manifolds. The goal is to make curved-space reasoning concrete enough for ML practice without turning the section into a pure topology course.

Working scope for this subsection: smooth manifolds, charts, atlases, tangent spaces, differentials, tangent bundles, embedded submanifolds, and ML manifold intuition. The recurring pattern is localize, linearize, measure, move, and return to the manifold.

$$\varphi:U\subseteq M\to \varphi(U)\subseteq \mathbb{R}^d.$$

Operational definition.

Smooth maps between manifolds belongs to the canonical scope of Manifolds: smooth manifolds, charts, atlases, tangent spaces, differentials, tangent bundles, embedded submanifolds, and ML manifold intuition.

Worked reading.

Start from a concrete embedded example, compute the local tangent or metric object, then translate back to intrinsic notation.

Three examples of smooth maps between manifolds:

1. Sphere geometry.
2. Embedding-space local coordinates.
3. Matrix-manifold parameter constraints.

Two non-examples clarify the boundary:

1. A flat Euclidean approximation used globally.
2. A geometric claim made without metric or tangent space.

Proof or verification habit for smooth maps between manifolds:

The proof habit is to compute locally and verify coordinate-independent meaning.

global object      -> curved manifold or constraint set
local object       -> chart, tangent space, or coordinate patch
linear operation   -> derivative, gradient, velocity, Hessian approximation
geometric measure  -> metric, length, distance, curvature
algorithmic move   -> tangent step followed by geodesic or retraction

In AI systems, smooth maps between manifolds matters because learned representations and constrained parameter spaces are rarely globally flat. A local linear approximation may be useful, but it must be attached to the point where it is valid.

The AI relevance is that model spaces are often curved even when implemented as arrays.

Mini derivation lens.

1. Choose a point $p$ on the manifold $M$ and name the local representation used near $p$.
2. Move the question into a chart, tangent space, or embedded constraint where first-order calculus is available.
3. Compute the local object: derivative, tangent projection, metric-weighted gradient, path velocity, or retraction step.
4. Translate the result back into coordinate-free language so the answer is not tied to one chart by accident.
5. Check the invariant: the point remains on $M$, the direction remains in $T_pM$, or the distance/gradient uses the stated metric.

Implementation lens.

A practical ML implementation should store both the ambient array representation and the geometric contract attached to it. For example, a normalized embedding is not just a vector; it is a point on a sphere. An orthogonal weight matrix is not just a matrix; it is a point on a Stiefel-type constraint. A covariance matrix is not just a symmetric array; it must stay positive definite.

The clean computational pattern is: encode the state, compute an ambient derivative if needed, convert it into a tangent or metric-aware object, take a small local step, and then return to the manifold with a geodesic formula or retraction. This is the same pattern used in the companion notebooks, just scaled down to visible two- and three-dimensional examples.

The important warning is that coordinate code can pass shape checks while still violating geometry. Differential geometry adds checks that are semantic: tangentness, smooth compatibility, metric choice, path validity, and constraint preservation.

Practical checklist:

• State the manifold and whether it is abstract, embedded, or quotient-like.
• State the local coordinates or tangent representation being used.
• Separate ambient vectors from tangent vectors.
• Name the metric before computing distances, angles, or gradients.
• Use geodesics or retractions when moving on the manifold.
• For ML claims, identify whether geometry is data geometry, parameter geometry, or statistical geometry.

Local diagnostic: Name the manifold, tangent space, metric, and map being used.

The companion notebook uses low-dimensional synthetic examples: circles, spheres, tangent projections, spherical interpolation, SPD matrices, and orthogonality constraints. These examples keep geometry visible while preserving the same update logic used in higher-dimensional ML systems.

A useful learning move is to compute everything first on a sphere. The sphere has visible curvature, simple tangent spaces, closed-form geodesics, and practical retractions. Once those are clear, Stiefel, Grassmann, SPD, and information-geometric examples become less mysterious.

For implementation, the main discipline is to avoid leaving the manifold silently. If a gradient step violates a constraint, either project the gradient into the tangent space before stepping or use a method whose update is intrinsic by design.

The final question for this subsection is whether a Euclidean formula is being used as an approximation, a coordinate expression, or a mistaken replacement for geometry. Differential geometry is the habit of telling those cases apart.

2.5 Embedded and immersed submanifolds

Embedded and immersed submanifolds belongs to the canonical scope of Manifolds. The goal is to make curved-space reasoning concrete enough for ML practice without turning the section into a pure topology course.

Working scope for this subsection: smooth manifolds, charts, atlases, tangent spaces, differentials, tangent bundles, embedded submanifolds, and ML manifold intuition. The recurring pattern is localize, linearize, measure, move, and return to the manifold.

$$\varphi_\beta\circ\varphi_\alpha^{-1}:\varphi_\alpha(U_\alpha\cap U_\beta)\to\varphi_\beta(U_\alpha\cap U_\beta).$$

Operational definition.

Embedded and immersed submanifolds belongs to the canonical scope of Manifolds: smooth manifolds, charts, atlases, tangent spaces, differentials, tangent bundles, embedded submanifolds, and ML manifold intuition.

Worked reading.

Start from a concrete embedded example, compute the local tangent or metric object, then translate back to intrinsic notation.

Three examples of embedded and immersed submanifolds:

1. Sphere geometry.
2. Embedding-space local coordinates.
3. Matrix-manifold parameter constraints.

Two non-examples clarify the boundary:

1. A flat Euclidean approximation used globally.
2. A geometric claim made without metric or tangent space.

Proof or verification habit for embedded and immersed submanifolds:

The proof habit is to compute locally and verify coordinate-independent meaning.

global object      -> curved manifold or constraint set
local object       -> chart, tangent space, or coordinate patch
linear operation   -> derivative, gradient, velocity, Hessian approximation
geometric measure  -> metric, length, distance, curvature
algorithmic move   -> tangent step followed by geodesic or retraction

In AI systems, embedded and immersed submanifolds matters because learned representations and constrained parameter spaces are rarely globally flat. A local linear approximation may be useful, but it must be attached to the point where it is valid.

The AI relevance is that model spaces are often curved even when implemented as arrays.

Mini derivation lens.

1. Choose a point $p$ on the manifold $M$ and name the local representation used near $p$.
2. Move the question into a chart, tangent space, or embedded constraint where first-order calculus is available.
3. Compute the local object: derivative, tangent projection, metric-weighted gradient, path velocity, or retraction step.
4. Translate the result back into coordinate-free language so the answer is not tied to one chart by accident.
5. Check the invariant: the point remains on $M$, the direction remains in $T_pM$, or the distance/gradient uses the stated metric.

Implementation lens.

A practical ML implementation should store both the ambient array representation and the geometric contract attached to it. For example, a normalized embedding is not just a vector; it is a point on a sphere. An orthogonal weight matrix is not just a matrix; it is a point on a Stiefel-type constraint. A covariance matrix is not just a symmetric array; it must stay positive definite.

The clean computational pattern is: encode the state, compute an ambient derivative if needed, convert it into a tangent or metric-aware object, take a small local step, and then return to the manifold with a geodesic formula or retraction. This is the same pattern used in the companion notebooks, just scaled down to visible two- and three-dimensional examples.

The important warning is that coordinate code can pass shape checks while still violating geometry. Differential geometry adds checks that are semantic: tangentness, smooth compatibility, metric choice, path validity, and constraint preservation.

Practical checklist:

• State the manifold and whether it is abstract, embedded, or quotient-like.
• State the local coordinates or tangent representation being used.
• Separate ambient vectors from tangent vectors.
• Name the metric before computing distances, angles, or gradients.
• Use geodesics or retractions when moving on the manifold.
• For ML claims, identify whether geometry is data geometry, parameter geometry, or statistical geometry.

Local diagnostic: Name the manifold, tangent space, metric, and map being used.

The companion notebook uses low-dimensional synthetic examples: circles, spheres, tangent projections, spherical interpolation, SPD matrices, and orthogonality constraints. These examples keep geometry visible while preserving the same update logic used in higher-dimensional ML systems.

A useful learning move is to compute everything first on a sphere. The sphere has visible curvature, simple tangent spaces, closed-form geodesics, and practical retractions. Once those are clear, Stiefel, Grassmann, SPD, and information-geometric examples become less mysterious.

For implementation, the main discipline is to avoid leaving the manifold silently. If a gradient step violates a constraint, either project the gradient into the tangent space before stepping or use a method whose update is intrinsic by design.

The final question for this subsection is whether a Euclidean formula is being used as an approximation, a coordinate expression, or a mistaken replacement for geometry. Differential geometry is the habit of telling those cases apart.