Singular Value Decomposition (SVD)

What is SVD?

Singular Value Decomposition (SVD) is a way to break down any matrix into three special matrices:

A = U Σ VT

• U is an orthogonal matrix (columns are orthonormal)
• Σ (sigma) is a diagonal matrix with singular values
• V^T is the transpose of another orthogonal matrix

SVD works for any matrix — square or rectangular, real or complex!

Finding Singular Values & Vectors

The singular values are the square roots of the eigenvalues of ATA.

Steps:

1. Compute ATA
2. Find its eigenvalues λ_i
3. Singular values σ_i = √λ_i
4. Find corresponding eigenvectors to form V
5. Compute U using: ui = (1/σi) A vi

These values tell us how the matrix stretches or squishes space along certain directions.

Applications of SVD

• 📉 Dimensionality Reduction — reduces features in data while preserving the most important patterns (used in PCA)
• 🎯 Recommendation Systems — Netflix, YouTube, and Amazon use SVD to recommend what you might like
• 📷 Image Compression — reduce file size while keeping the visual quality
• 📊 Latent Semantic Analysis (LSA) — helps search engines understand the context of words in documents

🔗 SVD is one of the most powerful tools in linear algebra, used widely in data science, signal processing, and machine learning.