BasicModule2.org

1 Singular Value Decomposition

$$Am*n = Um*m Σm*n Vn*n^T$$

In linear algebra, the singular value decomposition (SVD) is a factorization of a real or complex matrix. It is the generalization of the eigendecomposition of a positive semidefinite normal matrix (for example, a symmetric matrix with positive eigenvalues) to any ${\displaystyle m× n} m× n$ matrix via an extension of the polar decomposition. It has many useful applications in signal processing and statistics.

Formally, the singular value decomposition of an ${\displaystyle m× n} m× n$ real or complex matrix ${\displaystyle \mathbf {M} } \mathbf {M}$ is a factorization of the form ${\displaystyle \mathbf {UΣ V} } {\displaystyle \mathbf {UΣ V} }$, where ${\displaystyle \mathbf {U} } \mathbf {U}$ is an ${\displaystyle m× m} m× m$ real or complex unitary matrix, ${\displaystyle \mathbf {Σ } } \mathbf{Σ}$ is a ${\displaystyle m× n} m× n$ rectangular diagonal matrix with non-negative real numbers on the diagonal, and ${\displaystyle \mathbf {V} } \mathbf {V}$ is an ${\displaystyle n× n} n× n$ real or complex unitary matrix. The diagonal entries ${\displaystyle σ _i} σ _i$ of ${\displaystyle \mathbf {Σ } } \mathbf{Σ}$ are known as the singular values of ${\displaystyle \mathbf {M} } \mathbf {M}$ . The columns of ${\displaystyle \mathbf {U} } \mathbf {U}$ and the columns of ${\displaystyle \mathbf {V} } \mathbf {V}$ are called the left-singular vectors and right-singular vectors of ${\displaystyle \mathbf {M} } \mathbf {M}$ , respectively.

The singular value decomposition can be computed using the following observations:

The left-singular vectors of M are a set of orthonormal eigenvectors of MM∗. The right-singular vectors of M are a set of orthonormal eigenvectors of M∗M. The non-zero singular values of M (found on the diagonal entries of Σ) are the square roots of the non-zero eigenvalues of both M∗M and MM∗. Applications that employ the SVD include computing the pseudoinverse, least squares fitting of data, multivariable control, Matrix Approximation, and determining the rank, range and null space of a matrix.

2 Convolve Networks(CNN)