Linear Algebra

Linear algebra is the language of machine learning. Every model is a composition of linear transformations and nonlinearities, every dataset is a matrix, and every optimization step is a vector operation.

• Vectors and Matrices -- Vector spaces, norms, inner products, orthogonality, and projection
• Matrix Operations -- Multiplication, transpose, inverse, rank, trace, and determinant
• Eigendecomposition -- Eigenvalues, eigenvectors, spectral theorem, and positive definiteness
• SVD and PCA -- Singular value decomposition, low-rank approximation, and principal component analysis
• Matrix Calculus -- Gradients of scalar, vector, and matrix functions, Jacobians, and Hessians