Linear Algebra

Linear algebra is the language of machine learning. Every model is a composition of linear transformations and nonlinearities, every dataset is a matrix, and every optimization step is a vector operation.

• Vectors and Matrices: Vector spaces, norms, inner products, orthogonality, and projection
• Matrix Operations: Multiplication, transpose, inverse, rank, trace, and determinant
• Eigendecomposition: Eigenvalues, eigenvectors, spectral theorem, and positive definiteness
• SVD and PCA: Singular value decomposition, low-rank approximation, and principal component analysis
• Matrix Calculus: Gradients of scalar, vector, and matrix functions, Jacobians, and Hessians