Probability and Statistics

Probability is the other foundational language of machine learning. Models define distributions, training maximizes likelihood, and generalization is a statistical guarantee.

• Probability Basics -- Axioms, conditional probability, Bayes' theorem, expectation, variance
• Distributions -- Bernoulli, Gaussian, Categorical, Softmax, multivariate Gaussian, mixture models
• Bayesian Inference -- Prior, likelihood, posterior, MAP vs MLE, variational inference
• Information Theory -- Entropy, cross-entropy, KL divergence, mutual information
• Statistical Learning Theory -- PAC learning, VC dimension, generalization bounds, bias-variance decomposition