Max Welling's Classnotes in Machine Learning

Statistical Estimation [ps]
- bayesian estimation
- maximum a posteriori (MAP) estimation
- maximum likelihood (ML) estimation
- Bias/Variance tradeoff & minimum description length (MDL)

Expectation Maximization (EM) Algorithm [ps]
detailed derivation plus some examples

Supervised Learning (Function Approximation) [ps]
- mixture of experts (MoE)
- cluster weighted modeling (CWM)

Clustering [ps]
- mixture of gaussians (MoG)
- vector quantization (VQ) with k-means.

Linear Models [ps]
- factor analysis (FA)
- probabilistic principal component analysis (PPCA)
- principal component analysis (PCA)

Independent Component Analysis (ICA) [ps]
- noiseless ICA
- noisy ICA
- variational ICA

Mixture of Factor Analysers (MoFA) [ps]
- derivation of learning algorithm

Hidden Markov Models (HMM) [ps]
- viterbi decoding algorithm
- Baum-Welch learning algorithm

Kalman Filters (KF) [ps]
- kalman filter algorithm (very detailed derivation)
- kalman smoother algorithm (very detailed derivation)

Approximate Inference Algorithms [ps]
- variational EM
- laplace approximation
- importance sampling
- rejection sampling
- markov chain monte carlo (MCMC) sampling
- gibbs sampling
- hybrid monte carlo sampling (HMC)

Belief Propagation (BP) [ps]
- Introduction to BP and GBP: powerpoint presentation [ppt]
- converting directed acyclic graphical models (DAG) into junction trees (JT)
- Shafer-Shenoy belief propagation on junction trees
- some examples

Boltzmann Machine (BM) [ps]
- derivation of learning algorithm

Generative Topographic Mapping (GTM) [ps]
- derivation of learning algorithm

Introduction to Kernel Methods: powerpoint presentation [ppt]

Kernel Principal Components Analysis [pdf]

Kernel Canonical Correlation Analysis [pdf]

Kernel Support Vector Machines [pdf]

Kernel Ridge-Regression [pdf]

Kernel Support Vector Regression [pdf]

Convex Optimization [pdf]
A brief introduction based on Stephan Boyd’s book, chapter 5.

Fisher Linear Discriminant Analysis [pdf]

This material is based upon work supported by the National Science Foundation under Grant No. 0447903. and Grant No. 0535278. Any opinions, findings and conclusions or recomendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF).

[Back to Max Welling's's home page