| CompSci 274A PROB LEARNING | ||||||||||||||||
| Code | Typ | Sec | Unt | Instructor | Time | Place | Final | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 35300 | Lec | A | 4 | IHLER, A. | TuTh 2:00- 3:20p | SSL 206 | Thu, Jun 12, 1:30-3:30pm | |||||||||
Introduction to probabilistic models, inference, and learning.
CS274A is an introductory course to probabilistic approaches to learning from data. Probabilistic models form an important part of many areas of computer science, and probabilistic learning (in this context, automatically constructing probabilistic models from data) has become an important tool in sub-fields such as artificial intelligence, data mining, speech recognition, computer vision, bioinformatics, signal processing, and many more. CS274A will provide an introduction to the concepts and principles which underly probabilistic models, and apply these principles to the development, analysis, and practical application of machine learning algorithms.
The course will focus primarily on parametric probabilistic modeling, including data likelihood, parameter estimation using likelihood and Bayesian approaches, hypothesis testing and classification problems, density estimation, clustering, and regression. Related problems, including model selection, overfitting, and bias/variance trade-offs will also be discussed.
Background. The course is intended to be an introduction to probabilistic learning, and thus has few explicit requirements. Students are expected to be familiar with basic concepts from probability, linear algebra, multivariate calculus, etc. Homeworks will use the MATLAB programming environment, but no prior experience with MATLAB is required for the course.
From the course catalog: Probabilistic Learning: Theory and Algorithms. An introduction to probabilistic and statistical techniques for learning from data, including parameter estimation, density estimation, regression, classification, and mixture modeling.
Course format. Two lectures per week. Homeworks due approximately every two weeks. Two exams (midterm and final). Grading: 40% homework, 30% midterm, 30% final.
Textbooks. The required textbook for the course is Bishop's "Pattern Recognition and Machine Learning", but lectures are likely to follow the book only loosly. Other recommended reading include MacKay's "Information Theory, Inference, and Learning Algorithms" (available online at http://www.inference.phy.cam.ac.uk/mackay/itila/), Duda, Hart, and Stork's "Pattern Classification", and Hastie, Tibshirani, and Friedman's "Elements of Statistical Learning".
(Tentative) Schedule of Topics.
Week 1 | 04/01/2008 |
Introduction; probability distributions, Bayes' rule, etc. |
|
|
|
| 04/03/2008 |
introduction continued; multivariate distributions |
|
|
|
Week 2 | 04/07/2008 |
Intro to learning. Parameters, likelihood functions. Bias/Variance. |
|
|
|
| 04/09/2008 |
Maximum likelihood learning I: univariate exponential family |
|
|
|
Week 3 | 04/14/2008 |
Maximum likelihood learning II: multivariate models |
|
|
|
| 04/16/2008 |
Bayesian learning I: prior and posterior distributions; MAP estimation |
|
|
|
Week 4 | 04/21/2008 |
Bayesian learning II: conjugate prior distributions |
|
|
|
| 04/23/2008 |
Bayesian learning III: Bayesian estimation; Monte Carlo methods |
|
|
|
Week 5 | 04/28/2008 |
Regression I: linear and logistic regression |
|
|
|
| 04/30/2008 |
MIDTERM EXAM |
|
|
|
Week 6 | 05/06/2008 |
Regression II: probabilistic interpretations, parameter estimation |
|
|
|
| 05/08/2008 |
Classification I: introduction, risk, optimality; hypothesis testing |
|
|
|
Week 7 | 05/13/2008 |
Classification II: discriminants and classification boundaries |
|
|
|
| 05/15/2008 |
Classification III: linear classifiers; connections to regression |
|
|
|
Week 8 | 05/20/2008 |
Mixture models and EM: mixtures of Gaussians; k-means; EM |
|
|
|
| 05/22/2008 |
Mixture models and EM: more on expectation-maximization |
|
|
|
Week 9 | 05/27/2008 |
Graphical models I: structured multivariate distributions |
|
|
|
| 05/29/2008 |
Graphical models II: using structure for efficient computation; Kalman filtering, forward-backward |
|
|
|
Week 10 | 06/03/2008 |
Additional topics: TBD |
|
|
|
| 06/05/2008 |
Additional topics: TBD |
|
|
|
Final Exam | 06/12/2008 |
In class final exam, 1:30-3:30pm |