| DATE | Lecture | Topic | Scribe |
| 3/27/2000 | Lecture 1 /pdf | Overview, Intro DGMs I | Source | 3/29/2000 | Lecture 2 /pdf | DGMs, Bayes-ball, d-sep, example, start elimination algorithm | Source | 4/3/2000 | Lecture 3 /pdf | Finish elimination, undirected graphs, linear Gaussian, LMS | Source | 4/5/2000 | Lecture 4 /pdf | Why LGMs? 3 parts of a GM. Linear regression, econometrics, PCA, FA, Static Kalman, Inference, EM as coordinate ascent. | Source | 4/10/2000 | Lecture 5 /pdf | EM, Mixture models as GMs, Gaussian Mixtures & EM, Gaussians & marginal independence | Source | 4/12/2000 | Lecture 6 /pdf | Gaussians and marginal vs. conditional independence, partitioned matrices and Shur complement, Gaussians and UGMs, Gaussians and DGMs, from mixture models to HMMs, facts about HMMs. | Source | 4/17/2000 | Lecture 7 /pdf | HMMs and inference (e.g., alpha/beta and other recursions), HMMs and EM, dynamic linear Gaussian models, Kalman filters and LG-HMMs. | Source | 4/19/2000 | Lecture 8 /pdf | Finish LG-HMMs, other extensions of factor analysis such as mixtures of FA, PCA, and PPCA, Independent FA, Independent component analysis, Linear discriminant analysis (LDA) as directed graphical models, and other generalizatoins of LDA such as HDA, MDA, and HMDA. Begin more formal definitions of graphs. | Source | 4/24/2000 | Lecture 9 /pdf | Formal notation and terminology. Decomposable, triangulated or chordal, start equivalent conditions for graphs proof (decomposable == chordal == every minimal alpha-beta sep is complete). | Source | 4/26/2000 | Lecture 10 /pdf | Finish decomposable == chordal == every min alpha/beta sep is complete proof. Junction tree === decomposability proof, running intersection property (rip), junction tree construction using rip | Source | 5/1/2000 | Lecture 11 /pdf | review rip & junction tree construction, perfect DAGs, perfect numberings, and chordality, chordality tests via elimination, eliminatable graphs, maximum cardinality search, ladder based clique construction, elimination is NP-Hard, elimination as an approximation method for triangulation | Source | 5/3/2000 | Lecture 12 /pdf | When are trees of cliques junction trees, weight of a tree of cliques, maximum spanning tree, JT if and only if tree of cliques is a maximum spanning tree over the graph of cliques, formal conditional independence, properties thereof, positive densities, graphoids & semi-graphoids, Markov properties on undirected graphs (pairwise, local, and global), when which property implies the others, factorization. | Source | 5/8/2000 | Lecture 13 /pdf | Review, factorization, proof that (F) implies (G), Mobius Inversion Lemma, Hammersley-Clifford Theorem, decomposability and factorization, junction trees and factorization, Markov properties on directed graphs (directed factorization (DF), directed global (DG)), d-separation again), Markov blankets. | Source | 5/10/2000 | Lecture 14 /pdf | Review of DGM Markov properties including the directed factorization (DF), direted global (DG), d-separation in the UGM and separation in the moralized graph of the smallest ancestral set containing A, B, and S, local directed (DL), ordered directed (DO), and equivalence of all the properties (and Bayes ball). General exact inference procedure: DGM to UGM via moralization, inference by achieving local consistency on right data structure. | Source | 5/15/2000 | Lecture 15 | Message passing algorithm to achieve local consistency, this is a forward-backward procedure, introducing evidence, consistency in tree of cliques, message passing protocol, algorithms that satisfy the message passing protocol, Hugin and Collect/Distribute Evidence, proof for why junction tree is key for why local consistency in scheme that obeys message passing protocol implies clique potentials as marginal probabilities over those nodes, reason for wanting triangulated graphs revisited. | Source | 5/17/2000 | Lecture 16 | Complexity of the following operations: Moralization, finding optimal triangulations, constructing junction trees, and performing inference. So, finding small cliques is NP-hard but is key. Junction tree algorithm on HMMs can lead to standard alpha probabilities from HMM algorithms (but many more algorithms for HMM inference exist). Structure Learning I: scientific modeling (finding the truth), markov equivalent graphs, causal models vs. belief networks, examples from epidemiology, four aspects of a GM, four types of learning in Bayesian networks (known or unknown structure, complete or missing data). | Source | 3/22/2000 | Lecture 17 | Structure Learning II: Known structure, complete data is easy, decomposable sub-problems so easy, for DPTs, just counting. Incomplete data, known structure, EM, MCMC. Complete data, unknown structure: likelyhood, penalized likelyhood (BIC, MDL), Bayesian inference (conjugate priors such as multinomials and Dirichlets), Bayesian model scores, Bayesian approach to model structure, Bayesian and BIC, model selection, selective model averaging. Model building: tree dependent distributions, information theory, Chow-Tree algorithm, 2nd-order tree dependent distributions, 2nd order Chow-Tree. Naive Bayes Classifiers, Tree-augmented models (TAM), Bayesian Multinets. | Source | 3/24/2000 | Lecture 18 | Structure Learning III: likelihood vs posterior learning for classifier problems. Tree augmented models, Bayesian Multinets, Using conditional MI to compute multinets TAMs, missing data/unknown structure reasons, scores and decomposability, naive approach, Structural EM algorithm, heuristics for inducing new hidden structure. | Source |