Stat 700, Probabilistic graphical models
Suggestion for projects
Instructor: Long Nguyen
Course homepage
Types of Projects
A class project can be in forms of either an application research or
a methodological (computational/theoretical) research project,
or a survey paper related to graphical models.
A project can be done individually or jointly in pairs.
An application project is typically an application of
graphical models to real data, where computational
issue plays a significant role in the modeling and inference.
A computational/theoretically inclined project can be an
investigation/comparison of some existing inference/estimation
algorithm(s) in a class of graphical models. Or, it could also be
a proposal of a new method/new model.
For survey papers, see below.
Please email me by November 12 a paragraph
describing the type of project and topic that you plan to do.
Final project report is due December 17th.
Survey topics
The following are topic suggestions for survey papers
Some references listed here serve as initial pointers,
and are by no means the representative papers of a given
topic. (Many topics listed below are currently seeing intense
research activities, so a survey could be used as a
starting point for your research exploration).
You are of course free to choose topics other than those
listed below.
- Model reparameterization and computational efficiency
of inference.
- Gelfand, A. E., Sahu, S. K., and Carlin, B. P. (1995),
"Efficient Parameterizations for Normal Linear Mixed Models," Biometrika, 82, 479-488.
- O. Papaspiliopoulos.
A General Framework for the Parametrization of Hierarchical Models,
Statistical Science, 2007.
- Convergence of the EM algorithm and the Gibbs sampler, e.g.:
- On convergence of the EM algorithm and the Gibbs sampler,
S. Sahu and G. Roberts, Statistics and Computing,
Volume 9, 55 - 64, 1999.
- Estimation methods for undirected graphical models,
including non-likelihood based methods (e.g., methods
based on surrogates for the likelihood such as
pseudolikelihood, contrastive divergence, etc) .
- A good place to start is Section 7 in Wainwright-Jordan's
monster paper.
- A recent paper on consistency analysis:
S. Chatterjee, Estimation in spin glasses: A first step.
Annals of Statistics, Vol 35, No. 5, 1931--1946, 2007.
- Estimation of sparse graphical models, structure estimation. E.g.:
- N. Meinshausen and P. Buhlmann. High-dimensional graphs
and variable selection with the Lasso. Annals of Statistics,
Vol 34, No. 3, 1436--1462, 2006.
- P.J. Bickel and E. Levina. Regularized Estimation of Large
Covariance Matrices. Annals of Statistics 36(1),199-227, 2008.
- P. Ravikumar, M. J. Wainwright and J. Lafferty.
High-dimensional Ising model selection using l1-regularized logistic
regression. To appear in the Annals of Statistics.
- Sequential Monte Carlo sampling methods
- Variational inference methods (such as higher-order based methods,
convex relaxations via linear programming, etc)
- A good place to start is Sections 8 and 9 of the Wainwright-Jordan's
paper.
- Inference algorithms (both variational and sampling methods)
for curved exponential families of distribution
- Sampling methods for computation of log-partition function
(normalizing constant) in graphical and hierarchical models.
- Graphical models for infinite-dimensional objects (functions, curves,
distributions): Hierachical nonparametric Bayesian models
(e.g., mixture models using Dirichlet process or
Gaussian process as the mixing mechanism)
- Hierarchical Dirichlet processes. Y. W. Teh, M. I. Jordan, M. J. Beal and D. M. Blei. Journal of the American Statistical Association, 101, 1566-1581, 2006.
[ pdf ]
- The nested Dirichlet process. A Rodriguez, D. Dunson and A. Gelfand.
Journal of the American Statistical Association, 2008.
[ pdf ]
- Bayesian Nonparametric Spatial Modeling with Dirichlet Process Mixing.
AE Gelfand, A Kottas, SN MacEachern. Journal of the American Statistical Association, 2005.
[ pdf ]
- Generalized spatial Dirichlet process models.
JA Duan, M Guindani, AE Gelfand. Biometrika, 2007.
[ pdf ]
- Variational inference methods for nonparametric Bayes
models
- D. Blei and M. Jordan. Variational inference for Dirichlet process mixtures. Journal of Bayesian Analysis, 1[1]:121–144, 2006.
[ pdf ]
- Sampling methods for nonparametric Bayes models
- Asymptotic analysis, model identifiability of nonparametric Bayes models
- Dirichlet process, related priors and posterior asymptotics. In Bayesian Nonparametrics, (N. L. Hjort et al., eds.), pages 36-83, Cambridge University Press (in press).
[ pdf ]
- Ishwaran, H. and Zarepour, M. (2002), “Dirichlet prior sieves in finite normal mixtures,” Statistica Sinica, 12, 941–963.
- Markov random fields for infinite graphs (Gibbs measure)
- H.-O. Georgii, O. Häggström and C. Maes (2001).
The Random Geometry of Equilibrium Phases, Phase Transitions and
Critical Phenomena, Volume 18 (C. Domb and J.L. Lebowitz, eds), pp 1-142, Academic Press, London. [ pdf ]
- Mixing time of MCMC, efficiency of sum-product algorithms
in graphical models and relations to phase transition and Gibbs measure
-
The Ising model on trees: Boundary conditions and mixing time.
Fabio Martinelli, Alistair Sinclair and Dror Weitz. Technical Report UCB//CSD-03-1256, UC Berkeley, July 2003. Slightly modified version appeared in Communications in Mathematical Physics 250 (2004), pp. 301-334.
[ pdf ]
- Loopy belief propagation and Gibbs measures. S. Tatikonda and M. I. Jordan. In D. Koller and A. Darwiche (Eds)., Uncertainty in Artificial Intelligence (UAI), Proceedings of the Eighteenth Conference, 2002.
[ pdf ]