Stat 700, Probabilistic graphical models
Suggestion for projects

Types of Projects

A class project can be in forms of either an application research or a methodological (computational/theoretical) research project, or a survey paper related to graphical models. A project can be done individually or jointly in pairs.

An application project is typically an application of graphical models to real data, where computational issue plays a significant role in the modeling and inference.

A computational/theoretically inclined project can be an investigation/comparison of some existing inference/estimation algorithm(s) in a class of graphical models. Or, it could also be a proposal of a new method/new model.

For survey papers, see below.

Please email me by November 12 a paragraph describing the type of project and topic that you plan to do.

Final project report is due December 17th.

Survey topics

The following are topic suggestions for survey papers Some references listed here serve as initial pointers, and are by no means the representative papers of a given topic. (Many topics listed below are currently seeing intense research activities, so a survey could be used as a starting point for your research exploration). You are of course free to choose topics other than those listed below.

1. Model reparameterization and computational efficiency of inference.
   • Gelfand, A. E., Sahu, S. K., and Carlin, B. P. (1995), "Efficient Parameterizations for Normal Linear Mixed Models," Biometrika, 82, 479-488.
   • O. Papaspiliopoulos. A General Framework for the Parametrization of Hierarchical Models, Statistical Science, 2007.
2. Convergence of the EM algorithm and the Gibbs sampler, e.g.:
   • On convergence of the EM algorithm and the Gibbs sampler, S. Sahu and G. Roberts, Statistics and Computing, Volume 9, 55 - 64, 1999.
3. Estimation methods for undirected graphical models, including non-likelihood based methods (e.g., methods based on surrogates for the likelihood such as pseudolikelihood, contrastive divergence, etc) .
   • A good place to start is Section 7 in Wainwright-Jordan's monster paper.
   • A recent paper on consistency analysis: S. Chatterjee, Estimation in spin glasses: A first step. Annals of Statistics, Vol 35, No. 5, 1931--1946, 2007.
4. Estimation of sparse graphical models, structure estimation. E.g.:
   • N. Meinshausen and P. Buhlmann. High-dimensional graphs and variable selection with the Lasso. Annals of Statistics, Vol 34, No. 3, 1436--1462, 2006.
   • P.J. Bickel and E. Levina. Regularized Estimation of Large Covariance Matrices. Annals of Statistics 36(1),199-227, 2008.
   • P. Ravikumar, M. J. Wainwright and J. Lafferty. High-dimensional Ising model selection using l1-regularized logistic regression. To appear in the Annals of Statistics.
5. Sequential Monte Carlo sampling methods
6. Variational inference methods (such as higher-order based methods, convex relaxations via linear programming, etc)
   • A good place to start is Sections 8 and 9 of the Wainwright-Jordan's paper.
7. Inference algorithms (both variational and sampling methods) for curved exponential families of distribution
8. Sampling methods for computation of log-partition function (normalizing constant) in graphical and hierarchical models.
   • X. Meng and W. Wong, Simulating ratios of normalizing constants via a simple identity: A theoretical exploration. Statistica Sinica 6(1996), 831-860.
   • A. Gelman and X. Meng, Simulating normalizing constants: from importance sampling to bridge sampling to path sampling. Statist. Sci. Volume 13, Number 2 (1998), 163-185.
   • M. Jerrum and A. Sinclair, Polynomial-time Approximation Algorithms for the Ising Model, SIAM Journal on Computing 22 (1993), pp. 1087-1116.
9. Graphical models for infinite-dimensional objects (functions, curves, distributions): Hierachical nonparametric Bayesian models (e.g., mixture models using Dirichlet process or Gaussian process as the mixing mechanism)
   • Hierarchical Dirichlet processes. Y. W. Teh, M. I. Jordan, M. J. Beal and D. M. Blei. Journal of the American Statistical Association, 101, 1566-1581, 2006. [ pdf ]
   • The nested Dirichlet process. A Rodriguez, D. Dunson and A. Gelfand. Journal of the American Statistical Association, 2008. [ pdf ]
   • Bayesian Nonparametric Spatial Modeling with Dirichlet Process Mixing. AE Gelfand, A Kottas, SN MacEachern. Journal of the American Statistical Association, 2005. [ pdf ]
   • Generalized spatial Dirichlet process models. JA Duan, M Guindani, AE Gelfand. Biometrika, 2007. [ pdf ]
10. Variational inference methods for nonparametric Bayes models
    • D. Blei and M. Jordan. Variational inference for Dirichlet process mixtures. Journal of Bayesian Analysis, 1[1]:121–144, 2006. [ pdf ]
11. Sampling methods for nonparametric Bayes models
12. Asymptotic analysis, model identifiability of nonparametric Bayes models
    • Dirichlet process, related priors and posterior asymptotics. In Bayesian Nonparametrics, (N. L. Hjort et al., eds.), pages 36-83, Cambridge University Press (in press). [ pdf ]
    • Ishwaran, H. and Zarepour, M. (2002), “Dirichlet prior sieves in finite normal mixtures,” Statistica Sinica, 12, 941–963.
13. Markov random fields for infinite graphs (Gibbs measure)
    • H.-O. Georgii, O. Häggström and C. Maes (2001). The Random Geometry of Equilibrium Phases, Phase Transitions and Critical Phenomena, Volume 18 (C. Domb and J.L. Lebowitz, eds), pp 1-142, Academic Press, London. [ pdf ]
14. Mixing time of MCMC, efficiency of sum-product algorithms in graphical models and relations to phase transition and Gibbs measure
    • The Ising model on trees: Boundary conditions and mixing time. Fabio Martinelli, Alistair Sinclair and Dror Weitz. Technical Report UCB//CSD-03-1256, UC Berkeley, July 2003. Slightly modified version appeared in Communications in Mathematical Physics 250 (2004), pp. 301-334. [ pdf ]
    • Loopy belief propagation and Gibbs measures. S. Tatikonda and M. I. Jordan. In D. Koller and A. Darwiche (Eds)., Uncertainty in Artificial Intelligence (UAI), Proceedings of the Eighteenth Conference, 2002. [ pdf ]