Stats 270/370

A Course in Bayesian Statistics

This class is the first of a two-quarter sequence that will serve as an introduction to the Bayesian approach to inference, its theoretical foundations and its application in diverse areas. The instructors are Persi Diaconis, Chiara Sabatti and Wing Wong. Following is a tentative outline of lectures. As we move through the quarter, it will become more precise and reflect, week by week, the material actually covered in class.

1. Introduction to Bayesian Statistics. Examples of current application of the Bayesian inferential framework. The fundamentals: prior, likelihood, posterior.
   • Good (1983) 46656 Varieties of Bayesians (#765) in Good Thinking. The Foundations of Probability and Its Applications, Univeristy of Minnesota Peress, Minneapolis.
   • Diaconis and Ylvisaker (1983) Quantifying Prior Opinion, Bayesian Statistics 2. Proc. 2nd Valencia Int'l Meeting, 9-83. J. M. Bernardo, M. H. Degroot, D. V. Lindley, A. F. M. Smith (eds.) North-Holland, Amsterdam 133-156 (1985).
   • Lawrence CE, Altschul SF, Boguski MS, Liu JS, Neuwald AF, Wootton JC. (1993) Detecting subtle sequence signals: a Gibbs sampling strategy for multiple alignment, Science 262:208-14.
   • Liu, Neuwald and Lawrence (1995) Bayesian Models for Multiple Local Sequence Alignment and Gibbs Sampling Strategies JASA 90: 1165-1170.
   • Bayes (1763) An essay toward solving a problem in the doctrine of chances, reprinted in 1958, Biometrika 45:296-315.
   • Stigler (1982) Thomas Bayes's Bayesian Inference Journal of the Royal Statistical Society. Series A, 145:250-258.
   • Zabell, Sandy W. E. Johnson's "Sufficientness" Postulate, Ann. Statist. 10 (1982), no. 4, 1090--1099.
   • Diaconis and Holmes (2002) A Bayesian peak into Feller volume I Sankhya Series A 3:820-841.
2. Deeper understanding of Bayesian calculus
   1. Bayes theorem for the non dominated case.
      • "Probability and potentials", by Paul A. Meyer. Blaisdell, New York, 1966
      • Look for 'Regular conditional probability' in "Probability" by Breiman, "Probability and measure", Billingsley.
   2. Exponential families and conjugate priors
      • Diaconis and Ylvisaker (1979) Conjugate priors for exponential families The Annals of Statistics, 7:269-281.
   3. Gaussian
      • Jackman (2009) Bayesian Analysis for the Social Sciences, Wiley (Example: Suspected voter fraud in Pennsylvania)
3. Choices of prior distributions. Invariance, non informative priors.
   • Kass and Wasserman (1996) The selection of prior distributions by formal rules, JASA 91:1343-1370.
4. Subjective definition of probability. Coherence, betting schemes.
   • Tversky and Koehler (1994) Support Theory: a nonextensional representation of subjective probability, Psycological Review, 101: 547-567.
   • Brier (1950) Verification of forecasts expressed in terms of probability, Monthly weather review 78:1-3.
   • Cifarelli and Regazzini (1996) De Finetti's contribution to probability and statistics Statistical Science, 11:253-282.
   • Chapter 15 of Jaynes (2003)
5. Exchangeability
   • Diaconis (1977) Finite forms of de Finetti's theorem on exchangeability, Synthese, 36: 271-281.
   • Diaconis and Friedman (1982) Partial exchangeability and sufficiency Tech Rep 190, Department of Statistics, Stanford. Published in Proc. IndianStat. Inst. Golden Jubilee Int'l Conf. Stat.Applications and New Directions , J. K. Ghosh and J. Roy (eds.), Indian Statistical Institute, Calcutta, 205-236
   • P. Diaconis and D. Freedman (1980) Finite Exchangeable Sequences Ann. Probab. 8: 745-764.
6. Inferential principles Sufficiency, likelihood principle, ancillarity
   • Cox and Hinkley (1979) Theoretical Statistics, Chapman & Hall. Chapter 2.
   • Fisher (1922) On the Mathematical Foundations of Theoretical Statistics, Philosophical Transactions of the Royal Society, A: 222: 309-368.
   • Fisher (1925) Theory of Statistical Estimation Proceedings of the Cambridge Philosophical Society, 22: 700-725
   • Hinkley (1980) Theory of statistical estimation: the 1925 paper, in "R.A. Fisher: an appreciation," Fienberg and Hinkley editors, Springer.
7. Bayesian regression Standard framework for multivariate gaussian distributios; prior distribution on variance-covariance matrix; multivariate regression methods.
   • Zellner (1986) On assessing prior distributions and Bayesian regression analysis with g-prior distributions in "Bayesian Inference and decision techniques", Goel and Zellner ed., Elsevier science publishers.
   • George and McCulloch (1997) Approaches for Bayesian variable selection Statistica Sinica 7: 339-373.
   • Liang, Paulo, Molina, Clyde and Berger (2008) Mixtures of g Priors for Bayesian Variable Selection JASA 103: 410-423.
   • Park and Casella (2008) The Bayesian lasso JASA 103: 681-686.
8. Computation How to explore the posterior: numerical integration, importance sampling, MCMC schemes.
   • Metropolis, Rosenbluth, Rosenbluth, Teller, Teller, E. (1953) Equation of State Calculations by Fast Computing Machines J. Chem. Phys. 21, 1087-92.
   • Hastings (1970) Monte Carlo Sampling Methods Using Markov Chains and Their Applications Biometrika 57: 97-109
   • Geman and Geman (1984) Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images IEEE Transactions on Pattern Analysis and Machine Intelligence 6: 721-741.
   • Tanner and Wong (1987) The Calculation of Posterior Distributions by Data Augmentation JASA 82:528-540.
9. Hierarchical models and computation . Examples of MCMC; empirical bayes; applications.
   • Chapter 5 of BDA3
   • Efron and Morris (1975) Data analysis using Stein's estimator and its generalizations, JASA 70:311-319.
   • Abramovich F., Sapatinas T. & Silverman B.W. (1998) Wavelet thresholding via a Bayesian approach . Journal of the Royal Statistical Society, Ser. B, 60: 725-749.