STAT 812/420 Computational Statistics

Longhai Li, Dept of Math and Statistics, University of Saskatchewan

Description

Computationally intensive methods have become widely used in statistical inference. The objective of this course is to teach students important computational techniques used in statistical inference (evaluation of statistical methods, MLE and Bayesian inference). After learning this course, students are expected to gain understanding of algorithms behind statistical inferential methods, be able to develop new statistical methods, be able to use computer to investigate the properties of statistical methods, and be able to implement a combination of standard statistical toolkits for analyzing real data sets.

Prerequisites: multivariate calculus (MATH 225), basic algebra (MATH 164), an introductory statistics (eg. STAT 245), and a probability course (eg. STAT 241 is good enough but STAT 342 is preferred).

List of Topics

1. Introduction to Statistical Inference and R

• A very short introduction to R and Rstudio

• Introduction to R by R core team

• R code for introducing basic R features (Unit 1)

2. Computer Arithmetic

• R code demonstrating computer arithmetic (overflow, underflow and rounding errors) (Unit 2)

3. Random Numbers Generators, Monte Carlo, Simulation for Evaluating Point Estimators, Permutation test

• Simple Sampling (Random numbers generator and inverting CDF sampling, Monte Carlo) (Unit 3)

• Simulation for Studying Point Estimation and Hypothesis Testing (Unit 4)

4. Introduction to Likelihood and Maximum Likelihood Estimation

• Wikipedia page explaining maximum likelihood estimation.

5. Univariate Optimization and Gradient Descent Algorithm

• Univariate Optimization(Unit 5)

• Multivariate Newton Methods (Unit 6)

6. Expectation-Maximization (EM) algorithm

• Wikipedia page explaining EM algorithm

• EM Algorithm (Unit 7)

7. Introduction to Bayesian Inference

8. Numerical Quadrature, Laplace approximation, BIC

• Animations for Demonstrating Numerical Integration

• Numerical Quadrature (Unit 8)

• Laplace Approximation (Unit 9)

9. Rejection Sampling (Unit 10)

10. Importance Sampling (Unit 11)

11. Introduction to MCMC Convergence (Unit 12)

12. Gibbs Sampling and Metropolis-Hastings Sampling

• Gibbs sampling (Unit 13)

• An Animation for Illustrating Metropolis-Hasting Sampling

• Metropolis-Hastings Sampling (Unit 14)

13. Use of BUGS/JAGS/STAN

• JAGS sampling (Unit 15)

• STAN sampling for a stochastic volatility model (Unit 16)

Back to Longhai Li's teaching page