| Date | Acad. Week | Topic | Remark |
|---|---|---|---|
| Aug 31 | 1 | 1 Introduction: Stat. Inference, R, R Studio, Quarto | Course Starts (Sep 02) |
| Sep 07 | 2 | 2 Computer Arithmetics: Overflow, Underflow, Rounding Error | |
| Sep 14 | 3 | 3 Monte Carlo Methods: RNG, Inverting CDF Sampling | |
| Sep 21 | 4 | 3 Monte Carlo Methods: Simulation for Estimation and Testing | |
| Sep 28 | 5 | 4 Maximum Likelihood Estimation: Univariate Optimization | Assignment 1 due |
| Oct 05 | 6 | 4 Maximum Likelihood Estimation: Multivariate Optimization | |
| Oct 12 | 7 | 4 Maximum Likelihood Estimation: EM Algorithm | |
| Oct 19 | 8 | 5 Bayesian Inference & MCMC: Intro Bayesian Inference and Numerical Quadrature | Assignment 2 due |
| Oct 26 | 9 | 5 Bayesian Inference & MCMC: Laplace Approx, Rejection Sampling | Midterm (during class) |
| Nov 02 | 10 | 5 Bayesian Inference & MCMC: Importance Sampling | |
| Nov 09 | N/A | — | Reading Week – No classes |
| Nov 16 | 11 | 5 Bayesian Inference & MCMC: Convergence, Gibbs Sampling | |
| Nov 23 | 12 | 5 Bayesian Inference & MCMC: Metropolis-Hastings Sampling | |
| Nov 30 | 13 | 5 Bayesian Inference & MCMC: General-purpose Samplers (JAGS, STAN) | Assignment 3 due |
| Dec 07 | 14 | TBD | Course Ends (Dec 07) |
STAT 812/420 Computational Statistics (Univ. of Saskatchewan, 2026-09)
Description
Computationally intensive methods have become widely used in statistical inference. The objective of this course is to teach students important computational techniques used in statistical inference (evaluation of statistical methods, MLE and Bayesian inference). After learning this course, students are expected to gain understanding of algorithms behind statistical inferential methods, be able to develop new statistical methods, be able to use computer to investigate the properties of statistical methods, and be able to implement a combination of standard statistical toolkits for analyzing real data sets.
Prerequisites
- Multivariate calculus (MATH 225)
- Linear algebra (MATH 164)
- Calculus-based Probability (eg. STAT 342 or STAT 241)
- Multiple Linear Regression (eg. STAT 344)
Instructor
- Longhai Li, Professor
- Department of Mathematics and Statistics, University of Saskatchewan
- Email: longhai.li@usask.ca.
Times and Places
- Lectures: TTH 10:00-11:20, MCLN 42.1
- Office Hours: TBA with Students
- No lab
Textbook and Course Materials
The course page contains the links to my lecture notes and assingments.
Recommended Text: My own lecture note.
Tentative Schedule / List of Topics
Important
The schedule may change depending on the course pace. The exact assignment and test dates are given on Canvas.
Learning Outcomes
After completing this course, students are expected to grasp the following knowledges and skills:
| Topic | Knowledge | Skills | Perc |
|---|---|---|---|
| Intro & Arithmetic | Understand R fundamentals, computer arithmetic limits, overflow/underflow, and numerical rounding errors. | Write modular R code and diagnose numerical stability issues in statistical computations. | 10% |
| Monte Carlo Methods | Understand random number generation, the inverse CDF method, and simulation strategies for evaluating statistical methods. | Implement sampling algorithms from scratch and design simulation studies for point estimation and hypothesis testing. | 20% |
| Optimization & MLE | Understand the mathematical principles of univariate and multivariate optimization techniques for likelihood functions. | Apply Newton-Raphson and other multivariate optimization techniques computationally to find Maximum Likelihood Estimates. | 25% |
| EM Algorithm | Understand the theoretical framework of the Expectation-Maximization algorithm for latent variable models. | Implement the EM algorithm to solve problems involving missing data or hidden states. | 15% |
| Bayesian & MCMC | Grasp the concepts of numerical quadrature, rejection/importance sampling, and MCMC theory (Gibbs, Metropolis-Hastings). | Simulate from complex posterior distributions using custom MCMC algorithms and general-purpose samplers like JAGS and STAN. | 30% |
Computing
We will use RStudio and R for this course.
Personal Computer: Download R, RStudio, Positron, VS-code to your local machine.
USASK vlab: If you don’t have a personal computer, you can use the USask remote desktop, the browser-based vlab (https://vlab.usask.ca/),
Posit Cloud: (https://posit.cloud/).
Google Colab: You can also run R in the cloud using Google Colaboratory. To open a notebook with R pre-configured, use this direct link: https://colab.research.google.com/#create=true&language=r. Alternatively, you can create a new notebook in Colab and change the runtime type to R (Runtime > Change runtime type > R).
Evaluation
Grading Scheme
3 Assignments: 3 x 10%, 1 Term Test: 20%, 1 Final Exam: 50%.
Assignments and Tests
Assignment questions are released in the one-drive folder. You will submit your solutions via Canvas. If you miss an assignment without proper excuse, the weight will NOT be shifted to the final. Undergraduate students will be assigned with different assignments and tests.
Assignments
- I will accept late assignments only for three (3) days beyond the due date. The penalty for your delay is 10 percentage points per day of lateness from the value of the assignment (including weekends). Extensions are only granted in rare instances (notably as a result of family or medical emergencies) and upon receipt of adequate documentation/proof.
- Answer the questions in the order they appear in the assignment. Neatness is important.
- Solutions to problems are to be included. Hence, simple answers without work will receive few (or no!) marks.
- Most problems in statistics have a “real-life” basis. Hence, solutions should include not only numerical solutions but also a statement as to what the numbers say about the problem.
- The work handed in must not be an exact duplicate of others.
- Submitting Assignments: The assignment can be typed and/or handwritten. Save your assignment as one PDF file (for handwritten assignments, feel free to take a picture/scan of your work and save it as one PDF file). Upload the PDF file as an assignment submission in Canvas.
- More details will be provided ahead of each assignment.
- Due Date: See Course Schedule.
Midterm
- The midterm is given in class period.
- Midterms must be written on the dates scheduled. Students must do midterms completely on their own. More details (including syllabus) will be provided ahead of each midterm.
- Type: Short-answer questions, problem-solving, open-book.
- Calculator: A scientific calculator is allowed.
- Make-up exam will not be given. If you miss an exam for a legitimate reason (e.g., illness, emergency) and notify me within 48 hours of the scheduled exam, the weight of the missed exam will be transferred to the final exam.
Final Exam
- Scheduling: Final examinations may be scheduled at any time during the examination period; students should therefore avoid making prior travel, employment, or other commitments for this period. If a student is unable to write an exam through no fault of their own for medical or other valid reasons, documentation must be provided and an opportunity to write the missed exam may be given. Students are encouraged to review all examination policies and procedures: http://students.usask.ca/academics/exams.php.
- The final exam will cover material of the entire course. More details will be provided ahead of the exam.
- Length: 3-hour in-person exam.
- Type: Short-answer questions, problem-solving, open-book.
Criteria That Must Be Met to Pass
The final exam is a required component of the course. Students must complete the final exam in order to be eligible to receive a passing grade in this class.
Attendance Expectation
Attendance is highly correlated with student performance. While a syllabus and suggested readings are provided, it is not an adequate substitute for attending class. Your attendance is highly recommended but not required, and you will not be graded on your attendance.
Recording of the Course
Recording of the lectures will only be allowed in certain circumstances. Please see the instructor for information on how to receive approval. In general, there will be no videos available for in-person lectures. Therefore, attendance is strongly recommended.
Use of Generative AI and Electronic Devices
- AI for Learning vs. Assessment. Students are free (and encouraged) to use Generative AI tools as a study aid to understand course concepts, debug code, or explain complex theorems. However, all submitted work for assignments must be your own. You must write your own solutions. Directly copying text, derivations, or code from an AI tool and submitting it as your own may receive a severe penalty (up to receiving a 0% on the assignment).
- Electronic Devices During Tests. All term tests and the final exam are Open Book, meaning you may bring printed notes, textbooks, and lecture slides.
- No Electronic Devices: You are NOT allowed to use laptops, tablets, smartwatches, or any other electronic devices during the exam.
- Phone Exception: You are permitted to bring a smartphone, but it must remain stowed away during the writing period. It may only be used at the very end of the exam for the specific purpose of taking photos of your answer sheets for submission (if required). Using the phone for any other reason during the exam will be treated as academic misconduct.