| Date | Acad. Week | Topic | Remark |
|---|---|---|---|
| Sep 07 | 1 | 1 Introduction: Stat. Inference, R, R Studio, Quarto | Course Starts (Sep 08) |
| Sep 14 | 2 | 2 Computer Arithmetics: Overflow, Underflow, Rounding Error | |
| Sep 21 | 3 | 3 Monte Carlo Methods: RNG, Inverting CDF Sampling | |
| Sep 28 | 4 | 3 Monte Carlo Methods: Simulation for Estimation and Testing | |
| Oct 05 | 5 | 4 Maximum Likelihood Estimation: Univariate Optimization | Assignment 1 due |
| Oct 12 | 6 | 4 Maximum Likelihood Estimation: Multivariate Optimization | |
| Oct 19 | 7 | 4 Maximum Likelihood Estimation: EM Algorithm | |
| Oct 26 | 8 | 5 Bayesian Inference & MCMC: Intro Bayesian Inference and Numerical Quadrature | Assignment 2 due |
| Nov 02 | 9 | 5 Bayesian Inference & MCMC: Laplace Approx, Rejection Sampling | Midterm (during class) |
| Nov 09 | N/A | — | Reading Week – No classes |
| Nov 16 | 10 | 5 Bayesian Inference & MCMC: Importance Sampling | |
| Nov 23 | 11 | 5 Bayesian Inference & MCMC: Convergence, Gibbs Sampling | |
| Nov 30 | 12 | 5 Bayesian Inference & MCMC: Metropolis-Hastings Sampling | |
| Dec 07 | 13 | 5 Bayesian Inference & MCMC: General-purpose Samplers (JAGS, STAN) | Assignment 3 due Course Ends (Dec 07) |
STAT 812/420 Computational Statistics
Univ. of Saskatchewan, 2026-09
Description
This course covers the fundamental concepts in computational methods used the areas of statistics, machine learning, and data sciences. The topics include Introduction to R programming; Computer Arithmetics (Overflow, Underflow, Rounding Error); Monte Carlo Methods (RNG, Inverting CDF Sampling, Simulation for Estimation and Testing); Maximum Likelihood Estimation (Univariate Optimization, Multivariate Optimization, EM Algorithm); and Bayesian Inference & MCMC (Intro to Bayesian Inference, Numerical Quadrature, Laplace Approximation, Rejection Sampling, Importance Sampling, Convergence, Gibbs Sampling, Metropolis-Hastings Sampling, General-purpose Samplers like JAGS and STAN). After learning this course, students are expected to gain an understanding of the algorithms behind these statistical inferential methods, be able to develop new statistical methods, use computers to investigate the properties of statistical methods, and implement a combination of standard statistical toolkits for analyzing real data sets.
Prerequisites
- Multivariate calculus (MATH 225)
- Linear algebra (MATH 164)
- Calculus-based Probability (eg. STAT 342 or STAT 241)
- Multiple Linear Regression (eg. STAT 344)
Contact of Instructor
- Longhai Li, Professor
- Department of Mathematics and Statistics, University of Saskatchewan
- Email: longhai.li@usask.ca.
- Don’t send messages in Canvas, which I don’t check.
Times and Places
- Lectures: TTH 10:00-11:20, MCLN 42.1
- Office Hours: Thursday 1:00-2:00 PM, MCLN 219, or online via Zoom
- No lab
Textbook and Course Materials
The course page contains the links to my lecture notes and other materials. No textbooks are required.
Tentative Schedule / List of Topics
Important
The schedule may change depending on the course pace. The exact assignment and test dates are given on Canvas.
Learning Outcomes
After completing this course, students are expected to grasp the following knowledges and skills:
| Topic | Knowledge | Skills | Perc1 |
|---|---|---|---|
| Intro & Arithmetic | Understand R fundamentals, computer arithmetic limits, overflow/underflow, and numerical rounding errors. | Write modular R code and diagnose numerical stability issues in statistical computations. | 10% |
| Monte Carlo Methods | Understand random number generation, the inverse CDF method, and simulation strategies for evaluating statistical methods. | Implement sampling algorithms from scratch and design simulation studies for point estimation and hypothesis testing. | 20% |
| Optimization & MLE | Understand the mathematical principles of univariate and multivariate optimization techniques for likelihood functions. | Apply Newton-Raphson and other multivariate optimization techniques computationally to find Maximum Likelihood Estimates. | 25% |
| EM Algorithm | Understand the theoretical framework of the Expectation-Maximization algorithm for latent variable models. | Implement the EM algorithm to solve problems involving missing data or hidden states. | 15% |
| Bayesian & MCMC | Grasp the concepts of numerical quadrature, rejection/importance sampling, and MCMC theory (Gibbs, Metropolis-Hastings). | Simulate from complex posterior distributions using custom MCMC algorithms and general-purpose samplers like JAGS and STAN. | 30% |
| 1 Percentage of the final exam that covers each topic. | |||
Computing
We will use RStudio and R for this course.
Personal Computer: Download R, RStudio/Positron/VS-code to your local machine.
USASK vlab: If you don’t have a personal computer, you can use the USask remote desktop, the browser-based vlab (https://vlab.usask.ca/),
Posit Cloud: (https://posit.cloud/).
GitHub Codespaces: You can also run R and RStudio in the cloud with a GitHub Codespace, without installing anything locally. See https://github.com/codespaces.
Google Colab: You can also run R in the cloud using Google Colaboratory. To open a notebook with R pre-configured, use this direct link: https://colab.research.google.com/#create=true&language=r. Alternatively, you can create a new notebook in Colab and change the runtime type to R (Runtime > Change runtime type > R).
Evaluation
Grading Scheme
3 Assignments: 3 x 10%, 1 Term Test: 20%, 1 Final Exam: 50%.
Assignments and Tests
Assignment questions are released in the one-drive folder. You will submit your solutions via Canvas. If you miss an assignment without proper excuse, the weight will NOT be shifted to the final. Undergraduate students will be assigned with different assignments and tests.
Assignments
- I will accept late assignments only for three (3) days beyond the due date. The penalty for your delay is 10 percentage points per day of lateness from the value of the assignment (including weekends). Extensions are only granted in rare instances (notably as a result of family or medical emergencies) and upon receipt of adequate documentation/proof.
- Answer the questions in the order they appear in the assignment. Neatness is important.
- Solutions to problems are to be included. Hence, simple answers without work will receive few (or no!) marks.
- Most problems in statistics have a “real-life” basis. Hence, solutions should include not only numerical solutions but also a statement as to what the numbers say about the problem.
- The work handed in must not be an exact duplicate of others.
- Submitting Assignments: The assignment can be typed and/or handwritten. Save your assignment as one PDF file (for handwritten assignments, feel free to take a picture/scan of your work and save it as one PDF file). Upload the PDF file as an assignment submission in Canvas.
- More details will be provided ahead of each assignment.
- Due Date: See Course Schedule.
Midterm
- The midterm is given in class period.
- Midterms must be written on the dates scheduled. Students must do midterms completely on their own. More details (including syllabus) will be provided ahead of each midterm.
- Type: Short-answer questions, problem-solving, open-book.
- Calculator: A scientific calculator is allowed.
- Make-up exam will not be given. If you miss an exam for a legitimate reason (e.g., illness, emergency) and notify me within 48 hours of the scheduled exam, the weight of the missed exam will be transferred to the final exam.
Final Exam
- Scheduling: Final examinations may be scheduled at any time during the examination period; students should therefore avoid making prior travel, employment, or other commitments for this period. If a student is unable to write an exam through no fault of their own for medical or other valid reasons, documentation must be provided and an opportunity to write the missed exam may be given. Students are encouraged to review all examination policies and procedures: http://students.usask.ca/academics/exams.php.
- The final exam will cover material of the entire course. More details will be provided ahead of the exam.
- Length: 3-hour in-person exam.
- Type: Short-answer questions, problem-solving, open-book.
Criteria That Must Be Met to Pass
The final exam is a required component of the course. Students must complete the final exam in order to be eligible to receive a passing grade in this class.
Attendance Expectation
Attendance is highly correlated with student performance. While a syllabus and suggested readings are provided, it is not an adequate substitute for attending class. Your attendance is highly recommended but not required, and you will not be graded on your attendance.
Recording of the Course
Recording of the lectures will only be allowed in certain circumstances. Please see the instructor for information on how to receive approval. In general, there will be no videos available for in-person lectures. Therefore, attendance is strongly recommended.
Use of Generative AI and Electronic Devices
- AI for Learning vs. Assessment. Students are free (and encouraged) to use Generative AI tools as a study aid to understand course concepts, debug code, or explain complex theorems. However, all submitted work for assignments must be your own. You must write your own solutions. Directly copying text, derivations, or code from an AI tool and submitting it as your own may receive a severe penalty (up to receiving a 0% on the assignment).
- Electronic Devices. All term tests and the final exam are Open Book, meaning you may bring printed notes, textbooks, and lecture slides.
- No Electronic Devices: You are NOT allowed to use laptops, tablets, smartwatches, or any other electronic devices during the exam.
- Phone Exception: You are permitted to bring a smartphone, but it must remain stowed away during the writing period. It may only be used at the very end of the exam for the specific purpose of taking photos of your answer sheets for submission (if required). Using the phone for any other reason during the exam will be treated as academic misconduct.