| Date | Academic Week | Topic | Task |
|---|---|---|---|
| Jan 04 | 1 | 1. Introduction: Likelihood Function, and MLE | Course Starts (Jan 06) |
| Jan 11 | 2 | 2. Likelihood Theory: Sufficient Statistic | |
| Jan 18 | 3 | 3. Likelihood Theory: Bartlett's Identities, CR Lower Bound | |
| Jan 25 | 4 | 3. Likelihood Theory: Exponential Families | |
| Feb 01 | 5 | 4. Maximum Likelihood Estimation: Score, Fisher Information, Newton-Raphson Methods | |
| Feb 08 | 6 | 4. Maximum Likelihood Estimation: Asymptotics of MLE, AIC, Deep Learning | Assignment 1 due |
| Feb 15 | N/A | — | Reading Week – No classes |
| Feb 22 | 7 | 5. Hypothesis Testing: NP Lemma, Monotone Likelihood Test, Likelihood-based Tests | Midterm (during class) |
| Mar 01 | 8 | 6. Uniformly Minimum Variance Unbiased Tests: Complete Statistic, UMVUE | |
| Mar 08 | 9 | 7. Decision Theory: Risk, Minimaxity Theorem | Assignment 2 due |
| Mar 15 | 10 | 7. Bayesian Inference: Posterior, Bayes Rules, Bayes Risk | |
| Mar 22 | 11 | 7. Bayesian Inference: Minimax Rules, James-Stein Estimator | |
| Mar 29 | 12 | 7. Bayesian Inference: Empirical Bayes, Hierarchical Bayesian, MCMC, Case Study | |
| Apr 05 | 13 | Review | Assignment 3 due Course Ends (Apr 08) |
STAT 850/442 Statistical Inference (Univ. of Saskatchewan, 2027 Winter)
Description
This course presents a rigorous theoretical treatment of statistical inference, offering a comparative analysis of frequentist and Bayesian paradigms. The curriculum explores several core areas of statistical theory, including Likelihood theory (Sufficient Statistic, Bartlett’s Identities, Cramér-Rao Lower Bound, Exponential Families) and the mechanics of Maximum Likelihood Estimation (Score, Fisher Information, Newton-Raphson Methods, Asymptotics of Maximum Likelihood Estimators, Akaike Information Criteria, Deep Learning). The course also covers hypothesis testing and optimal point estimation through the lens of the Likelihood ratio test (Neyman-Pearson Lemma, Monotone Likelihood Test, Likelihood-based Tests) and UMVUE (Complete Statistic, Uniformly Minimum Variance Unbiased Estimators/Tests). Additionally, the syllabus addresses foundational concepts in Decision theory (Risk Function, Minimaxity Theorem) and provides a comprehensive treatment of Bayesian inference (Posterior, Bayes Rules, Bayes Risk, Minimax Rules, James-Stein Estimator, Empirical Bayes, Hierarchical Bayesian, MCMC, Case Study).
Prerequisite(s): STAT 342.
This course requires a strong command of multivariate calculus, alongside a rigorous foundation in intermediate probability theory including asymptotic theorey for probability. Students should also possess prior exposure to applied statistical methods and familiar with basic statistical concepts such as standard error, p-value, and confidence internal.
Instructor
- Longhai Li, Professor
- Department of Mathematics and Statistics, University of Saskatchewan
- Email: longhai.li@usask.ca.
Times and Places
- Lectures: TTH 11:30-12:50, MCLN 42.1
- Office Hours: TBA with Students
- No lab
Textbook and Course Materials
The course page contains the links to my lecture notes and assingments.
Recommended Text:
Young G. A. & R. L. Smith, Essentials of Statistical Inference, Cambridge University Press, 2005.
Tentative Schedule
Important
The schedule may change depending on the course pace. The exact assignment and test dates are given on Canvas.
Learning Outcomes
After completing this course, students are expected to grasp the following knowledges and skills:
| Topic | Knowledge | Skills | Perc |
|---|---|---|---|
| Likelihood Theory | Define minimal sufficient statistic. Understand Bartlett's Identities, the Cramér-Rao Lower Bound (CRLB), and the mathematical properties of Exponential Families. | Identify minimal sufficient statistic, compute Fisher Information matrices, Bartlett Identities, verify CRLB attainment, identify natural parameters and calculate moments of suff. statistics in Exponential Families | 10% |
| Maximum Likelihood Estimation | Define the score function and Fisher Information. Understand the Newton-Raphson algorithm, MLE asymptotic normality, and information criteria (AIC). | Derive MLEs analytically, calculate asymptotic variances using Fisher information, find the asymptotic distributions of estimators, compute AIC. | 30% |
| Hypothesis Testing | Grasp the size, power, Neyman-Pearson Lemma, Monotone Likelihood Ratios (MLR), and the theoretical logic of likelihood-based testing. | Construct UMP tests using MLR, derive Likelihood Ratio tests, Score tests, and Wald tests with specified rejection regions. | 20% |
| UMVUE | Understand completeness, the Lehmann-Scheffé theorem, and Rao-Blackwell Theorem. | Derive UMVUE in exponential families. | 10% |
| Decision Theory | Understand risk functions, admissibility, and the theoretical foundation of decision rules. | Calculate frequentist risk function for given estimators and evaluate admissibility under simple loss functions. | 10% |
| Bayesian Inference | Define prior/posterior distributions and Bayes risk. Understand shrinkage via the James-Stein estimator, Empirical Bayes, hierarchical models, and MCMC principles. | Derive posterior distributions, find Bayes estimators, calculate Bayes risk, and find minimax rules. | 20% |
Evaluation
Grading Scheme
3 Assignments: 3 x 10%, 1 Term Test: 20%, 1 Final Exam: 50%.
Assignments and Tests
Assignment questions are released in the one-drive folder. You will submit your solutions via Canvas. If you miss an assignment without proper excuse, the weight will NOT be shifted to the final. Undergraduate students will be assigned with different assignments and tests.
Assignments
- I will accept late assignments only for three (3) days beyond the due date. The penalty for your delay is 10 percentage points per day of lateness from the value of the assignment (including weekends). Extensions are only granted in rare instances (notably as a result of family or medical emergencies) and upon receipt of adequate documentation/proof.
- Answer the questions in the order they appear in the assignment. Neatness is important.
- Solutions to problems are to be included. Hence, simple answers without work will receive few (or no!) marks.
- Most problems in statistics have a “real-life” basis. Hence, solutions should include not only numerical solutions but also a statement as to what the numbers say about the problem.
- The work handed in must not be an exact duplicate of others.
- Submitting Assignments: The assignment can be typed and/or handwritten. Save your assignment as one PDF file (for handwritten assignments, feel free to take a picture/scan of your work and save it as one PDF file). Upload the PDF file as an assignment submission in Canvas.
- More details will be provided ahead of each assignment.
- Due Date: See Course Schedule.
Midterm
- The midterm is given in class period.
- Midterms must be written on the dates scheduled. Students must do midterms completely on their own. More details (including syllabus) will be provided ahead of each midterm.
- Type: Short-answer questions, problem-solving, open-book.
- Calculator: A scientific calculator is allowed.
- Make-up exam will not be given. If you miss an exam for a legitimate reason (e.g., illness, emergency) and notify me within 48 hours of the scheduled exam, the weight of the missed exam will be transferred to the final exam.
Final Exam
- Scheduling: Final examinations may be scheduled at any time during the examination period; students should therefore avoid making prior travel, employment, or other commitments for this period. If a student is unable to write an exam through no fault of their own for medical or other valid reasons, documentation must be provided and an opportunity to write the missed exam may be given. Students are encouraged to review all examination policies and procedures: http://students.usask.ca/academics/exams.php.
- The final exam will cover material of the entire course. More details will be provided ahead of the exam.
- Length: 3-hour in-person exam.
- Type: Short-answer questions, problem-solving, open-book.
Criteria That Must Be Met to Pass
The final exam is a required component of the course. Students must complete the final exam in order to be eligible to receive a passing grade in this class.
Attendance Expectation
Attendance is highly correlated with student performance. While a syllabus and suggested readings are provided, it is not an adequate substitute for attending class. Your attendance is highly recommended but not required, and you will not be graded on your attendance.
Recording of the Course
Recording of the lectures will only be allowed in certain circumstances. Please see the instructor for information on how to receive approval. In general, there will be no videos available for in-person lectures. Therefore, attendance is strongly recommended.
Use of Generative AI and Electronic Devices
AI for Learning vs. Assessment. Students are free (and encouraged) to use Generative AI tools as a study aid to understand course concepts, debug code, or explain complex theorems. However, all submitted work for assignments must be your own. You must write your own solutions. Directly copying text, derivations, or code from an AI tool and submitting it as your own may receive a severe penalty (up to receiving a 0% on the assignment.
Electronic Devices During Tests. All term tests and the final exam are Open Book, meaning you may bring printed notes, textbooks, and lecture slides.
No Electronic Devices: You are NOT allowed to use laptops, tablets, smartwatches, or any other electronic devices during the exam.
Phone Exception: You are permitted to bring a smartphone, but it must remain stowed away during the writing period. It may only be used at the very end of the exam for the specific purpose of taking photos of your answer sheets for submission (if required). Using the phone for any other reason during the exam will be treated as academic misconduct.