| Topic | Learning Outcomes (Knowledge & Skills) | Perc |
|---|---|---|
| Vector Spaces | Understanding the geometric interpretation of subspaces, orthogonal projections with the Hat matrix (H) and the Partial Residual matrix. Numerically and symbolically calculating orthogonal projections onto these spaces using various approaches | 15% |
| Matrix Algebra | Understanding advanced matrix theory, including spectral decomposition and idempotent properties. Deriving properties of projection matrices, symbolically manipulating matrices, including calculating generalized inverses. | 10% |
| Multivariate Normal | Understanding the properties of multivariate normal vectors and multiple correlation coefficients. Deriving the mean and covariance of transformed normal vectors and calculating conditional and marginal distribution of MVN. | 10% |
| Quadratic Forms | Understanding the distributions of quadratic forms. Verifying the independence of quadratic forms and applying Cochran’s Theorem to derive the distributions and degrees of freedom for sums of squares. | 10% |
| Multiple Regression | Understanding the general linear hypothesis and the relationship between RSS, SSH, and SSE. Calculating LS estimators, error variance, and adjusted \(R^2\) with CI. Implementing F-tests and t-tests; constructing confidence/prediction regions; and symbolically deriving SSH, SSE, and the F-distribution. | 40% |
| Non-full-rank Models | Understanding identifiability and estimable functions in over-parameterized models. Identifying estimable linear combinations and conducting hypothesis tests and estimation for non-full rank linear models | 15% |
STAT 443/851 Theory of Linear Models (University of Saskatchewan, 2026-01)
Description
This course is a rigorous examination of the general linear models using vector space theory, in particular the approach of regarding least square as projection. The topics includes: vector space; projection; matrix algebra; generalized inverses; quadratic forms; theory for point estimation; theory for hypothesis test; theory for non-full-rank models.
Prerequisite(s): MATH 164 (formerly MATH 264) or MATH 266, STAT 342, and STAT 344 or 345.
Instructor
Longhai Li, Professor,
Department of Mathematics and Statistics, University of Saskatchewan
Email: longhai.li@usask.ca.
Times and Places
Lecture Classroom: MCLN 242.1, MWF 9:30-10:20; Office hour: TBA; Lab: no lab.
Course Materials
- See the course page, which is the primary source for learning.
- Recommended reading: LINEAR MODELS IN STATISTICS, Second Edition, by Alvin C. Rencher and G. Bruce Schaalje, ISBN 978-0-471-75498-5 (cloth). The book is not required but it is good to have it.
Learning Outcomes
After completing this course, students are expected to grasp the following knowledges and skills:
Tentative Schedule
| Date (Mon) | Week | Lecture Topic | Tests & Notes |
|---|---|---|---|
| Jan 05 | 1 | 1. Vector Space and Projection | Jan 07: First day of class |
| Jan 12 | 2 | 1. Vector Space and Projection | |
| Jan 19 | 3 | 2. Matrix Algebra | Jan 25: Assignment 1 due |
| Jan 26 | 4 | 2. Matrix Algebra | |
| Feb 02 | 5 | 3. Distribution of Multivariate Normal | |
| Feb 09 | 6 | 4. Distribution of Quadratic Forms | |
| Feb 16 | N/A | — | Reading Week – No classes |
| Feb 23 | 7 | 4. Distribution of Quadratic Forms | Mar 01: Assignment 2 due |
| Mar 02 | 8 | 5. Theory for Multiple Regression | Mar 06: Midterm (during class) |
| Mar 09 | 9 | 5. Theory for Multiple Regression | |
| Mar 16 | 10 | 5. Theory for Multiple Regression | |
| Mar 23 | 11 | 6. Non-full-rank Models | |
| Mar 30 | 12 | 6. Non-full-rank Models | Apr 05: Assignment 3 due |
| Apr 06 | 13 | Review | Apr 06: Last lecture |
Important
The schedule may change depending on the course pace. The exact assignment and test dates are given on Canvas.
Evaluation
Grading Scheme
3 assignments: 3x10% = 30%, 1 term test: 20%, final exam: 50%.
Assignments and Tests
Assignment questions are released in the one-drive folder. You will submit your solutions via Canvas. If you miss an assignment without proper excuse, the weight will NOT be shifted to the final. Undergraduate students will be assigned with different assignments and tests.
Assignments
- I will accept late assignments only for three (3) days beyond the due date. The penalty for your delay is 10 percentage points per day of lateness from the value of the assignment (including weekends). Extensions are only granted in rare instances (notably as a result of family or medical emergencies) and upon receipt of adequate documentation/proof.
- Answer the questions in the order they appear in the assignment. Neatness is important.
- Solutions to problems are to be included. Hence, simple answers without work will receive few (or no!) marks.
- Most problems in statistics have a “real-life” basis. Hence, solutions should include not only numerical solutions but also a statement as to what the numbers say about the problem.
- The work handed in must not be an exact duplicate of others.
- Submitting Assignments: The assignment can be typed and/or handwritten. Save your assignment as one PDF file (for handwritten assignments, feel free to take a picture/scan of your work and save it as one PDF file). Upload the PDF file as an assignment submission in Canvas.
- More details will be provided ahead of each assignment.
- Due Date: See Course Schedule.
Midterm
- The midterm is given in class period.
- Midterms must be written on the dates scheduled. Students must do midterms completely on their own. More details (including syllabus) will be provided ahead of each midterm.
- Type: Short-answer questions, problem-solving, open-book.
- Calculator: A scientific calculator is allowed.
- Make-up exam will not be given. If you miss an exam for a legitimate reason (e.g., illness, emergency) and notify me within 48 hours of the scheduled exam, the weight of the missed exam will be transferred to the final exam.
Final Exam
- Scheduling: Final examinations may be scheduled at any time during the examination period; students should therefore avoid making prior travel, employment, or other commitments for this period. If a student is unable to write an exam through no fault of their own for medical or other valid reasons, documentation must be provided and an opportunity to write the missed exam may be given. Students are encouraged to review all examination policies and procedures: http://students.usask.ca/academics/exams.php.
- The final exam will cover material of the entire course. More details will be provided ahead of the exam.
- Length: 3-hour in-person exam.
- Type: Short-answer questions, problem-solving, open-book.
Criteria That Must Be Met to Pass
The final exam is a required component of the course. Students must complete the final exam in order to be eligible to receive a passing grade in this class.
Attendance Expectation
Attendance is highly correlated with student performance. While a syllabus and suggested readings are provided, it is not an adequate substitute for attending class. Your attendance is highly recommended but not required, and you will not be graded on your attendance.
Recording of the Course
Recording of the lectures will only be allowed in certain circumstances. Please see the instructor for information on how to receive approval. In general, there will be no videos available for in-person lectures. Therefore, attendance is strongly recommended.
Use of Generative AI and Electronic Devices
AI for Learning vs. Assessment. Students are free (and encouraged) to use Generative AI tools as a study aid to understand course concepts, debug code, or explain complex theorems. However, all submitted work for assignments must be your own. You must write your own solutions. Directly copying text, derivations, or code from an AI tool and submitting it as your own may receive a severe penalty (up to receiving a 0% on the assignment.
Electronic Devices During Tests. All term tests and the final exam are Open Book, meaning you may bring printed notes, textbooks, and lecture slides.
No Electronic Devices: You are NOT allowed to use laptops, tablets, smartwatches, or any other electronic devices during the exam.
Phone Exception: You are permitted to bring a smartphone, but it must remain stowed away during the writing period. It may only be used at the very end of the exam for the specific purpose of taking photos of your answer sheets for submission (if required). Using the phone for any other reason during the exam will be treated as academic misconduct.