C1 · Objectives and probabilistic modelling
Lab
Requires Python 3.12 or newer, check-shapes, numpy, pydantic.
Extract the archive, then inside that folder:
uv run python verify.py
Assessment
C1 exit assessment: objective interpretations and EM claims
Time: 70 minutes, then a 5 minute live challenge.
Submission
1. Forward derivation, C1.1
From the conditionally independent heteroscedastic Gaussian model declared in the lab guide, derive the negative log-likelihood up to parameter-independent terms. List support, conditional distribution, parameterization, independence, known positive scales, and measure assumptions. Verify the expression numerically against the supplied implementation.
2. Non-identifying reverse reading, C1.2
For the squared-error objective J(theta) declared in the lab guide, give at
least two compatible interpretations drawn from likelihood, decision theory,
or surrogate-risk minimization. State the additional assumptions required to
select a likelihood interpretation. Do not claim that the objective alone
identifies a data-generating distribution.
3. Object classification, C1.3
For the four Gaussian-process expressions A through D in the lab guide, state what is fixed, what varies, the domain, whether normalization is required, and which measure is involved. Classify data density, likelihood, posterior, and marginal likelihood. Explain why B and D share an algebraic expression while their statistical role changes with the argument being varied.
4. EM audit, C1.4
Run and inspect the seeded EM implementation. Repair the trace check if needed, compare all three declared initializations, and separate:
- exact E and M step semantics;
- monotone non-decrease of observed-data likelihood;
- convergence under stated conditions;
- stationarity or local optimum;
- global maximum, which is not guaranteed.
Live challenge
The instructor changes an independence assumption, reveals a new decision target, changes what is integrated, or introduces an inexact M-step. Revise the smallest affected claim and run the decisive check.
References
- Casella & Berger, Statistical Inference, 2nd ed., section 6.3, 2002
- Bishop, Pattern Recognition and Machine Learning, sections 1.2.5, 1.5.5, 3.1.1, 3.1.4, 3.1.5, 4.3.2, and 4.3.4, 2006
- Miller, Goodman & Smyth, On loss functions which minimize to conditional expected values and posterior probabilities, IEEE Transactions on Information Theory, 1993
- Dempster, Laird & Rubin, Maximum Likelihood from Incomplete Data via the EM Algorithm, JRSS-B, 1977
- Wu, On the Convergence Properties of the EM Algorithm, Annals of Statistics, 1983
- Rasmussen & Williams, Gaussian Processes for Machine Learning, chapters 2, 4, and 5, 2006