Module 01: Foundations

Prerequisites

• Write and debug short Python programs.
• Manipulate equations with vectors, sums, and logarithms.

Outcomes

• Track shapes, dtypes, devices, and memory through tensor operations.
• Explain gradient descent, stochastic estimates, loss surfaces, and regularization.
• Use probability and measurement to form testable hypotheses.

Ordered free resources

1. MIT Introduction to Deep Learning (mit-intro-dl) — establish the vocabulary and overall loop.
2. Dive into Deep Learning (d2l) — work through linear algebra, probability, and optimization interactively.
3. MIT OpenCourseWare Machine Learning (mit-ocw-ml) — strengthen statistical and optimization intuition.
4. PyTorch Tutorials (pytorch-tutorials) — translate equations into tensor operations.
5. Zero to Hero (karpathy-zero-to-hero) — observe gradients and representations in compact models.

Checkpoints

• Annotate every intermediate shape in a two-layer network.
• Estimate parameter and activation bytes before running it.
• Compare analytical and finite-difference gradients on one scalar function.

Self-tests

1. Why can a numerically decreasing loss still produce a poor model?
2. Where do batch size and dtype enter a memory estimate?
3. What result would falsify your current optimization hypothesis?

Capstone

Implement a tiny classifier from tensor primitives. Record the shape and memory ledger, verify gradients numerically, compare two learning rates, and explain one failure mode without relying on framework jargon.

Next: Deep Learning