Module 02: Deep Learning

Prerequisites

• Complete the Foundations capstone.
• Understand backpropagation, minibatches, and train/validation splits.

Outcomes

• Relate architecture and inductive bias to learned representations.
• Diagnose optimization, generalization, and data-pipeline failures separately.
• Build a reproducible training loop with meaningful baselines.

Ordered free resources

1. Dive into Deep Learning (d2l) — study multilayer networks, regularization, and optimization.
2. MIT Introduction to Deep Learning (mit-intro-dl) — connect modern architectures to common training issues.
3. PyTorch Tutorials (pytorch-tutorials) — implement datasets, modules, autograd, and checkpoints.
4. Zero to Hero (karpathy-zero-to-hero) — inspect representation learning at a human-manageable scale.

Checkpoints

• Overfit one minibatch, then explain why this is a useful diagnostic.
• Plot training and validation curves for a deliberately over-parameterized model.
• Introduce one data bug and identify it from measurements.

Self-tests

1. Why does evaluation mode matter for some layers?
2. How would you separate insufficient capacity from poor optimization?
3. Which data transformations must be fit only on training data?

Capstone

Train and diagnose a small sequence or image model. Provide a baseline, ablations for one regularizer and one optimizer setting, a data-quality check, and a concise error analysis.

Next: Transformers