Module 01: Foundations

Prerequisites

• Write and debug short Python programs.
• Manipulate equations with vectors, sums, and logarithms.

Tensor, in plain language: a multidimensional array of numbers used to represent data, parameters, and intermediate activations.

Outcomes

• Track shapes, dtypes, devices, and memory through tensor operations.
• Explain gradient descent, stochastic estimates, loss surfaces, and regularization.
• Use probability and measurement to form testable hypotheses.

Ordered free resources

1. MIT OpenCourseWare 6.036 Introduction to Machine Learning · Tier A · Course / video / exercises — strengthen statistical and optimization intuition.
2. Dive into Deep Learning · Tier A · Book / course / labs — work through linear algebra, probability, and optimization interactively.
3. MIT Introduction to Deep Learning · Tier A · Course / video / labs — establish the vocabulary and overall training loop.
4. Practical AI · Tier C · Podcast / interviews — connect foundational ideas to applied decisions and questions worth verifying.

Checkpoints

• Annotate every intermediate shape in a two-layer network.
• Estimate parameter and activation bytes before running it.
• Compare analytical and finite-difference gradients on one scalar function.

Self-tests

1. Why can a numerically decreasing loss still produce a poor model?
2. Where do batch size and dtype enter a memory estimate?
3. What result would falsify your current optimization hypothesis?

Capstone

Implement a tiny classifier from tensor primitives. Record the shape and memory ledger, verify gradients numerically, compare two learning rates, and explain one failure mode without relying on framework jargon.

Next: Deep Learning