Modern AI Canon

Concept implementations from 14 seminal AI papers. Each notebook distills the core idea into minimal PyTorch code with shape annotations, verification cells, and gradient flow checks. These are not full paper replications - just the key concepts made concrete.

Notebooks

#	Paper	Year	Concept	Notebook
01	Deep Residual Learning for Image Recognition	2016	Residual blocks, skip connections	01_ResNet
02	Attention Is All You Need	2017	Multi-head attention, positional encoding	02_Attention
03	BERT: Pre-training of Deep Bidirectional Transformers	2019	Masked language modeling	03_BERT
04	An Image is Worth 16x16 Words	2021	Patch embeddings, ViT	04_ViT
05	Generative Adversarial Networks	2014	Generator-discriminator minimax game	05_GAN
06	Auto-Encoding Variational Bayes	2014	VAE, reparameterization trick	06_VAE
07	Language Models are Few-Shot Learners	2020	Causal attention, autoregressive generation	07_GPT3
08	High-Resolution Image Synthesis with Latent Diffusion Models	2022	Noise schedule, forward/reverse diffusion	08_Diffusion
09	Retrieval-Augmented Generation	2020	Dense retrieval, marginalized generation	09_RAG
10	Chain-of-Thought Prompting	2022	CoT prompting, self-consistency	10_CoT
11	Training LMs to Follow Instructions with Human Feedback	2022	Reward model, RLHF, KL penalty	11_RLHF
12	Parameter-Efficient Transfer Learning for NLP	2019	Bottleneck adapters, selective fine-tuning	12_Adapters
13	LoRA: Low-Rank Adaptation of Large Language Models	2021	Low-rank weight updates, weight merging	13_LoRA
14	FlashAttention	2022	Tiled attention, online softmax trick	14_FlashAttention

Citations

Each notebook follows similar layout

1. Markdown header - paper citation, core equations, insight
2. Implementation - minimal PyTorch code with shape annotations
3. Verification - shape checks, forward pass sanity tests
4. Gradient flow - confirm backprop works end-to-end
5. Extra cells - parameter counts, comparisons, edge cases

Setup

pip install torch

All notebooks use only PyTorch and the Python standard library.

Note

Contributions are welcome. Please submit a pull request if you find any issues or inaccuracies.