How I Built Mini-GPT from Scratch
Everyone uses ChatGPT. Few people understand how it actually works. I built a decoder-only Transformer language model from scratch in PyTorch — no nn.TransformerDecoder, no Hugging Face — to demystify the architecture behind GPT-2. Here is every layer, every matrix multiply, every design decision.