Python

Gemma 4 12B is celebrated as the most complete local model. For Python I ran three variants against each other on an M3 Max, with real code execution, including a parser that defeats all three.

Eight articles of theory, one bonus chapter of practice. We combine all the code fragments from the LLM series into a working mini language model, train it on TinyShakespeare, and tack on a tiny SFT step. With a wink to Star Trek TNG.

How a base model that completes text becomes a helpful assistant. Supervised Fine-Tuning, RLHF, DPO, and Constitutional AI — the last piece of the LLM pipeline, with the candid question of what alignment actually solves.

Position, depth, stability — what gets added on top of attention to make a complete transformer. Positional encodings, feed-forward layers, residual connections, layer normalization, the entire block in Python, and the leap to GPT, BERT, Llama, and Claude.

How the bottleneck of RNNs was replaced by a mechanism that links every token to every other. Query, Key, Value, Multi-Head, and the paper that flipped the NLP world, explained with library metaphors and just enough math for the curious.

Why language needs memory, and how the first language models learned that context. Recurrent networks, LSTMs, and the long-sentence problem, explained without a formula storm.

How neural networks learn from errors. Loss, gradients, chain rule, gradient descent — backpropagation implemented by hand on a 2-layer MLP that learns XOR and token prediction.

What happens between embedding and logit. Neurons, layers, forward pass, and activation functions — fully implemented in numpy, no framework magic.

How language models encode meaning in numbers. Embedding tables, cosine similarity, vector arithmetic, and why King minus Man plus Woman equals Queen.

What happens between input and output? Tokens, probability distributions, and sampling strategies — explained step by step, with real code.