Applied AI engineering — in-depth explorations of software, deep learning, and the systems that put them to work.
From derivatives to backpropagation: the theory behind Micrograd
Build from derivatives to the chain rule and backpropagation, then map each rule onto Micrograd’s forward values and accumulated gradients.
Explores
- Derivatives and partial derivatives
- Chain rule on a computation graph
- Backpropagation and gradient accumulation
- Micrograd’s forward and backward pass
Vector decomposition and subspaces: what a matrix reads and erases
Break activation vectors into components and see which input directions a matrix keeps, which it erases, and why different inputs can give the same output.
Explores
- Decomposing activation vectors
- The four fundamental subspaces
- Null space and vanishing directions
- Identical outputs from distinct inputs
What a weight matrix does and why we need nonlinearity
See a weight matrix as a transformation of the input space, then find out why stacked linear layers buy nothing new and what activations add.
Explores
- Matrices as rotations, scalings, projections
- Why linear layers collapse to one
- What activation functions make possible
A deep-dive on how to assemble context for an LLM agent
Compare how coding agents assemble context, and measure how those choices affect cost, cache reuse, and task completion.
Explores
- Full history, recent turns, and summaries
- Prompt caching and tool output design
- Context strategies in real coding agents
- Benchmarks and failure modes
Understanding CNNs — convolutions, feature maps, and pooling
Follow how convolutions turn pixels into feature maps, and how deeper layers combine simple patterns into richer representations.
Explores
- Convolution filters and feature maps
- Stride, padding, and receptive fields
- Pooling and spatial dimensions
- Combining features across layers
How neural networks learn: backpropagation and gradient descent
Follow how a neural network learns: measure prediction errors, trace gradients backward, and use them to update weights with interactive examples.
Explores
- Loss functions and parameter sensitivity
- The chain rule through network layers
- Gradient descent and learning rates
- Stochastic and mini-batch updates
Recent
All posts →How neurons became linear algebra
How to build a decision tree regressor from scratch
How to build a decision tree classifier from scratch
Negative sampling and contrastive learning — from word2vec to CLIP
Word embeddings — word2vec and vectors that mean something
Building Model Armor: multi-layer safety filtering for LLMs