tiny_cnn
Neural network primitives, one line at a time. No black boxes. No magic.
What It Is
Every beginner asks: what really happens inside model.fit()?
tiny_cnn is my answer to that question. It's a from-scratch implementation of neural network fundamentals — neurons, layers, activation functions, loss functions, optimizers, and eventually convolutions — all built using only Python and NumPy. No TensorFlow. No PyTorch. Just the raw mathematics, written out and tested.
The goal is not to compete with frameworks. The goal is understanding. Every component in tiny_cnn exists because I wrote it, debugged it, and can explain why it works. This is learning in public — with all the failed experiments, the shape-mismatch errors, and the quiet victories that come with building something real.
Built So Far
Dense Layer
Fully connected layer with forward and backward pass. Weight initialisation, bias handling, and gradient computation from scratch.
ReLU / Sigmoid / Tanh
Activation functions with their derivatives. Essential for introducing non-linearity and enabling backpropagation.
Mean Squared Error
Loss function for regression tasks. Includes gradient computation for the backward pass through the loss.
Cross-Entropy Loss
Loss function for classification tasks. Softmax activation fused with cross-entropy for numerical stability.
Stochastic Gradient Descent
SGD optimizer with configurable learning rate. Parameter updates computed from accumulated gradients.
Sequential Model
Container for stacking layers. Handles forward pass through all layers, backpropagation, and parameter updates.
More components coming soon: convolutional layers, pooling, batch normalisation, dropout, and advanced optimisers.
The Philosophy
There's a difference between using a neural network and understanding one. Most tutorials show you how to import a library, call .fit(), and read the accuracy. Fewer show you what actually happens inside that call — how gradients flow backward, how weights update, why certain shapes need to match.
tiny_cnn exists for the same reason someone might build a car engine from scratch instead of just driving one: because taking things apart is the best way to learn how they work. Every component is written clearly, tested independently, and documented with the intuition behind it.
"Do not just use neural networks. Understand them."
— Inspired by the philosophy of building from first principles
Roadmap
Core building blocks
Dense layer, activation functions, loss functions, SGD optimizer, sequential model container.
Convolutional operations
Conv2D layer with forward and backward pass. Kernel initialisation, padding, stride, and feature map computation.
Pooling and normalisation
Max pooling, average pooling, batch normalisation, and dropout for regularisation.
Training examples & visualisations
End-to-end training scripts with loss curves, gradient flow visualisations, and comparison with framework implementations.
Active project — new components added regularly. Want to contribute, discuss, or build something together? Reach out.