beginner · ~20 min
Your First Neural Net
Stack layers of neurons to classify points a single line can't separate, and watch the decision boundary bend as it trains.
The linear regression playground fit a single straight line. That works when a line can separate — or fit — your data. The four groups of points below can't be separated by any single line, no matter how you rotate it. To learn a bent boundary, we need to stack layers.
A single Dense layer computes activation(W·x + b) — still fundamentally a straight line (or
flat plane) unless activation is nonlinear. Stack a nonlinear activation function
like tanh between layers, and each hidden neuron
contributes its own bent piece to the final boundary — enough of them, combined, can wrap around
almost any shape.
Beginner tip
Think of each hidden neuron as its own tiny linear-regression line, each looking at the data from a slightly different angle. The output layer then combines their opinions into one decision.
🔍 Deep dive: Why stacking linear layers without an activation does nothing
If every layer were purely linear, two stacked Dense layers would compute
W2(W1x + b1) + b2 = (W2W1)x + (W2b1 + b2) — which has exactly the same form as ONE linear
layer with weights W2W1 and bias W2b1+b2. No amount of stacking helps without a nonlinearity
breaking that collapse. This is the single most important reason activation functions exist.
Try the datasets below. "Quadrants" and "Circles" are usually solvable with one hidden layer of a handful of units. "Spiral" is deliberately hard — that's this module's mini-project.
Production note
This uses binary cross-entropy loss (via tf.losses.logLoss here) and
a sigmoid output — the classification analogue of the mean-squared-error regression from the
previous module. Same gradient descent, same optimizer, different loss function for a different
kind of prediction.
Playground
Click empty space to add a point, drag a point to move it, double-click to remove it.
🔍 Deep dive: What does the model's confidence surface look like in 3D?
The decision boundary above is just the 0.5 contour line of this surface — everywhere above it the model is more than 50% confident in Class A.
Mini project
The default network from the playground above can't untangle this spiral. Add capacity — more hidden units, a second layer, a higher learning rate — until accuracy passes 95%.