beginner · ~15 min
Linear Regression Playground
Drag points, watch gradient descent fit a line to them in real time, then fit real housing data.
Linear regression fits a straight line, y = w·x + b, through a set of points. TensorFlow finds
w (the weight/slope) and b (the bias/intercept) by gradient descent —
starting from a guess, measuring how wrong it is (the loss),
and nudging w and b a little in the direction that reduces that loss. Each such nudge is one
step —
watch the "Step" counter below tick up as training runs.
Beginner tip
Think of the loss as a ball rolling downhill on a bowl-shaped surface. Each training step rolls it a little further toward the bottom, where the line fits the data best. Don't expect the loss to ever hit exactly zero, though — real data has noise, so a flattening-out curve (not a zero curve) is what convergence normally looks like.
Drag the points below (or add/remove some), tune the learning rate, and hit Play. Watch the line rotate into place and the loss curve fall as training progresses.
🔍 Deep dive: Why mean squared error, and not something else?
Squaring the error before averaging (rather than, say, averaging the absolute error) does two things: it punishes large errors disproportionately more than small ones, and it gives a smooth, everywhere-differentiable loss surface that gradient descent can follow cleanly — absolute error has a sharp corner at zero. There's also a probabilistic justification: minimizing squared error is exactly equivalent to maximum-likelihood estimation under the assumption that the noise in your data is Gaussian, which is why MSE is the default choice for regression unless you have a specific reason to expect outlier-heavy, non-Gaussian noise (in which case MAE is more robust).
🔍 Deep dive: What learning rate is 'too big'?
There's an actual numeric answer, not just "try smaller values until it works." For this
playground's seeded data, mean(x²) ≈ 23, and gradient descent on a least-squares problem like
this is stable only below roughly 1 / mean(x²) ≈ 0.04 — this is exactly the calculation that
caught a real bug while building this course: the housing-price mini-project below originally
used a learning rate of 0.05, which is just past that threshold, and its loss diverged to the
millions within 20 steps instead of converging. Dropping it to 0.02 fixed it. Try pushing the
learning rate slider above roughly 0.4-0.5 above and you should see the same kind of blow-up
live.
Production note
This uses tf.train.sgd — plain stochastic gradient descent — because it's the clearest
illustration of the mechanism. Production Keras code almost always reaches for Adam instead,
which adapts its effective learning rate per-parameter and converges faster with far less
tuning. The Training Dynamics Lab module compares them side by side.
Production note
Every loss number on this page is a training loss — measured on the exact same points the model is fitting. That number alone can't tell you whether the fit will generalize to a house the model has never seen; for that you need a held-out validation set, which is exactly what the Overfitting & Regularization module covers next.
Playground
Click empty space to add a point, drag a point to move it, double-click to remove it.
🔍 Deep dive: What does gradient descent look like from above?
Every (weight, bias) pair has a loss — together they form a bowl-shaped surface. Training is gradient descent rolling downhill on this exact surface toward the bottom.
Mini project
300 houses (square footage vs. price — see dataset notes). Train the same linear regression on real-shaped data, then predict a price for a house size you pick.