What is a Tensor? | TensorFlow, Interactively

Every number TensorFlow works with — a single price, a row of house features, a whole image — is stored as a tensor. The name sounds intimidating, but a tensor is nothing more than "numbers arranged in a grid of some number of dimensions."

Rank 0 (scalar) — a single number, like 72.5.
Rank 1 (vector) — a list of numbers, like [72.5, 3, 12] (maybe: price, bedrooms, age).
Rank 2 (matrix) — a grid, like one grayscale image's pixel values.
Rank 3+ — stacks of matrices, like a color image (height × width × color channel) or a batch of images.

Beginner tip

If you've used a spreadsheet, a rank-2 tensor is just the grid of cells. A rank-1 tensor is a single row or column. A rank-0 tensor is a single cell. A quick trick for reading rank straight off the page: count the opening brackets before the first number. [1, 2, 3] has one bracket → rank 1. [[1, 2], [3, 4]] has two → rank 2.

Every tensor also has a shape — the size along each axis — and a dtype, the type of number it holds.

Play with the explorer below: switch between scalar, vector, and matrix, resize it, and reroll the values. The shape [rows, cols] — and the rank and size TensorFlow reports for it — updates live underneath.

Production note

In production code you'll almost always work with a batch dimension too — even a single image becomes rank 4: [batch, height, width, channels]. Keras layers expect the batch dimension even when you're only predicting on one example, which is the most common "why is my shape wrong" bug for people new to the framework.

🔍 Deep dive: Why does the batch dimension exist?

Training in batches (rather than one example at a time) lets the GPU parallelize the matrix multiplications behind every layer, and it gives gradient descent a smoother, less noisy signal to follow. tf.tensor([1, 2, 3]) has shape [3]; tf.tensor([[1, 2, 3]]) has shape [1, 3] — a batch of one 3-element example. The extra bracket is doing real work.

🔍 Deep dive: Broadcasting: how tensors of different shapes combine

Every arithmetic op in this course so far — xsT.mul(w).add(b) in the linear regression playground, for instance — quietly relies on broadcasting: when shapes don't match exactly, TensorFlow stretches the smaller tensor to fit, without copying data, as long as the shapes are compatible. A scalar w (shape []) multiplying a vector xsT (shape [16]) broadcasts w across all 16 elements. Two shapes are compatible for broadcasting if, comparing dimensions from the right, each pair is either equal or one of them is 1 — [16, 1] and [16, 4] broadcast together (the 1 stretches to 4), but [16, 3] and [16, 4] do not.

🔍 Deep dive: dtype: why float32 is the default, and when it bites you

tf.tensor([1, 2, 3]) silently becomes dtype float32, not the integers you typed — float32 is TensorFlow's default because gradient descent needs to represent tiny, continuously-varying updates, which integers can't. This matters in practice: dividing two int32 tensors truncates like integer division in most languages, and mixing dtypes in one operation (say, an int32 label tensor against a float32 prediction tensor) throws rather than silently casting — you'll hit this the first time you build a label tensor from raw class indices and forget to cast it.

Production note

Every lib/tf/*.ts helper elsewhere in this course wraps its math in tf.tidy(() => ...). TensorFlow.js doesn't garbage-collect tensors automatically the way plain JavaScript objects are — each one holds real GPU/CPU memory until explicitly disposed, and tf.tidy disposes every intermediate tensor created inside it once it returns (except ones you explicitly return out). Forgetting this in a loop that runs every animation frame — like every "Play" button in this course — leaks a little memory on every single tick.