Deep Learning Tensors and Math Basics
Deep learning models do not see photos, words, or sounds. They see numbers arranged in grids. A small set of math ideas explains how those grids move through a network and how the network improves. You need no advanced degree to follow them. Spreadsheets, hills, and dice cover almost everything.
What Is a Tensor
A tensor is a container for numbers. Picture a spreadsheet that can grow extra dimensions. Every piece of data in deep learning lives inside a tensor.
Four Common Tensor Types
Scalar (0 dimensions): one number
7
Vector (1 dimension): a list of numbers
[2, 5, 9]
Matrix (2 dimensions): a table of numbers
[1 2 3]
[4 5 6]
3D Tensor: a stack of tables
Page 1: [1 2] Page 2: [5 6]
[3 4] [7 8]
A grayscale photo is a matrix. Each number stores the brightness of one pixel. A colour photo is a 3D tensor with three stacked pages for red, green, and blue.
Understanding Tensor Shape
Shape lists the size of every dimension. Frameworks print the shape of each tensor, and you will read these numbers constantly.
Shape examples One score ..................... () Three house features .......... (3) 28 x 28 grayscale image ....... (28, 28) 224 x 224 colour image ........ (224, 224, 3) Batch of 32 colour images ..... (32, 224, 224, 3)
Shape mismatches cause most beginner errors. Check the shape of your data before every new operation.
Core Tensor Operations
Addition and Scaling
Adding two tensors of the same shape adds matching positions. Multiplying a tensor by a single number scales every value at once. Networks use both actions to shift and resize signals.
[1, 2, 3] + [10, 20, 30] = [11, 22, 33] [1, 2, 3] x 2 = [2, 4, 6]
The Dot Product
The dot product is the heart of every neuron. Multiply each input by its weight and add the results.
Inputs: [2, 3, 1]
Weights: [0.5, 0.2, 0.8]
Dot product = (2 x 0.5) + (3 x 0.2) + (1 x 0.8)
= 1.0 + 0.6 + 0.8
= 2.4
Matrix Multiplication
A layer holds many neurons, and each neuron runs its own dot product. Matrix multiplication performs all of them in one step. Graphics processors handle this operation extremely fast, which explains why they power modern training.
Derivatives: Measuring Slope
Imagine standing on a hillside. A derivative tells you how steep the ground is under your feet. Steep ground means one small step changes your height a lot. Flat ground means a step changes almost nothing.
Error = x squared At x = 3 slope = 6 steep, far from the bottom At x = 1 slope = 2 gentler At x = 0.1 slope = 0.2 almost flat, close to the bottom
Training uses slope as a guide. A steep slope says the weight needs a big correction. A flat slope says the weight is nearly right.
Gradients: Slope in Many Directions
A network owns millions of weights. A gradient is a list that stores the slope for each weight. It points uphill toward more error, so the network steps in the opposite direction.
Weight A slope: +4.0 lower A Weight B slope: -1.5 raise B Weight C slope: 0.0 leave C alone
Probability Basics
A probability is a number between 0 and 1. A value of 0 means impossible, and a value of 1 means certain. A model that outputs 0.9 for the label cat expresses 90 percent confidence. The probabilities across all labels add up to 1.
Mean and Spread
The mean is the average of a group of numbers. The standard deviation measures how far values wander from that average. Data preparation and normalization rely on both.
Heights: 150, 160, 170 Mean = 160 Standard deviation = about 8.2
How the Pieces Connect
Data -> Tensor -> Dot products -> Prediction
|
Error measured
|
Gradients (slopes) -> Weights adjusted -> Better prediction
Key Terms
- Tensor - a grid of numbers with any number of dimensions
- Shape - the size of each tensor dimension
- Dot Product - multiply pairs of numbers and add the results
- Derivative - the slope at one point
- Gradient - the slopes of all weights collected in one list
- Probability - a confidence score from 0 to 1