TensorFlow Tensors
A tensor is the fundamental unit of data in TensorFlow. Every piece of information that flows through a TensorFlow model — images, text, sound, numbers — gets converted into a tensor. Understanding tensors deeply makes you a better TensorFlow developer because you know exactly what your data looks like at every stage of processing.
What Is a Tensor? The Filing Cabinet Analogy
Imagine a filing cabinet. A single sheet of paper is a 0-D tensor (one number). A drawer full of sheets stacked in one column is a 1-D tensor (a list). Multiple drawers lined up side by side form a 2-D tensor (a grid or table). Multiple filing cabinets stacked on shelves create a 3-D tensor. TensorFlow organizes all data in this layered filing cabinet structure.
Tensor Dimensions (Ranks)
Rank 0 — Scalar
A scalar is a single number with no direction or dimension.
scalar = tf.constant(7) # Output shape: () # Value: 7
Example in real life: the temperature reading of 36.5°C is a scalar.
Rank 1 — Vector
A vector is a list of numbers arranged in a single row or column.
vector = tf.constant([1.0, 2.0, 3.0, 4.0]) # Output shape: (4,) # Values: [1, 2, 3, 4]
Example in real life: the five daily temperatures for a week [28, 30, 27, 29, 31] form a vector.
Rank 2 — Matrix
A matrix organizes numbers in rows and columns, like a spreadsheet.
matrix = tf.constant([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])
# Output shape: (3, 3)
# 3 rows, 3 columns
Example in real life: a grayscale image 28 pixels wide and 28 pixels tall is a 28×28 matrix where each cell holds a brightness value from 0 to 255.
Rank 3 — 3D Tensor
A 3D tensor adds a third dimension. Color images use this structure.
color_image = tf.constant([
[[255, 0, 0], [0, 255, 0]],
[[0, 0, 255], [128, 128, 0]]
])
# Shape: (2, 2, 3)
# 2 rows, 2 columns, 3 color channels (R, G, B)
Rank 4 — 4D Tensor
A batch of color images uses a 4D tensor. TensorFlow processes multiple images at once (a batch) to speed up training.
# Shape: (batch_size, height, width, channels) # (32, 224, 224, 3) means: # 32 images in the batch # Each image is 224×224 pixels # Each pixel has 3 color values (R, G, B)
Visualizing Tensor Dimensions
Rank 0 (Scalar): • ← A single dot (one number)
Rank 1 (Vector): • • • • ← A row of dots
Rank 2 (Matrix): • • • • ← A grid of dots
• • • •
• • • •
Rank 3 (3D): Layer 1: • • • ← Multiple grids stacked
Layer 2: • • •
Layer 3: • • •
Rank 4 (4D): [Batch of multiple 3D tensors]
Creating Tensors in TensorFlow
tf.constant()
Creates a tensor whose value cannot be changed after creation.
a = tf.constant([[1, 2], [3, 4]]) print(a) # tf.Tensor([[1 2] [3 4]], shape=(2, 2), dtype=int32)
tf.zeros()
Creates a tensor filled entirely with zeros.
zeros = tf.zeros([3, 4]) # 3 rows, 4 columns, all values are 0.0
tf.ones()
Creates a tensor filled entirely with ones.
ones = tf.ones([2, 3]) # 2 rows, 3 columns, all values are 1.0
tf.random.normal()
Creates a tensor filled with random numbers drawn from a normal (bell curve) distribution. Useful for initializing neural network weights.
random = tf.random.normal([3, 3], mean=0.0, stddev=1.0)
tf.range()
Creates a tensor of numbers in a sequence, like Python's range() function.
sequence = tf.range(0, 10, 2) # [0, 2, 4, 6, 8]
Tensor Attributes You Must Know
Shape
Shape describes the size of each dimension. A tensor with shape (3, 4) has 3 rows and 4 columns.
t = tf.constant([[1, 2, 3, 4],
[5, 6, 7, 8],
[9, 0, 1, 2]])
print(t.shape) # (3, 4)
Rank
Rank is the number of dimensions a tensor has.
print(tf.rank(t)) # 2 (a matrix has 2 dimensions)
Data Type (dtype)
Every tensor holds a specific type of number. Common types are:
- tf.float32 — decimal numbers with 32-bit precision (most common in models)
- tf.float64 — decimal numbers with 64-bit precision (higher accuracy, more memory)
- tf.int32 — whole numbers
- tf.bool — True or False values
- tf.string — text data
t_float = tf.constant([1.5, 2.5]) print(t_float.dtype) # tf.float32
Converting Tensors
Casting: Changing Data Type
int_tensor = tf.constant([1, 2, 3]) # int32 float_tensor = tf.cast(int_tensor, tf.float32) # convert to float32 print(float_tensor) # [1.0, 2.0, 3.0]
Reshaping: Changing Structure Without Changing Data
Reshaping rearranges a tensor's values into a new shape. The total number of values must stay the same.
original = tf.constant([1, 2, 3, 4, 5, 6]) # Shape: (6,) reshaped = tf.reshape(original, [2, 3]) # Shape: (2, 3) # Visualized: # Before: [1, 2, 3, 4, 5, 6] # After: [[1, 2, 3], # [4, 5, 6]]
Converting to NumPy
You can convert any TensorFlow tensor into a NumPy array for easy inspection or use with other libraries.
tensor = tf.constant([10, 20, 30]) numpy_array = tensor.numpy() print(type(numpy_array)) # numpy.ndarray
Tensor Operations
Element-wise Operations
Most operations apply to each element individually.
a = tf.constant([1, 2, 3]) b = tf.constant([4, 5, 6]) print(a + b) # [5, 7, 9] print(a * b) # [4, 10, 18] print(a ** 2) # [1, 4, 9]
Matrix Multiplication
Matrix multiplication is the core computation inside every neural network layer.
A = tf.constant([[1, 2], [3, 4]]) B = tf.constant([[5, 6], [7, 8]]) C = tf.matmul(A, B) # C = [[1×5+2×7, 1×6+2×8], # [3×5+4×7, 3×6+4×8]] # C = [[19, 22], [43, 50]]
Why Tensor Shape Errors Are Common
The most frequent error in TensorFlow is a shape mismatch. For example, trying to multiply a tensor of shape (3, 4) with a tensor of shape (3, 4) directly fails because matrix multiplication requires the inner dimensions to match: (3, 4) × (4, 3) works, but (3, 4) × (3, 4) does not.
Always print tensor shapes while debugging:
print(my_tensor.shape) print(tf.shape(my_tensor))
Understanding tensors — their shape, rank, dtype, and how to manipulate them — forms the foundation for every TensorFlow operation. The next topic shows you the two key ways to store tensor values: constants and variables, and when to use each one.