TensorFlow Shapes and Ranks
Shape and rank describe the dimensional structure of a tensor. They tell you how many dimensions a tensor has and how many elements exist along each dimension. Shape errors account for the majority of beginner TensorFlow bugs. When a layer receives a tensor with the wrong shape, the entire model crashes. Learning to read, predict, and manipulate shapes makes you a confident TensorFlow developer who spends less time debugging and more time building.
Rank: How Many Dimensions
The rank of a tensor is simply the number of dimensions (axes) it has. A single number has rank 0. A list has rank 1. A table has rank 2. A stack of tables has rank 3.
import tensorflow as tf
r0 = tf.constant(42) # Rank 0 — scalar
r1 = tf.constant([1, 2, 3, 4]) # Rank 1 — vector
r2 = tf.constant([[1, 2], [3, 4]]) # Rank 2 — matrix
r3 = tf.constant([[[1, 2], [3, 4]],
[[5, 6], [7, 8]]]) # Rank 3
print(tf.rank(r0)) # 0
print(tf.rank(r1)) # 1
print(tf.rank(r2)) # 2
print(tf.rank(r3)) # 3
Shape: Size Along Each Dimension
Shape describes how many elements exist along each axis. Shape is always written as a tuple of integers, one per dimension.
Tensor Shape Meaning ────────────────────────────────────────────────────────────── 42 () Scalar, no dimensions [1, 2, 3] (3,) 3 elements in 1D [[1,2],[3,4],[5,6]] (3, 2) 3 rows, 2 columns (color image 64×64) (64,64,3) Height, Width, Channels (batch of 32 images 64×64) (32,64,64,3) Batch, H, W, C ──────────────────────────────────────────────────────────────
import tensorflow as tf
x = tf.constant([[1, 2, 3],
[4, 5, 6]])
print(x.shape) # (2, 3)
print(x.shape[0]) # 2 — number of rows
print(x.shape[1]) # 3 — number of columns
print(len(x.shape)) # 2 — the rank
print(tf.size(x)) # 6 — total number of elements
Static vs. Dynamic Shape
TensorFlow recognizes two kinds of shape information:
Static Shape
Determined at graph-build time (before any data runs through). Keras layers use static shapes to validate connections and print summaries. Accessible via tensor.shape.
x = tf.constant([[1.0, 2.0], [3.0, 4.0]]) print(x.shape) # TensorShape([2, 2]) — known at build time print(x.shape.as_list()) # [2, 2]
Dynamic Shape
Computed at runtime when the actual data flows through. The batch size is often unknown at build time (different batches have different sizes) and shows as None in the static shape.
# In model summaries, None means "unknown until runtime" # Output Shape: (None, 128) # → batch size unknown, 128 features per sample # Get the actual shape at runtime print(tf.shape(x)) # tf.Tensor([2 2], shape=(2,), dtype=int32)
None in Shapes: Flexible Batch Size
# Keras input layer with flexible batch size
inputs = tf.keras.Input(shape=(784,))
# shape parameter omits the batch dimension
# Full shape is (None, 784) — None = any batch size works
model = tf.keras.Sequential([
tf.keras.layers.Dense(128, input_shape=(784,)),
tf.keras.layers.Dense(10)
])
model.summary()
# Layer (type) Output Shape Param #
# dense (Dense) (None, 128) 100480
# dense_1 (Dense) (None, 10) 1290
Shape Transformations
Reshape
Reorganizes elements into a new shape. The total element count must remain the same.
x = tf.range(24) # [0, 1, 2, ..., 23] shape: (24,)
a = tf.reshape(x, [4, 6]) # 4 rows, 6 columns
b = tf.reshape(x, [2, 3, 4]) # 2 groups of 3 rows of 4 columns
c = tf.reshape(x, [24, 1]) # Column vector
d = tf.reshape(x, [1, 24]) # Row vector
e = tf.reshape(x, [2, -1]) # TF calculates second dim: 24÷2=12
# shape: (2, 12)
Diagram — Reshape Visualization: (24,) → (4, 6): [0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23] becomes: [ 0 1 2 3 4 5] [ 6 7 8 9 10 11] [12 13 14 15 16 17] [18 19 20 21 22 23]
Expand Dims
Inserts a new dimension of size 1. Commonly used to add a batch dimension to a single sample before feeding it to a model.
# Single image: (224, 224, 3) — no batch dimension image = tf.zeros([224, 224, 3]) # Model expects: (batch, 224, 224, 3) batch = tf.expand_dims(image, axis=0) print(batch.shape) # (1, 224, 224, 3)
Squeeze
Removes dimensions of size 1.
x = tf.zeros([1, 224, 224, 3]) print(tf.squeeze(x).shape) # (224, 224, 3) print(tf.squeeze(x, axis=0).shape) # (224, 224, 3) — remove only axis 0
Transpose
Reorders the dimensions of a tensor.
x = tf.zeros([32, 224, 224, 3]) # (batch, height, width, channels) # Convert to channels-first format: (batch, channels, height, width) y = tf.transpose(x, perm=[0, 3, 1, 2]) print(y.shape) # (32, 3, 224, 224)
Shape Errors and How to Fix Them
Incompatible Shapes in Matrix Multiplication
A = tf.ones([3, 4]) B = tf.ones([3, 4]) # Error: shapes [3,4] and [3,4] cannot be multiplied # tf.matmul(A, B) → fails # Fix: transpose B so inner dimensions match [4 × 3] tf.matmul(A, tf.transpose(B)) # (3,4) × (4,3) = (3,3) ✓
Wrong Input Shape for a Layer
model = tf.keras.Sequential([
tf.keras.layers.Dense(64, input_shape=(784,))
])
x = tf.ones([32, 100]) # Wrong: model expects 784 features, not 100
model(x) # Error: Input 0 is incompatible with layer: expected shape=(None, 784)
# Fix: reshape x to match
x_correct = tf.ones([32, 784])
model(x_correct) # Works
Missing Batch Dimension
# Single image without batch dimension img = tf.ones([224, 224, 3]) # Passes it directly — fails because model expects (batch, 224, 224, 3) model(img) # Error # Fix: add batch dimension model(tf.expand_dims(img, axis=0)) # shape (1, 224, 224, 3) — works
Broadcasting: Operating on Tensors With Different Shapes
Broadcasting allows arithmetic operations between tensors of different shapes by automatically expanding the smaller tensor to match the larger one. This follows the same rules as NumPy broadcasting.
matrix = tf.ones([3, 4]) # Shape: (3, 4) vector = tf.constant([1.0, 2.0, 3.0, 4.0]) # Shape: (4,) # Broadcasting adds the vector to every row of the matrix result = matrix + vector print(result.shape) # (3, 4) # What TensorFlow does internally: # vector gets "broadcast" to shape (3, 4) by repeating it 3 times # Then element-wise addition happens
Checking Shapes at Every Stage
Always print shapes when debugging a new model. One print per layer output catches shape problems immediately:
x = tf.ones([32, 224, 224, 3])
print("After input: ", x.shape)
x = conv1(x)
print("After conv1: ", x.shape)
x = pool1(x)
print("After pool1: ", x.shape)
x = tf.keras.layers.Flatten()(x)
print("After flatten: ", x.shape)
Mastering shapes means you can follow data through any neural network architecture — no matter how complex — by tracing how each layer changes the shape. The next topic covers eager execution, the default TensorFlow 2.x mode that makes this kind of interactive debugging natural and immediate.