PyTorch Tensors Basics

A tensor is the central data structure in PyTorch. Everything — images, text, audio, model weights, predictions — gets stored as a tensor before PyTorch can work with it. Understanding tensors is the single most important step before moving to neural networks.

What Is a Tensor

A tensor is a container for numbers. The number of dimensions a tensor has is called its rank. A single number is a 0D tensor (scalar). A list of numbers is a 1D tensor (vector). A table of numbers is a 2D tensor (matrix). A stack of tables is a 3D tensor, and so on.

0D Tensor (Scalar):   42

1D Tensor (Vector):   [10, 20, 30]

2D Tensor (Matrix):   [[ 1,  2,  3],
                        [ 4,  5,  6]]

3D Tensor:            [[[ 1,  2],
                         [ 3,  4]],
                        [[ 5,  6],
                         [ 7,  8]]]

Real-World Tensor Examples

Type of Data          Tensor Shape
──────────────────────────────────────────
Single pixel            ()  or scalar
Grayscale image       (height, width)
Color image           (3, height, width)
Batch of images       (32, 3, 224, 224)
Single word embedding (128,)
Sentence embedding    (seq_len, 128)

Creating Tensors

From a Python List

import torch

t = torch.tensor([1.0, 2.0, 3.0])
print(t)        # tensor([1., 2., 3.])
print(t.shape)  # torch.Size([3])

Filled with Zeros or Ones

zeros = torch.zeros(3, 4)   # 3 rows, 4 columns, all zeros
ones  = torch.ones(2, 2)    # 2x2 matrix of ones
print(zeros)
print(ones)

Filled with Random Numbers

r = torch.rand(2, 3)    # values between 0 and 1
n = torch.randn(2, 3)   # values from normal distribution (mean=0, std=1)
print(r)
print(n)

Range of Numbers

t = torch.arange(0, 10, 2)   # start=0, stop=10, step=2
print(t)  # tensor([0, 2, 4, 6, 8])

Checking Tensor Properties

Three properties you check most often are shape, data type, and device.

import torch

t = torch.tensor([[1.0, 2.0], [3.0, 4.0]])

print(t.shape)   # torch.Size([2, 2])
print(t.dtype)   # torch.float32
print(t.device)  # cpu

Shape Diagram

t = [[1, 2],
     [3, 4]]

t.shape → [2, 2]
            │  └── 2 columns
            └───── 2 rows

Accessing:
t[0]    → [1, 2]   (first row)
t[1]    → [3, 4]   (second row)
t[0][1] → 2        (row 0, column 1)

Changing Shape with reshape and view

You often need to change a tensor's shape without changing its data — for example, converting a flat list of 6 numbers into a 2×3 matrix.

import torch

t = torch.arange(6)      # tensor([0, 1, 2, 3, 4, 5])
r = t.reshape(2, 3)      # shape: 2 rows, 3 columns
print(r)
# tensor([[0, 1, 2],
#         [3, 4, 5]])

view works the same way but requires the tensor to be stored contiguously in memory. reshape is safer because it handles both cases.

Basic Tensor Math

import torch

a = torch.tensor([1.0, 2.0, 3.0])
b = torch.tensor([4.0, 5.0, 6.0])

print(a + b)       # tensor([5., 7., 9.])
print(a - b)       # tensor([-3., -3., -3.])
print(a * b)       # tensor([ 4., 10., 18.])
print(a / b)       # tensor([0.25, 0.4, 0.5])
print(torch.dot(a, b))  # tensor(32.)  ← dot product: 1*4 + 2*5 + 3*6

Scalar Operations (Broadcasting)

You can apply a single number to every element of a tensor without writing a loop. PyTorch calls this broadcasting.

t = torch.tensor([10.0, 20.0, 30.0])
print(t * 2)   # tensor([20., 40., 60.])
print(t + 5)   # tensor([15., 25., 35.])

Matrix Multiplication

Matrix multiplication is the core operation inside every neural network layer. In PyTorch, use torch.matmul or the @ operator.

A = torch.tensor([[1.0, 2.0],
                  [3.0, 4.0]])   # shape: 2x2

B = torch.tensor([[5.0],
                  [6.0]])        # shape: 2x1

C = torch.matmul(A, B)          # result shape: 2x1
print(C)
# tensor([[17.],
#         [39.]])

Matrix Multiplication Diagram:
   A          B         C
[1  2]   [5]       [1*5 + 2*6] = [17]
[3  4] × [6]   =  [3*5 + 4*6] = [39]

Aggregation Functions

t = torch.tensor([3.0, 1.0, 4.0, 1.0, 5.0])

print(t.sum())    # tensor(14.)
print(t.mean())   # tensor(2.8)
print(t.max())    # tensor(5.)
print(t.min())    # tensor(1.)
print(t.std())    # standard deviation

Summary

A tensor is a multi-dimensional container for numbers. A 1D tensor is a list. A 2D tensor is a table. A 3D tensor is a stack of tables. PyTorch provides many ways to create tensors — from lists, filled with zeros or ones, or with random values. You check a tensor using its shape, dtype, and device. Tensor math includes element-wise operations, broadcasting, dot products, and matrix multiplication. These operations form the mathematical engine of every neural network.