DS Computer Vision

Computer vision lets computers read meaning from images and video. Phones unlock with a face scan, shops count items on shelves from camera feeds, and farms check crop leaves from drone photos. The field turns pixels into answers such as cat, damaged part, or empty parking space.

How a Computer Sees an Image

A computer sees an image as a grid of numbers. Each number is a pixel brightness from 0 (black) to 255 (white). A color image stacks three grids: red, green, and blue.

 Tiny 4 x 4 grey image          Numbers stored
 +----+----+----+----+          [  0  50 200 255 ]
 |    | ## |####|####|          [  0  60 210 250 ]
 |    | ## |####|####|          [ 10  70 220 255 ]
 |    | ## |####|####|          [  5  65 205 250 ]
 +----+----+----+----+

from PIL import Image
import numpy as np

img = Image.open("leaf.jpg")
arr = np.array(img)
print(arr.shape)

The shape might read 480, 640, 3, meaning 480 rows, 640 columns, and 3 color layers.

Main Vision Tasks

Task	Question Answered	Example
Image classification	What is in this picture?	Cat or dog
Object detection	What is where?	Boxes around cars on a road
Segmentation	Which pixels belong to which object?	Outline of a road surface
Optical character reading	What text appears?	Reading a number plate
Face and pose analysis	Where are faces or body points?	Camera focus on faces

Basic Image Operations

import cv2

img = cv2.imread("leaf.jpg")
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
small = cv2.resize(gray, (224, 224))
blur = cv2.GaussianBlur(small, (5, 5), 0)
edges = cv2.Canny(blur, 100, 200)

OpenCV is the classic library for such steps. Grey conversion removes color, resizing standardizes the size, blurring softens noise, and Canny edge detection outlines shapes.

Convolutional Neural Networks

A convolutional neural network, or CNN, learns visual patterns on its own. Small windows called filters slide across the image. Early filters notice edges. Middle layers combine edges into shapes such as circles and corners. Deep layers combine shapes into objects.

 Image --> [Conv + ReLU] --> [Pool] --> [Conv + ReLU] --> [Pool]
            finds edges      shrinks     finds shapes      shrinks
                                                               |
                                                          [Flatten]
                                                               |
                                                        [Dense layers]
                                                               |
                                                    Class chances: cat 0.92
                                                                   dog 0.08

Key pieces

• Convolution: a filter scans the image and highlights a pattern.
• Pooling: keeps the strongest signal in each small region and shrinks the data.
• Flatten: turns the final maps into one long list.
• Dense layers: convert the list into class chances.

Build a Small CNN

from tensorflow import keras

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train[..., None] / 255.0
x_test = x_test[..., None] / 255.0

model = keras.Sequential([
    keras.layers.Conv2D(32, 3, activation="relu", input_shape=(28, 28, 1)),
    keras.layers.MaxPooling2D(),
    keras.layers.Conv2D(64, 3, activation="relu"),
    keras.layers.MaxPooling2D(),
    keras.layers.Flatten(),
    keras.layers.Dense(64, activation="relu"),
    keras.layers.Dense(10, activation="softmax")
])

model.compile(optimizer="adam",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(x_train, y_train, epochs=3, validation_split=0.1)
print(model.evaluate(x_test, y_test))

The MNIST dataset holds 70000 images of handwritten digits. A small CNN usually reaches about 99 percent accuracy within a few minutes.

Transfer Learning

Training a deep network from nothing needs huge data and computing power. Transfer learning starts from a network trained on millions of images and adapts it to a new task. A shop with 500 product photos can fine-tune a pretrained model and reach strong accuracy.

base = keras.applications.MobileNetV2(
    input_shape=(224, 224, 3), include_top=False, weights="imagenet")
base.trainable = False

model = keras.Sequential([
    base,
    keras.layers.GlobalAveragePooling2D(),
    keras.layers.Dense(3, activation="softmax")
])

Freezing the base keeps its learned vision skills. The new top layer learns the three classes you care about.

Data Augmentation

Augmentation creates varied copies of training images through flips, small rotations, zooms, and brightness changes. The network sees more variety and overfits less.

Responsible Use

• Check that training images cover different lighting, backgrounds, and people.
• Test accuracy across groups to catch unfair gaps.
• Get consent before collecting photos of people.
• Follow local privacy rules for cameras and face data.

Example: Checking Products on a Factory Line

A factory photographs every bottle cap on a conveyor belt. A CNN learns from thousands of labeled photos of good and cracked caps. During production, the camera sends each image to the model, and the model returns a chance of damage in milliseconds. Caps above a chosen chance are pushed into a reject bin. Workers review a sample of rejected caps each day, and their feedback becomes new training images. The loop keeps accuracy high as lighting and materials change.

Measure Vision Models

Metric	Meaning
Accuracy	Share of images labeled correctly
Precision	Of the items flagged, the share that were truly flagged correctly
Recall	Of all true cases, the share the model found
IoU	Overlap between a predicted box and the true box

Key Points

• Images are grids of pixel numbers that models learn to interpret.
• CNNs find edges, shapes, and objects through stacked layers.
• Transfer learning and augmentation make small datasets workable.