CV Pose Estimation

Pose estimation locates the joints of a body in an image or video. The output looks like a stick figure drawn over a person. Software reads positions of shoulders, elbows, wrists, hips, knees, and ankles and uses them to understand movement.

Keypoints and Skeletons

A keypoint is a labeled joint position. A skeleton connects keypoints with lines. The popular COCO format uses 17 body keypoints, while Google MediaPipe Pose reports 33 landmarks that include hands and feet details.

        (0) nose
         |
   (5)---+---(6)       shoulders
    |         |
   (7)       (8)       elbows
    |         |
   (9)       (10)      wrists
        |
   (11)-----(12)       hips
    |         |
   (13)      (14)      knees
    |         |
   (15)      (16)      ankles

Two Strategies

Top-Down

A detector finds each person first, and a pose network analyzes each box separately. Accuracy runs high, and speed drops as the crowd grows.

Bottom-Up

A network finds every joint in the image first and groups joints into people afterward. Speed stays steady in crowds, and grouping errors appear more often.

Top-down:   Image -> Person boxes -> Pose per box -> Skeletons
Bottom-up:  Image -> All joints   -> Group joints  -> Skeletons

How the Network Predicts Joints

Most models output a heatmap for each joint. A heatmap is a small grayscale picture where bright spots show likely joint positions. Software picks the brightest spot as the keypoint.

Heatmap for "left wrist"
. . . . . . .
. . . . . . .
. . . o O o .     O = peak, the predicted wrist location
. . . . o . .

Pose Estimation With MediaPipe

pip install mediapipe opencv-python

import cv2
import mediapipe as mp

pose = mp.solutions.pose.Pose(min_detection_confidence=0.5)
draw = mp.solutions.drawing_utils

cap = cv2.VideoCapture(0)
while True:
    ok, frame = cap.read()
    if not ok:
        break
    rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    result = pose.process(rgb)
    if result.pose_landmarks:
        draw.draw_landmarks(frame, result.pose_landmarks,
                            mp.solutions.pose.POSE_CONNECTIONS)
    cv2.imshow("Pose", frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

Remember to convert BGR to RGB, because MediaPipe expects RGB input.

Measuring Joint Angles

Keypoints become useful when you compute angles. The elbow angle comes from three points: shoulder, elbow, and wrist.

import numpy as np

def angle(a, b, c):
    a, b, c = map(np.array, (a, b, c))
    v1, v2 = a - b, c - b
    cos = np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2))
    return np.degrees(np.arccos(np.clip(cos, -1, 1)))

Shoulder (S)
    \
     \  angle at elbow
      E ------ Wrist (W)

A rep counter for arm curls follows the elbow angle: a low angle marks a lifted weight, and a high angle marks a lowered one. Each full swing counts one repetition.

Applications

• Fitness apps that count repetitions and check exercise form
• Physical therapy tools that track range of motion between visits
• Sports analysis that studies a swing, a jump, or a running stride
• Animation studios that capture motion without special suits
• Workplace ergonomics checks that spot awkward lifting posture

Common Challenges

• Occlusion: a hidden elbow or a crossed leg produces guessed positions
• Loose clothing: baggy garments blur body outlines
• Camera angle: side views hide the far limbs
• Low light and motion blur: weak images lower joint confidence

Filter results by confidence score and smooth positions over several frames to reduce shaking.

Responsible Use

Pose data describes body movement and can reveal health conditions. Ask users for permission, process video on the device when possible, and store only the numbers the product needs.

Practice Exercise

Record a ten-second clip of yourself doing arm curls. Extract the elbow angle on every frame and plot it over time. Count the peaks to get the repetition total and compare it with a manual count.

Smoothing Shaky Joints

Raw keypoints jitter from frame to frame, even when a person stands motionless. A moving average over three to five frames removes most of the shake. Exponential smoothing offers a lightweight option: each new position blends with the previous smoothed position using a weight such as 0.3.

smooth = 0.3 x new_position + 0.7 x previous_smooth

Choosing a Model

• MediaPipe Pose suits phones, laptops, and real-time demos
• Larger research models suit offline analysis where accuracy matters most
• Multi-person models suit sports fields and busy rooms

Test with your own camera angle and clothing, because published scores come from specific datasets.