DS Recommendation Systems

A recommendation system predicts which items a person will enjoy. Video apps suggest films, shops suggest products, and music apps build playlists. These systems keep users engaged and help them find items among millions of choices.

Think of a friendly librarian who remembers every book you borrowed. The librarian notices your taste and hands you a new title that fits. Recommendation systems do that for millions of people at once.

The Data Behind Recommendations

Most systems start with a user-item matrix. Rows list users, columns list items, and cells hold ratings. Many cells stay empty because nobody rates everything.

          Film A   Film B   Film C   Film D
 Anil       5        4        ?        1
 Bela       4        ?        2        1
 Chirag     1        1        5        4
 Divya      ?        2        4        5

 ? = unknown rating the system tries to predict

Three Main Approaches

Approach	Idea	Example
Content-based	Suggest items similar to ones the user liked	You liked a space film, so you see more space films
Collaborative filtering	Suggest items that similar users liked	People with your history loved Film D
Hybrid	Blend both approaches	Large streaming services

Content-Based Filtering

This method describes each item with features such as genre, actor, or price. The system builds a taste profile from the items a user liked and ranks new items by closeness to that profile. A reader who enjoyed three mystery novels receives more mystery novels.

Strength

Works even when the user is the only person in the system.

Weakness

Traps users in a narrow bubble and rarely surprises them.

Collaborative Filtering

This method ignores item details. It studies behavior only. Two users who rated many items alike probably share taste, so one user's favorites become suggestions for the other.

User-based version

Find users similar to the target user, and recommend items they rated highly.

Item-based version

Find items similar to the one the user liked, based on who rated them. Shops often use this version because item patterns stay steady over time.

Measure Similarity

Cosine similarity compares two rating lists as arrows. A value near 1 means the arrows point the same way, so the tastes match. A value near 0 means unrelated tastes.

import numpy as np

anil   = np.array([5, 4, 1])
bela   = np.array([4, 3, 1])
chirag = np.array([1, 1, 5])

def cosine(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

print("Anil and Bela:", round(cosine(anil, bela), 2))
print("Anil and Chirag:", round(cosine(anil, chirag), 2))

Anil and Bela score close to 1, so their tastes match. Anil and Chirag score much lower. The system would borrow ideas from Bela's list for Anil.

Predict a Missing Rating

 Goal: predict Anil's rating for Film C

 Step 1  Find users similar to Anil      -> Bela (high), Chirag (low)
 Step 2  Read their ratings of Film C    -> Bela: 2, Chirag: 5
 Step 3  Weight each rating by similarity -> Bela counts more
 Step 4  Combine into one number         -> predicted rating near 2.5

The low prediction tells the system to avoid pushing Film C to Anil.

Matrix Factorization

Large systems compress the giant matrix into two smaller tables of hidden traits. One table describes users, and the other describes items. A hidden trait might stand for how much a film leans toward comedy or action. Multiplying a user row by an item column estimates the missing rating. Libraries such as Surprise and LightFM provide ready-made versions.

Challenges

Challenge	Meaning	Common Fix
Cold start	New users or items have no history	Ask for favorite topics, or use content-based methods first
Sparsity	Most matrix cells are empty	Use matrix factorization
Popularity bias	Famous items crowd out niche ones	Add variety rules
Privacy	Systems use personal behavior	Collect only needed data and explain its use

Judge Recommendation Quality

• RMSE: average size of the error in predicted ratings.
• Precision at k: share of the top k suggestions that the user liked.
• Recall at k: share of all liked items that appear in the top k.
• Coverage and diversity: how much of the catalog and how many styles the system shows.

Live A/B tests give the final verdict. Teams compare click rates and watch time between old and new suggestion lists.

Key Points

• Content-based methods match item traits, and collaborative methods match user behavior.
• Similarity scores connect users or items with shared patterns.
• Cold start and sparsity remain the main practical hurdles.