DS Random Forest

A decision tree makes predictions by asking a chain of yes or no questions. A random forest builds hundreds of such trees and combines their answers. The crowd usually beats any single tree. Both methods handle numbers and categories, need little scaling, and work well for beginners.

How a Decision Tree Thinks

Picture the game of twenty questions. Each question splits the possibilities into smaller groups until one answer remains. A tree follows the same path.

                 Is income above 40000?
                  /                  \
                Yes                   No
                 |                     |
        Has job over 2 years?     Has savings above 5000?
         /            \             /               \
       Yes            No          Yes                No
        |              |           |                  |
    Approve         Review      Review              Decline

The path from the top to a final box is a rule. A person with income above 40000 and a job older than two years lands on Approve.

How a Tree Chooses Questions

The tree picks the question that creates the purest groups. A pure group holds mostly one label. Two common measures of impurity are Gini and entropy. Lower values mean cleaner groups. The tree tests every column and every cut point, and keeps the best split at each step.

The Weakness of One Tree

A deep tree memorizes the training data, including its random quirks. Data scientists call this overfitting. The tree scores perfectly on training rows and poorly on new rows. Limiting depth helps, and a forest helps more.

How a Random Forest Works

            Training data
                 |
   +-------------+-------------+
   |             |             |
Random rows   Random rows   Random rows
Random cols   Random cols   Random cols
   |             |             |
 Tree 1        Tree 2        Tree 3   ...  Tree 100
   |             |             |
 Approve       Decline       Approve
   +-------------+-------------+
                 |
        Majority vote: Approve

Each tree trains on a random sample of rows and looks at a random subset of columns at every split. The trees become different from each other. Different trees make different mistakes, and voting cancels many of those mistakes. Regression forests average the numbers instead of voting.

Build a Forest in Python

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score

X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42)

model = RandomForestClassifier(n_estimators=200, max_depth=5, random_state=42)
model.fit(X_train, y_train)

pred = model.predict(X_test)
print("Accuracy:", accuracy_score(y_test, pred))

The iris dataset describes flower measurements. The forest predicts the flower species with accuracy that usually exceeds 0.9.

Find the Important Columns

names = ["sepal_len", "sepal_wid", "petal_len", "petal_wid"]
for name, score in zip(names, model.feature_importances_):
    print(name, round(score, 3))

Importance scores add up to 1. Higher scores mark columns that help the forest most. Petal measurements usually score highest for iris flowers. Teams use these scores to explain models and to drop useless columns.

Settings That Matter

Setting	Meaning	Effect
n_estimators	Number of trees	More trees give steadier results and slower training
max_depth	Longest path in a tree	Smaller values reduce overfitting
min_samples_leaf	Minimum rows in a final box	Larger values smooth the model
max_features	Columns checked per split	Fewer columns create more varied trees

Strengths and Limits

Strengths

• Strong accuracy with little tuning.
• Handles mixed data types and nonlinear patterns.
• Provides importance scores.

Limits

• Large forests use memory and take time to predict.
• The model acts as a black box compared with a single tree.
• Predictions cannot go beyond the range seen in training data.

Example: Predicting Customer Purchases

A shop records age, minutes spent on the site, number of pages viewed, and whether the visitor bought something. A forest of 200 trees studies thousands of past visits. Each tree learns a slightly different rule set. One tree may focus on minutes spent, and another may focus on pages viewed. When a new visitor arrives, every tree votes. If 150 trees vote buy, the forest reports a 75 percent purchase chance. The shop can then show a discount banner only to visitors with a mid-range chance and save marketing effort.

Key Points

• A decision tree asks questions that split data into purer groups.
• A random forest combines many varied trees through voting or averaging.
• Depth and tree count control the balance between accuracy and overfitting.