DS Feature Engineering
A feature is one column of input data that a model uses to make a prediction. Feature engineering means creating, changing, or selecting those columns so that the model learns faster and predicts better. Many practitioners say good features beat fancy algorithms.
Imagine a taxi company that wants to predict trip time. Raw data lists pickup time and distance. A useful new feature is rush hour, a yes or no flag. The model now sees a pattern that was hidden in plain clock times.
From Raw Data to Features
RAW TABLE ENGINEERED TABLE +-------------------+ +------------------------------+ | pickup_time | |hour | is_weekend | rush_hour | | distance_km | ------> |distance_km | speed_band | | driver_rating | | rating_group | +-------------------+ +------------------------------+
Technique 1: Extract Parts of Dates
A date holds hidden detail. Split it into pieces that models can use.
df["pickup_time"] = pd.to_datetime(df["pickup_time"])
df["hour"] = df["pickup_time"].dt.hour
df["weekday"] = df["pickup_time"].dt.dayofweek
df["is_weekend"] = (df["weekday"] >= 5).astype(int)Technique 2: Combine Columns
Ratios and totals often tell a stronger story than the original columns.
| Original Columns | New Feature | Why It Helps |
|---|---|---|
| price, area | price per square meter | Compares homes of different sizes fairly |
| clicks, visits | click rate | Shows interest regardless of traffic |
| height, weight | body mass ratio | Captures a relationship neither column shows alone |
Technique 3: Group Numbers into Bins
Binning turns a number into labeled ranges. Age becomes child, adult, or senior. Binning reduces noise and helps with skewed data.
bins = [0, 17, 59, 120]
labels = ["minor", "adult", "senior"]
df["age_group"] = pd.cut(df["age"], bins=bins, labels=labels)Technique 4: Encode Categories as Numbers
Models work with numbers, so text categories need conversion.
One-hot encoding
One-hot encoding creates a new column for each category, filled with 1 or 0.
color red blue green red -> 1 0 0 blue -> 0 1 0 green -> 0 0 1
df = pd.get_dummies(df, columns=["color"])Label encoding
Label encoding assigns one whole number per category. Use it for ordered groups such as small, medium, and large. Avoid it for unordered groups because the model may read 3 as bigger than 1.
Technique 5: Scale Numbers
Columns can live on different scales. Salary may range up to 100000 while age stays below 100. Many algorithms treat big numbers as more important. Scaling puts columns on equal footing.
| Method | Result | Use When |
|---|---|---|
| Standard scaling | Mean 0, spread 1 | Data follows a bell shape |
| Min-max scaling | Values between 0 and 1 | You need a fixed range |
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
df[["salary", "age"]] = scaler.fit_transform(df[["salary", "age"]])Technique 6: Transform Skewed Data
Income, house prices, and website visits often have a few huge values. A log transform squeezes large values and spreads small ones, which makes the shape friendlier for many models.
import numpy as np
df["log_income"] = np.log1p(df["income"])Select the Best Features
More columns do not guarantee better models. Extra columns add noise and slow training. Remove columns that repeat the same information, such as both temperature in Celsius and in Fahrenheit. Check correlation tables and feature importance scores to spot weak columns.
Avoid Data Leakage
Leakage happens when a feature contains information that would not exist at prediction time. A column called refund_issued would reveal whether an order was returned, which makes a return prediction look perfect and useless. Ask one question for every feature: will I know this value when the prediction is needed?
Example: Predicting Online Order Size
A shop wants to predict basket value. The raw table holds customer age, signup date, last visit date, and number of past orders. Smart features make the table far more useful.
| New Feature | How to Build It | Signal It Captures |
|---|---|---|
| Customer tenure | Today minus signup date | Loyal customers spend differently |
| Days since last visit | Today minus last visit date | Recent shoppers act differently |
| Orders per month | Past orders divided by tenure in months | Shopping rhythm |
Domain knowledge drives these ideas. Talk with the people who run the business, and ask which signals they watch.
Key Points
- Features are the inputs, and better inputs raise model quality.
- Date parts, ratios, bins, encodings, and scaling cover most daily needs.
- Guard against leakage by checking when each value becomes known.