R Multiple Regression

Multiple regression extends simple linear regression to use two or more predictor variables simultaneously. It models how multiple inputs together explain an outcome, while isolating the unique contribution of each predictor after controlling for the others.

The Multiple Regression Equation

y = β₀ + β₁x₁ + β₂x₂ + β₃x₃ + ε

Example: Salary = 15000 + 800×(Years Exp) + 5000×(Degree) + 200×(Projects)
  β₁ = 800   → each extra year of experience adds ₹800/month
  β₂ = 5000  → having a master's degree adds ₹5000/month
  β₃ = 200   → each project completed adds ₹200/month

Fitting a Multiple Regression Model

employees <- data.frame(
  salary  = c(30000,42000,55000,38000,62000,48000,70000,35000,58000,45000),
  exp     = c(1,3,7,2,9,5,11,1,8,4),
  degree  = c(0,1,1,0,1,1,1,0,1,0),   # 0=bachelor, 1=master
  projects= c(5,12,20,8,25,15,30,6,22,10)
)

model <- lm(salary ~ exp + degree + projects, data=employees)
summary(model)

Coefficients:
             Estimate Std. Error t value Pr(>|t|)
(Intercept)   22140     1850     11.97   <0.001 ***
exp            1820      210      8.67   <0.001 ***
degree         4650     1100      4.23    0.004 **
projects        310       95      3.26    0.017 *

R-squared: 0.985 → model explains 98.5% of salary variance

Interpreting Coefficients

exp = 1820:      One more year of experience adds ₹1,820/month
                 (holding degree and projects constant)

degree = 4650:   Having a master's (vs bachelor's) adds ₹4,650/month
                 (holding exp and projects constant)

projects = 310:  Each additional project adds ₹310/month
                 (holding exp and degree constant)

Predictions

new_emp <- data.frame(exp=5, degree=1, projects=18)
predict(model, newdata=new_emp, interval="prediction")
#     fit    lwr    upr
# 55090  50100  60080

Model Selection — Adding and Removing Predictors

# Start with all predictors
full_model <- lm(salary ~ exp + degree + projects, data=employees)

# Stepwise selection (forward, backward, both)
library(MASS)
step_model <- stepAIC(full_model, direction="both")
summary(step_model)

Multicollinearity — Predictors Correlated With Each Other

# If predictors correlate strongly, coefficients become unreliable
library(car)
vif(model)      # Variance Inflation Factor
# exp      degree  projects
# 2.14     1.87    2.51

# VIF interpretation:
# VIF < 5   → acceptable
# VIF 5-10  → moderate concern
# VIF > 10  → serious multicollinearity problem

Categorical Predictors (Dummy Variables)

employees$region <- factor(c("N","S","N","E","S","N","E","S","N","E"))

# R creates dummy variables automatically for factors
model2 <- lm(salary ~ exp + region, data=employees)
summary(model2)
# regionN and regionS coefficients compare each to regionE (reference)

Interaction Terms

# Does effect of experience differ by degree level?
model3 <- lm(salary ~ exp * degree, data=employees)
# exp:degree coefficient shows how degree modifies the exp effect

Model Comparison

m1 <- lm(salary ~ exp, data=employees)
m2 <- lm(salary ~ exp + degree, data=employees)
m3 <- lm(salary ~ exp + degree + projects, data=employees)

# Compare with ANOVA
anova(m1, m2, m3)
# Pr(>F) tells if adding predictors significantly improves fit

# AIC — lower is better
AIC(m1, m2, m3)

Standardized Coefficients (Compare Importance)

library(lm.beta)
lm.beta(model)
# Standardized coefficients show which predictor has most impact
# regardless of original measurement scale

Multiple regression lets you model realistic situations where outcomes depend on many factors simultaneously. The key discipline is resisting the urge to add every available variable — use domain knowledge and diagnostic checks to build models that are both accurate and interpretable. A simpler model that explains 92% of variance is often better than a complex one that explains 95%.