R Linear Regression
Linear regression models the relationship between one outcome variable (y) and one predictor variable (x) as a straight line. The model learns the slope and intercept of the best-fitting line through your data, then uses that line to make predictions on new inputs.
The Linear Regression Equation
y = β₀ + β₁x + ε y = outcome variable (what you want to predict) x = predictor variable (the input) β₀ = intercept (y value when x=0) β₁ = slope (how much y changes when x increases by 1) ε = error / residual (unexplained variation) Example: Score = 40 + 5.5 × (Study Hours) 5 hours → 40 + 5.5×5 = 67.5 8 hours → 40 + 5.5×8 = 84
Fitting a Linear Model with lm()
study_hours <- c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10) exam_score <- c(45, 52, 58, 65, 70, 74, 80, 85, 88, 95) model <- lm(exam_score ~ study_hours) print(model) # Coefficients: # (Intercept) study_hours # 38.73 5.76
This tells you: Score = 38.73 + 5.76 × (study hours). Each additional hour of study adds about 5.76 points to the score.
Reading the summary() Output
summary(model)
Call: lm(formula = exam_score ~ study_hours)
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 38.73 1.42 27.2 <2e-16 ***
study_hours 5.76 0.23 25.4 <2e-16 ***
Residual standard error: 1.95 on 8 degrees of freedom
Multiple R-squared: 0.9873 ← 98.7% of score variance explained
Adjusted R-squared: 0.9857
F-statistic: 644.7 on 1 and 8 DF, p-value: 1.79e-10
Key output to read: Estimate → slope and intercept values Pr(>|t|) → p-value for each coefficient (*** = very significant) R-squared → how much variance the model explains (0 to 1) Residual SE → average prediction error F-statistic → overall model significance
Making Predictions
# Predict for new study hours new_data <- data.frame(study_hours = c(3, 6, 9, 12)) predict(model, newdata=new_data) # 1 2 3 4 # 56.01 73.29 90.57 107.85 # With confidence intervals predict(model, newdata=new_data, interval="confidence") # fit lwr upr # 56.01 54.19 57.84 # ... # With prediction intervals (wider — for individual observations) predict(model, newdata=new_data, interval="prediction")
Model Diagnostics
par(mfrow=c(2,2)) plot(model) # Four diagnostic plots: # 1. Residuals vs Fitted → check linearity and equal variance # 2. Normal Q-Q → check normality of residuals # 3. Scale-Location → check homoscedasticity # 4. Residuals vs Leverage → find influential points
Good model signs: Residuals vs Fitted → random scatter around zero (no pattern) Q-Q plot → points close to diagonal line Scale-Location → horizontal band, roughly equal spread
Visualizing the Regression Line
library(ggplot2)
df <- data.frame(hours=study_hours, score=exam_score)
ggplot(df, aes(x=hours, y=score)) +
geom_point(size=3, color="steelblue") +
geom_smooth(method="lm", color="red", fill="pink", alpha=0.2) +
labs(title="Score vs Study Hours",
subtitle=paste("R² =", round(summary(model)$r.squared,3)),
x="Study Hours", y="Exam Score") +
theme_minimal()
Model Performance Metrics
# Extract metrics manually r_squared <- summary(model)$r.squared # 0.987 adj_r2 <- summary(model)$adj.r.squared # 0.986 rmse <- sqrt(mean(residuals(model)^2)) # 1.85 (root mean sq error) # Residuals residuals(model) # actual - predicted for each point fitted(model) # predicted values
Assumptions of Linear Regression
Assumption How to Check ────────────────────────────────────────────────────────────────── Linearity Scatter plot of x vs y; Residuals vs Fitted Normality of errors Q-Q plot of residuals Equal variance Scale-Location plot (no funnel shape) Independence Study design; Durbin-Watson test for time series No multicollinearity Multiple regression only; check VIF values
Linear regression is the foundation of statistical modeling. Even complex machine learning models are extensions or refinements of this core idea. Master the interpretation of coefficients, R-squared, and diagnostic plots before moving to multiple regression and more advanced methods.