R Hypothesis Testing

Hypothesis testing is a formal method for making decisions based on data. You state a claim, collect data, and calculate the probability of seeing results this extreme if the claim were false. R provides built-in functions for all common tests. The output tells you whether the evidence supports or contradicts your claim.

The Hypothesis Testing Framework

Step 1: State hypotheses
  H₀ (Null):       "Nothing interesting is happening"
  H₁ (Alternative): "Something interesting is happening"

Step 2: Choose significance level (α)
  Usually α = 0.05 (5% false positive rate)

Step 3: Collect data and compute test statistic

Step 4: Calculate p-value
  p-value = P(seeing this result or more extreme | H₀ is true)

Step 5: Decision
  p ≤ α → Reject H₀ (evidence for H₁)
  p > α → Fail to reject H₀ (no strong evidence against H₀)

One-Sample t-Test

Tests whether the population mean equals a specific value.

# Question: Is the average exam score different from 75?
scores <- c(78, 85, 72, 90, 68, 88, 75, 80, 92, 65)

t.test(scores, mu=75)
# One Sample t-test
# t = 1.28, df = 9, p-value = 0.233
# 95% CI: [72.5, 85.7]
# mean = 79.3

# p = 0.233 > 0.05 → fail to reject H₀
# Not enough evidence that mean ≠ 75

Two-Sample t-Test

Tests whether two group means are equal.

class_A <- c(78, 85, 72, 90, 68, 88, 75, 80)
class_B <- c(65, 70, 72, 68, 75, 60, 74, 69)

t.test(class_A, class_B)
# t = 3.15, df = 13.8, p-value = 0.007
# 95% CI: [4.0, 20.7]

# p = 0.007 < 0.05 → Reject H₀
# Class A mean is significantly higher than Class B

Paired t-Test

Tests pre/post differences for the same subjects.

before <- c(120,135,145,130,125,140,150,128)  # blood pressure before
after  <- c(115,128,138,122,118,132,142,120)  # after medication

t.test(before, after, paired=TRUE)
# t = 8.0, df = 7, p-value = 0.00008
# Medication significantly reduced blood pressure

Chi-Square Test of Independence

Tests whether two categorical variables are associated.

# Is exam result independent of gender?
contingency <- matrix(c(30,20,15,35), nrow=2,
                       dimnames=list(c("Male","Female"), c("Pass","Fail")))

chisq.test(contingency)
# X-squared = 8.33, df = 1, p-value = 0.004
# p < 0.05 → Pass/Fail result is NOT independent of gender

ANOVA — Compare Three or More Groups

# Do three teaching methods produce different mean scores?
data <- data.frame(
  score  = c(78,82,85,80,76, 88,92,90,86,94, 70,74,72,68,76),
  method = rep(c("Lecture","Online","Workshop"), each=5)
)

model <- aov(score ~ method, data=data)
summary(model)
# method: F=17.7, p=0.0002
# p < 0.05 → At least one method differs significantly

Wilcoxon Test — Non-Parametric Alternative to t-Test

# Use when data is not normally distributed
wilcox.test(class_A, class_B)         # two-sample
wilcox.test(before, after, paired=TRUE)  # paired

p-Value Interpretation Diagram

p-value scale:
  0.001    0.01    0.05    0.10      1.0
  |────────|───────|───────|─────────|
  ████████████████████
  Strong          Weak          None
  evidence        evidence      ← fail to reject H₀

  p < 0.001  → very strong evidence
  p < 0.01   → strong evidence
  p < 0.05   → moderate evidence (standard threshold)
  p > 0.05   → insufficient evidence

Effect Size — Does Significance Mean Importance?

# A p-value tells you IF an effect exists — not HOW BIG it is
# Always report effect size alongside p-values

library(effectsize)
cohens_d(class_A, class_B)
# d = 1.57 → large effect

# Cohen's d interpretation:
# 0.2 = small, 0.5 = medium, 0.8+ = large

Hypothesis testing gives decisions a statistical foundation. The p-value answers one narrow question: "Is this result surprising if there were no real effect?" Always pair it with effect sizes and confidence intervals to understand practical significance, not just statistical significance.