Airflow Installation and Setup

This topic walks you through installing Apache Airflow on your local machine step by step. By the end, you will have a running Airflow instance ready for building your first workflow.

What You Need Before Installing

System Requirements

• Operating System: Linux or macOS (recommended). Windows users should use WSL 2 (Windows Subsystem for Linux).
• Python: Version 3.8 or higher
• pip: Python's package manager (comes with Python)
• RAM: At least 4 GB

Check Your Python Version

Open a terminal and run:

python3 --version

You should see something like Python 3.10.x. If Python is not installed, download it from python.org.

Installation Method: pip (Simplest for Beginners)

Apache Airflow installs through pip, Python's package manager. The installation command includes a constraint file that ensures all packages work together without version conflicts.

Step 1: Set the Airflow Home Directory

Airflow stores its configuration files and logs in a folder called AIRFLOW_HOME. Set this before installing:

export AIRFLOW_HOME=~/airflow

This tells Airflow to use a folder named airflow inside your home directory.

Step 2: Install Airflow With pip

AIRFLOW_VERSION=2.9.2
PYTHON_VERSION="$(python3 --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"

pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"

This downloads Airflow and all required libraries. The --constraint flag pins exact versions so nothing breaks due to a package mismatch.

Step 3: Initialize the Database

Airflow uses a database to store workflow runs, logs, and settings. Initialize it with:

airflow db init

This creates a SQLite database at ~/airflow/airflow.db. SQLite is fine for local learning. Production systems use PostgreSQL or MySQL.

Step 4: Create an Admin User

Create a user account to log into the Airflow web interface:

airflow users create \
  --username admin \
  --firstname Admin \
  --lastname User \
  --role Admin \
  --email admin@example.com \
  --password admin

Step 5: Start the Web Server

Open a new terminal window and run:

airflow webserver --port 8080

Step 6: Start the Scheduler

Open another terminal window and run:

airflow scheduler

The scheduler is the brain of Airflow. It reads your workflows and triggers tasks when their time comes.

Step 7: Open the Web Interface

Open your browser and go to http://localhost:8080. Log in with the username admin and password admin you set in Step 4.

What Just Happened — A Diagram

Your Machine
┌──────────────────────────────────────┐
│                                      │
│  ┌─────────────┐  ┌───────────────┐  │
│  │  Webserver  │  │   Scheduler   │  │
│  │ (port 8080) │  │  (triggers    │  │
│  │  UI / API   │  │   tasks)      │  │
│  └──────┬──────┘  └───────┬───────┘  │
│         │                 │          │
│         └────────┬────────┘          │
│                  │                   │
│         ┌────────▼────────┐          │
│         │   SQLite DB     │          │
│         │  (airflow.db)   │          │
│         └─────────────────┘          │
│                                      │
│  ~/airflow/dags/   ← your DAG files  │
└──────────────────────────────────────┘

The webserver shows you information. The scheduler does the work. Both read and write to the same database. Your workflow files live in the dags folder.

Install Airflow Using Docker (Alternative)

Docker is the preferred method for teams and production environments. It packages Airflow and all its dependencies into containers so the setup is identical on every machine.

Quick Start With Docker Compose

curl -LfO 'https://airflow.apache.org/docs/apache-airflow/stable/docker-compose.yaml'
mkdir -p ./dags ./logs ./plugins
echo -e "AIRFLOW_UID=$(id -u)" > .env
docker compose up airflow-init
docker compose up

Open http://localhost:8080 and log in with username airflow and password airflow.

Verify Everything Works

In the Airflow UI, you will see a list of example DAGs that come pre-loaded. Click on any DAG, then click the play button (Trigger DAG). Watch the tasks turn green as they complete. A green task means success.

Folder Structure After Setup

~/airflow/
├── airflow.cfg       ← main configuration file
├── airflow.db        ← SQLite database
├── logs/             ← task execution logs
├── dags/             ← put your workflow files here
└── plugins/          ← custom extensions go here

Common Setup Errors and Fixes