Airflow Security and Access Control

Security in Airflow means protecting your pipelines, credentials, and data from unauthorized access. Access control lets you decide who can view, edit, or trigger specific DAGs. Both become critical when Airflow moves from your laptop to a shared production environment.

Airflow's Role-Based Access Control (RBAC)

Airflow uses a role-based permission system. Every user gets assigned one or more roles. Roles define exactly what a user can see and do in the UI.

RBAC Model:
┌────────────┐      ┌──────────────┐      ┌─────────────────────────────┐
│    User    │─────▶│    Role      │─────▶│        Permissions          │
│ alice@co   │      │  Data Analyst│      │ can_read: DAGs              │
│            │      │              │      │ can_read: Task Instances    │
│            │      │              │      │ can_edit: DAGs (none)       │
└────────────┘      └──────────────┘      └─────────────────────────────┘

Built-In Roles

Creating and Managing Users

# Create a new user via CLI
airflow users create \
  --username alice \
  --firstname Alice \
  --lastname Smith \
  --role Viewer \
  --email alice@company.com \
  --password SecurePass!

# Change a user's role
airflow users add-role -u alice -r Op

# List all users
airflow users list

# Delete a user
airflow users delete -u alice

Custom Roles

Create custom roles in the UI under Security → List Roles. Click the plus button, name the role, and select individual permissions from the list. This lets you define roles like "Sales Team" that can only see and trigger sales-related DAGs.

DAG-Level Access Control

Restrict which users can see specific DAGs by adding an access_control parameter to your DAG definition:

with DAG(
    dag_id="confidential_finance_pipeline",
    schedule="@daily",
    start_date=datetime(2024, 1, 1),
    access_control={
        "Finance Team": {"can_read", "can_edit", "can_dag_run"},
        "Viewer Role":  {"can_read"},
    },
) as dag:
    ...

Users not in "Finance Team" or "Viewer Role" cannot even see this DAG in the UI.

Securing Connections and Variables

The Fernet Key: Encrypting Credentials

Airflow encrypts sensitive fields (passwords in Connections, variable values) using a Fernet key. Generate and set this key before storing any credentials:

# Generate a Fernet key
python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"

# Output example:
# 81HqDtbqAywKSOumSha3BhWNOdQ26slT6K0YaZeZyPs=

Add the key to airflow.cfg:

[core]
fernet_key = 81HqDtbqAywKSOumSha3BhWNOdQ26slT6K0YaZeZyPs=

Without a Fernet key, passwords in Connections are stored as plain text — a serious security risk.

Environment Variables for Secrets

Never put passwords directly in DAG code. Use environment variables or a secrets backend instead:

# Set a connection as an environment variable
export AIRFLOW_CONN_PROD_DB="postgresql://user:SecurePass@db.company.com:5432/sales"

# Set a variable as an environment variable
export AIRFLOW_VAR_ALERT_EMAIL="team@company.com"

Airflow reads these automatically. They never appear in the UI or database.

Secrets Backends

For production, store secrets in dedicated secret management services. Airflow supports these directly:

Secret Storage Hierarchy (Airflow checks in this order):
1. Environment Variables  (AIRFLOW_CONN_*, AIRFLOW_VAR_*)
2. Secrets Backend        (HashiCorp Vault, AWS Secrets Manager)
3. Airflow Database       (Admin → Connections / Variables in UI)

AWS Secrets Manager Example

# airflow.cfg
[secrets]
backend = airflow.providers.amazon.aws.secrets.secrets_manager.SecretsManagerBackend
backend_kwargs = {"connections_prefix": "airflow/connections", "variables_prefix": "airflow/variables"}

After configuration, Airflow reads connection credentials from AWS Secrets Manager instead of its own database. Your credentials never touch the Airflow database.

API Authentication

Airflow's REST API requires authentication. Configure it in airflow.cfg:

[api]
auth_backends = airflow.api.auth.backend.basic_auth

For production, use token-based authentication or integrate with your organization's SSO (Single Sign-On) provider through the Kerberos or OAuth backends.

Webserver Authentication Backends

Airflow's web UI can authenticate users through multiple providers:

Network Security Best Practices

• Run the Airflow webserver behind a reverse proxy (Nginx or Apache) with HTTPS enabled
• Block direct access to port 8080 from the public internet — only expose it inside a VPN or internal network
• Place worker machines in a private network subnet — they do not need public internet access
• Use a managed PostgreSQL service (RDS, Cloud SQL) instead of a self-managed database for the metadata store
• Rotate the Fernet key periodically and re-encrypt existing connections after each rotation

Auditing: Who Did What

Airflow logs user actions in its database. Go to Browse → Audit Logs in the UI to see a complete history of who triggered a DAG, who cleared a task, who changed a connection, and when each action happened.

Audit Log Example:
─────────────────────────────────────────────────────────
Event          │ Who    │ DAG            │ Time
───────────────┼────────┼────────────────┼──────────────
trigger_dag    │ alice  │ finance_report │ 2024-01-15 09:01
clear_task     │ bob    │ daily_etl      │ 2024-01-15 10:22
variable.set   │ admin  │ —              │ 2024-01-15 11:00
─────────────────────────────────────────────────────────