dbt Scheduling Runs

Scheduling ensures your dbt models run automatically at the right times without manual intervention. Fresh data reaches dashboards on time, tests catch problems overnight, and your team wakes up to accurate numbers every morning. This topic covers scheduling in both dbt Cloud and with external orchestrators.

Scheduling in dbt Cloud

dbt Cloud has a built-in scheduler. Configure it per job under Deploy > Jobs.

Simple Schedule Options

Run every day at a specific time:
  Frequency: Daily
  Time: 06:00 UTC

Run every N hours:
  Frequency: Every 6 hours

Run on specific days of the week:
  Frequency: Custom
  Days: Monday, Wednesday, Friday
  Time: 08:00 UTC

Cron Schedule

Use a cron expression for precise control:

Cron syntax: minute hour day-of-month month day-of-week

Examples:
  0 6 * * *        → Every day at 6:00 AM UTC
  0 */4 * * *      → Every 4 hours
  30 5 * * 1-5     → Mon–Fri at 5:30 AM UTC
  0 8 1 * *        → First day of every month at 8:00 AM
  0 6,12,18 * * *  → Daily at 6 AM, 12 PM, and 6 PM UTC

Common Scheduling Patterns

Use Case                    Schedule           Commands
------------------------    ----------         --------
Daily dashboard refresh     Every day 6 AM     dbt build
Hourly sales metrics        Every hour         dbt run --select tag:hourly
Weekly executive report     Monday 5 AM        dbt build
Monthly finance close       1st of month 1 AM  dbt build --select tag:finance
Real-time event pipeline    Every 15 minutes   dbt run --select tag:streaming

Scheduling with Tags

Assign tags to models that need different refresh frequencies. Run separate jobs for each tag:

# In schema.yml or dbt_project.yml
models:
  my_project:
    marts:
      +tags: ['daily']
      realtime:
        +tags: ['hourly']
      finance:
        +tags: ['weekly', 'daily']

# Hourly job command (runs fast subset)
dbt run --select tag:hourly

# Daily job command (runs everything)
dbt build

# Weekly job command
dbt build --select tag:weekly

Scheduling with Apache Airflow

Airflow is the most popular external orchestrator used with dbt Core. A dbt task in Airflow runs dbt CLI commands inside a Docker container or virtual environment.

# Airflow DAG (simplified Python)
from airflow import DAG
from airflow.operators.bash import BashOperator
from datetime import datetime

with DAG(
    dag_id='dbt_daily_run',
    schedule_interval='0 6 * * *',   ← cron: daily at 6 AM
    start_date=datetime(2025, 1, 1),
    catchup=False
) as dag:

    dbt_deps = BashOperator(
        task_id='dbt_deps',
        bash_command='dbt deps --project-dir /opt/dbt'
    )

    dbt_build = BashOperator(
        task_id='dbt_build',
        bash_command='dbt build --project-dir /opt/dbt --target prod'
    )

    dbt_deps >> dbt_build    ← deps runs before build

Scheduling with Prefect

from prefect import flow, task
from prefect.deployments import Deployment
from prefect.server.schemas.schedules import CronSchedule
import subprocess

@task
def run_dbt(command: str):
    result = subprocess.run(
        f"dbt {command} --target prod",
        shell=True, check=True
    )
    return result.returncode

@flow
def dbt_daily_flow():
    run_dbt("deps")
    run_dbt("build")

# Deploy with cron schedule
Deployment.build_from_flow(
    flow=dbt_daily_flow,
    name="dbt-daily",
    schedule=CronSchedule(cron="0 6 * * *")
).apply()

Scheduling After Ingestion Completes

The best time to run dbt is immediately after your ingestion tool finishes loading raw data — not on a fixed clock schedule. This avoids stale data or empty runs:

Pattern 1: Webhook trigger
  Fivetran finishes sync → calls dbt Cloud API → dbt job starts

Pattern 2: Airflow sensor
  AirflowSensor watches for new file in S3
  When file arrives → trigger dbt BashOperator

Pattern 3: Event-driven (Dagster, Prefect)
  Asset sensor detects new data in raw schema
  Materializes downstream dbt assets

Handling Time Zones

dbt Cloud scheduler uses UTC. Convert your desired local time to UTC when setting schedules:

Desired time        UTC equivalent
-----------         --------------
6 AM US Eastern     11:00 UTC (EST) or 10:00 UTC (EDT)
6 AM India (IST)    00:30 UTC
6 AM UK (GMT)       06:00 UTC

Monitoring Scheduled Runs

• Check the Run History page in dbt Cloud after each scheduled run
• Set up Slack or email alerts for failures (Topic 39)
• Track model-level run time trends to spot performance regressions
• Use dbt source freshness at the start of each job to validate raw data arrived before transforming