GitLab Pipeline Stages

A pipeline stage is one phase in the automated sequence that runs when you push code. Stages enforce order — the next stage starts only after all jobs in the current stage succeed. This topic goes deep into how stages work, how to design them well, and how to handle failures.

How Stages Control Job Order

  stages:
    - build
    - test
    - security
    - deploy

  Flow:
  ┌─────────┐    ┌────────────────┐    ┌──────────┐    ┌─────────┐
  │  BUILD  │ →  │      TEST      │ →  │ SECURITY │ →  │ DEPLOY  │
  │         │    │ unit  │ lint   │    │  scan    │    │ staging │
  │compile  │    │ tests │ check  │    │          │    │         │
  └─────────┘    └────────────────┘    └──────────┘    └─────────┘
   1 job           2 jobs (parallel)     1 job           1 job

  If any job fails → the pipeline stops → next stage does not run

Parallel Jobs Inside a Stage

Every job assigned to the same stage runs at the same time, in parallel. This cuts total pipeline time significantly on large projects.

  stage: test
  ─────────────────────────────────────────────────────
  Job A: unit-tests       runs at the same time ──┐
  Job B: integration-tests                        │ parallel
  Job C: lint-check       runs at the same time ──┘

  WITHOUT parallel:  8 min + 6 min + 2 min = 16 min total
  WITH parallel:     max(8, 6, 2) = 8 min total ✅

A Real-World Pipeline Design

Here is a pipeline that a typical web application team might use:

  stages:
    - install
    - build
    - test
    - security
    - package
    - deploy

  install-dependencies:
    stage: install
    script:
      - npm ci
    cache:
      paths:
        - node_modules/

  build-app:
    stage: build
    script:
      - npm run build
    artifacts:
      paths:
        - dist/

  unit-tests:
    stage: test
    script:
      - npm run test:unit

  e2e-tests:
    stage: test
    script:
      - npm run test:e2e

  dependency-scan:
    stage: security
    script:
      - npm audit --audit-level=high

  build-docker-image:
    stage: package
    script:
      - docker build -t myapp:$CI_COMMIT_SHORT_SHA .

  deploy-staging:
    stage: deploy
    script:
      - ./deploy.sh staging
    only:
      - main

Predefined CI/CD Variables

GitLab automatically injects dozens of variables into every pipeline job. You can use these in your scripts without defining them yourself.

Caching — Speeding Up Repeated Steps

A cache stores files between pipeline runs so you do not download or install the same dependencies every time. Unlike artifacts, caches are reused across multiple pipeline runs.

  Cache vs Artifact — Key Difference:
  ────────────────────────────────────────────────────────
  Cache:    Saved between pipeline runs (e.g. node_modules)
            Goal: make pipelines faster
  Artifact: Passed between jobs IN the same pipeline (e.g. dist/)
            Goal: share job output with the next stage
  ────────────────────────────────────────────────────────

  install-dependencies:
    script:
      - npm ci
    cache:
      key: "$CI_COMMIT_REF_SLUG"   ← unique cache per branch
      paths:
        - node_modules/

Needs — Breaking the Stage Order

By default, every job waits for all jobs in the previous stage to finish. The needs keyword lets a job start as soon as its specific dependencies finish, skipping the wait for other jobs in earlier stages.

  WITHOUT needs (default):
  build (2 min) → ALL tests finish (8 min) → deploy starts

  WITH needs:
  build (2 min) → deploy starts immediately (doesn't wait for tests)

  deploy-staging:
    stage: deploy
    needs:
      - job: build-app
        artifacts: true    ← start right after build-app, skip waiting
    script:
      - ./deploy.sh staging

Allow Failure — Non-Blocking Jobs

Mark a job with allow_failure: true so that if it fails, the pipeline continues instead of stopping. Use this for optional checks like code style warnings that should be visible but should not block deployment.

  code-style-check:
    stage: test
    script:
      - npm run lint
    allow_failure: true    ← pipeline continues even if linting fails

  Pipeline result with allow_failure:
  build ✅ → test ⚠️ (lint failed, allowed) → deploy ✅

Manual Jobs — Human Approval Before Proceeding

A manual job requires a human to click a button before it runs. Use this for production deployments that need explicit approval.

  deploy-production:
    stage: deploy
    script:
      - ./deploy.sh production
    when: manual          ← pipeline pauses here until someone clicks ▶

  Pipeline view:
  build ✅ → test ✅ → deploy-staging ✅ → deploy-production ▶ (waiting)

Only members with the Developer role or above can trigger manual jobs.

Retry — Handling Flaky Jobs

Some jobs fail due to temporary network issues, not actual bugs. The retry keyword tells GitLab to try the job again automatically before marking it as failed.

  integration-tests:
    stage: test
    script:
      - ./run-integration-tests.sh
    retry:
      max: 2              ← try up to 2 more times if it fails
      when:
        - runner_system_failure
        - stuck_or_timeout_failure

Timeout — Killing Stuck Jobs

Set a maximum duration for a job. If the job exceeds this time, GitLab cancels it and marks it as failed. This prevents a stuck job from holding up a runner indefinitely.

  long-running-tests:
    stage: test
    script:
      - ./run-full-test-suite.sh
    timeout: 30 minutes    ← cancel after 30 minutes if still running

Directed Acyclic Graph (DAG) Pipelines

A DAG pipeline uses needs throughout the entire configuration so jobs run as soon as their dependencies complete, ignoring stage boundaries entirely. This produces the fastest possible pipeline execution.

  Traditional pipeline (left-to-right stages):
  build → test A → test B → deploy    (total: 14 min)

  DAG pipeline (dependency-based):
  build ─┬─ test A ─┐
         └─ test B ─┴─ deploy         (total: 9 min)
  test A and test B run in parallel right after build

Downstream Pipelines — Triggering Other Projects

A job can trigger a pipeline in a completely different GitLab project. This is called a downstream pipeline and is useful in microservice architectures where one repo's change should trigger testing in a dependent repo.

  Project: api-server
  └── Pipeline stage: trigger
       └── Job: trigger-frontend-tests
            └── Triggers pipeline in: Project: web-frontend

  trigger-frontend-tests:
    stage: trigger
    trigger:
      project: acme/web-frontend
      branch: main