Jenkins Performance Tuning and Best Practices

A poorly configured Jenkins server becomes slow, unstable, and unreliable as your team grows and the number of pipelines increases. Performance tuning adjusts Jenkins and its Java runtime to handle more work efficiently. Best practices make pipelines easier to maintain, faster to run, and safer to operate.

Think of performance tuning as servicing a vehicle fleet. Each individual vehicle (pipeline) needs to be efficient. The garage (Jenkins server) needs to be the right size for the fleet. And the mechanics (admins) need clear procedures so nothing gets neglected.

JVM Memory Tuning

Jenkins runs on the Java Virtual Machine (JVM). The JVM has a fixed memory limit by default. As plugins, jobs, and build history grow, Jenkins needs more memory to avoid slowness and crashes.

Default JVM memory settings:
  -Xms256m   (start with 256 MB heap)
  -Xmx512m   (maximum heap 512 MB)

These are too small for a team with 50+ jobs.

Recommended settings for medium-sized Jenkins:
  -Xms1g     (start with 1 GB)
  -Xmx4g     (maximum 4 GB)

For large Jenkins (200+ jobs, many agents):
  -Xms2g
  -Xmx8g

Set JVM arguments on Linux (Ubuntu/Debian):
  Edit: /etc/default/jenkins
  Add or modify:
    JAVA_ARGS="-Xms1g -Xmx4g -XX:+UseG1GC"

Or with systemd override:
  sudo systemctl edit jenkins
  [Service]
  Environment="JAVA_OPTS=-Xms1g -Xmx4g -XX:+UseG1GC"
  sudo systemctl restart jenkins

Signs Jenkins needs more memory:
  - Dashboard takes 5+ seconds to load
  - Builds queue up even with available agents
  - Jenkins UI becomes unresponsive during heavy builds
  - OutOfMemoryError appears in system logs
  - Frequent garbage collection pauses (check logs)

Garbage Collector Selection

The JVM has multiple garbage collectors. The G1GC (Garbage First Garbage Collector) performs well for Jenkins because it handles large heaps with short pause times.

Recommended JVM flags for Jenkins:
  -XX:+UseG1GC
  -XX:MaxGCPauseMillis=200
  -XX:+ExplicitGCInvokesConcurrent

These settings tell the JVM to keep GC pauses under 200 milliseconds,
which keeps the Jenkins UI responsive even during heavy builds.

Limiting Executors on the Controller

As covered in the architecture topic, the Jenkins controller should not run builds. Build jobs consume CPU and memory on the controller, which competes with the UI and scheduling tasks.

Set controller executors to zero:
  Manage Jenkins → Nodes → Built-In Node → Configure
  Number of executors: 0
  Save

All builds will now wait for an available agent.
The controller stays responsive for UI and scheduling.

Parallel Stages for Faster Pipelines

Sequential pipelines are slow. Identify stages that do not depend on each other and run them in parallel.

Sequential pipeline (slow):
  Build: 2min → Unit Tests: 5min → Integration Tests: 5min → Lint: 3min
  Total: 15 minutes

Parallel pipeline (faster):
  Build: 2min → [Unit Tests: 5min] + [Integration Tests: 5min] + [Lint: 3min]
                        (all three run at the same time)
  Total: 7 minutes

  stage('Verify') {
      parallel {
          stage('Unit Tests')        { steps { sh 'mvn test' } }
          stage('Integration Tests') { steps { sh 'mvn verify -Pit' } }
          stage('Lint')              { steps { sh 'mvn checkstyle:check' } }
      }
  }

Workspace Cleanup

Build workspaces accumulate large files over time and fill up disk on agents. Clean workspaces as part of every pipeline.

Clean at end of every build:
  post {
      always {
          cleanWs()
      }
  }

Clean before checkout (avoid stale files):
  options {
      skipDefaultCheckout(true)
  }
  stages {
      stage('Checkout') {
          steps {
              cleanWs()
              checkout scm
          }
      }
  }

Limit build history per job:
  options {
      buildDiscarder(logRotator(
          numToKeepStr: '20',    // Keep last 20 builds
          daysToKeepStr: '30',   // Or builds older than 30 days
          artifactNumToKeepStr: '5'  // Keep artifacts for last 5 builds
      ))
  }

Pipeline Performance Best Practices

Avoid Heavy Logic on the Controller

Groovy code in a pipeline runs on the Jenkins controller. Heavy loops, large string processing, and blocking calls in pipeline code slow down the controller for all other jobs.

Bad: Processing large data in Groovy (runs on controller)
  script {
      def lines = readFile('big-log-file.txt').split('\n')
      for (line in lines) {
          // Processing 100,000 lines in Groovy on the controller
      }
  }

Good: Move heavy processing to a shell script (runs on agent)
  sh '''
    awk '/ERROR/ {print $0}' big-log-file.txt > errors.txt
    wc -l errors.txt
  '''

Use Declarative Pipeline Over Scripted

Declarative Pipeline is more efficient because Jenkins can validate and optimize it before running. Scripted Pipeline executes Groovy line by line, which is slower for complex logic.

Cache Dependencies

Re-downloading Maven, npm, or pip packages on every build wastes time and bandwidth. Cache the dependency directory between builds.

Cache Maven dependencies:
  agent {
      docker {
          image 'maven:3.9-eclipse-temurin-17'
          args '-v /var/jenkins_home/.m2:/root/.m2'
      }
  }

Cache npm dependencies (without Docker):
  steps {
      // Use npm ci with a pre-populated node_modules cache
      sh 'npm ci --prefer-offline'
  }

Build time impact:
  Without caching: 8 minutes (downloads 200 MB each time)
  With caching:    2 minutes (only downloads changed packages)

Plugin Management Best Practices

Keep plugins updated:
  Review updates monthly
  Test updates on staging Jenkins before production
  Read changelogs for breaking changes

Remove unused plugins:
  Unused plugins slow Jenkins startup
  Each plugin adds memory overhead
  Removed plugins reduce attack surface

Pin critical plugins to specific versions:
  If an update breaks something, pin to the
  last working version while investigating

Avoid plugin sprawl:
  Question every new plugin request
  One plugin that does 3 things is better than 3 separate plugins
  Some functionality is better handled in shell scripts or Shared Libraries

Monitoring Jenkins Performance

Install: Monitoring plugin
Access: Manage Jenkins → Monitoring (JavaMelody)

Key metrics to watch:
  HTTP sessions:     How many users are logged in
  Heap memory:       Should stay below 80% of max
  CPU usage:         Sustained high CPU = pipeline or plugin issue
  Build queue size:  Growing queue = not enough agents

Alert thresholds:
  Heap > 80%         → Increase -Xmx or investigate memory leak
  Queue > 20 builds  → Add more agents
  Build time growing → Review for new slow steps or missing caches

Jenkins system log (real-time):
  Manage Jenkins → System Log → All Jenkins Logs
  Filter for WARNING and SEVERE entries

Jenkins High Availability

For teams that cannot tolerate Jenkins downtime, high availability (HA) keeps Jenkins accessible even when one server fails.

Basic HA approaches:

  1. Active-Passive with shared storage:
     - Two Jenkins controllers share a network file system (NFS/EFS)
     - Only one is active at a time
     - If active fails, passive takes over (manual or automated switchover)

  2. Regular automated backups + fast restore:
     - Back up JENKINS_HOME every hour
     - If controller fails, spin up a new instance and restore
     - Achieves ~15 minute recovery time objective (RTO)

  3. CloudBees CI (Enterprise Jenkins):
     - Commercial product built on Jenkins
     - Provides true active-active HA
     - Suitable for very large organisations

For most teams, option 2 (regular backup + fast restore) provides
sufficient resilience without complex infrastructure.

Jenkins Pipeline Design Best Practices Summary

Pipeline code quality:
  ✓ Store Jenkinsfile in Git alongside application code
  ✓ Use Declarative Pipeline as the default syntax
  ✓ Keep stages focused — one purpose per stage
  ✓ Use Shared Libraries for code reused across jobs
  ✓ Never hardcode secrets — use Jenkins Credentials
  ✓ Set timeouts on every pipeline and long-running stage

Performance:
  ✓ Run stages in parallel where possible
  ✓ Cache dependency downloads (Maven .m2, npm node_modules)
  ✓ Clean workspace after builds
  ✓ Limit build history with buildDiscarder
  ✓ Move heavy processing to shell scripts, not Groovy

Reliability:
  ✓ Set controller executors to 0 — builds on agents only
  ✓ Use retry() for flaky network operations
  ✓ Add health check steps after deployments
  ✓ Build rollback steps into every deployment pipeline
  ✓ Test pipeline changes on non-production branches first

Security:
  ✓ Enable security — never run Jenkins unsecured
  ✓ Use role-based access control for large teams
  ✓ Keep Jenkins and plugins updated monthly
  ✓ Back up JENKINS_HOME daily
  ✓ Run Jenkins behind HTTPS and a firewall

Key Points

• Tune JVM memory with -Xms and -Xmx flags — Jenkins needs at least 2–4 GB heap for a medium-sized team.
• Set controller executors to zero so build jobs never compete with Jenkins UI and scheduling.
• Run independent stages in parallel to cut pipeline duration significantly.
• Cache dependency directories (Maven, npm) to avoid re-downloading packages on every build.
• Monitor heap memory, build queue size, and average build duration to catch performance problems before they impact the team.