GitLab Admin Troubleshooting
Administrators of self-managed GitLab servers fix problems that range from slow pages to failed email. A repeatable routine saves time: check service health, read logs, run built-in diagnostics, and change one thing at a time.
Think of a mechanic with a dashboard. Warning lights point to the area, and diagnostic tools reveal the exact fault. GitLab offers its own lights and tools.
The Diagnostic Routine
Problem reported
↓
1. gitlab-ctl status → Are all services running?
↓
2. Server resources → Disk, memory, CPU in range?
↓
3. Logs → What error appears at the time of failure?
↓
4. gitlab-rake gitlab:check → Do built-in checks pass?
↓
5. Fix one thing, retest
Check Service Status
sudo gitlab-ctl status
The output lists each service with its state and run time.
run: gitaly: (pid 1201) 86400s; run: log: (pid 1180) 86400s run: nginx: (pid 1305) 86400s; run: log: (pid 1301) 86400s down: sidekiq: 3s, normally up run: puma: (pid 1410) 86400s; run: log: (pid 1402) 86400s
A line that starts with "down" or shows a very short run time points to a crashing service. Restart one service with sudo gitlab-ctl restart sidekiq.
Know the Main Services
| Service | Job |
|---|---|
| nginx | Receives web requests |
| puma | Runs the main GitLab application |
| sidekiq | Processes background jobs such as email and imports |
| gitaly | Reads and writes Git repositories |
| postgresql | Stores users, issues, and settings |
| redis | Holds cache and job queues |
Read the Logs
Logs live under /var/log/gitlab. Stream all logs at once or follow one service.
sudo gitlab-ctl tail sudo gitlab-ctl tail sidekiq sudo tail -f /var/log/gitlab/gitlab-rails/production.log
| Log File | Shows |
|---|---|
| gitlab-rails/production.log | Web requests and application errors |
| sidekiq/current | Background job activity |
| nginx/gitlab_error.log | Web server errors |
| gitaly/current | Repository operations |
Run Built-In Checks
sudo gitlab-rake gitlab:check sudo gitlab-rake gitlab:env:info
The check command tests configuration, file permissions, and connections. Each line ends with a green "yes" or a red "no" and a hint for repair. The info command prints version numbers for GitLab, Ruby, and the database, which helps when you contact support.
Common Problems and Fixes
| Symptom | Common Cause | Fix |
|---|---|---|
| Error 502 after a restart | Puma needs time to start | Wait two minutes and check gitlab-ctl status |
| Error 502 that persists | Low memory or a crashed service | Check free memory and read the puma log |
| Pages load slowly | Memory pressure or slow disk | Add memory, move to faster storage |
| No emails arrive | SMTP settings or a stuck background queue | Test email and check sidekiq |
| Pipelines stay pending | No online runner | Check runner status and tags |
| Git push fails over SSH | Wrong port or key problem | Check firewall, SSH keys, and server logs |
| Disk full | Old artifacts, logs, or backups | Free space and set expiry rules |
Test Outgoing Email
Open the Rails console and send a test message.
sudo gitlab-rails console
Notify.test_email('you@example.com', 'Test subject', 'Test body').deliver_now
An error message in the output names the failing SMTP setting. Fix the values in /etc/gitlab/gitlab.rb and apply them with sudo gitlab-ctl reconfigure.
Free Disk Space
df -h # overall disk usage sudo du -sh /var/opt/gitlab/* # find large folders
- Set an expiry time for CI artifacts in the project settings and the instance settings.
- Delete old backups from /var/opt/gitlab/backups after copying them off the server.
- Clean unused container registry tags with cleanup policies.
- Rotate logs; GitLab includes log rotation by default.
Health Check Endpoints
| Address | Meaning |
|---|---|
| /-/health | The web service responds |
| /-/readiness | The server can handle traffic |
| /-/liveness | The process is alive |
Monitoring tools call these addresses from allowed IP addresses to raise alerts early.
Check the Background Queue
Open Admin Area → Monitoring → Background jobs. A queue with thousands of waiting jobs means Sidekiq runs behind. Add capacity, restart Sidekiq, or find the job type that floods the queue.
Work Safely
- Create a backup before any repair that changes data.
- Change one setting at a time and record each change.
- Avoid running unknown console commands from forums on a production server.
- Test fixes on a staging copy when possible.
- Check the official troubleshooting pages for your exact GitLab version.
Information to Collect Before Asking for Help
- GitLab version and install type.
- Exact error text and the time it appeared.
- Relevant log lines with passwords removed.
- Output of gitlab:check and gitlab:env:info.
- Recent changes such as upgrades or configuration edits.