Grafana Alert Templates
A default alert message lists labels and values in a rigid layout. Engineers who receive dozens of those messages learn to ignore them. Templates let you write messages that carry exactly the right facts in a clear shape. Grafana uses the Go templating language, and a handful of patterns cover most needs.
The Form Letter Analogy
A form letter has blank spaces for the name, the date, and the amount. Clerks fill the blanks and mail a personal-looking message. An alert template works the same way. You write the message once with placeholders, and Grafana fills them with live details for every alert.
Template: Server {{ $labels.instance }} runs at {{ $values.B.Value }}% CPU
Filled: Server web-02 runs at 91% CPU
Two Places to Use Templates
| Place | Purpose | Scope |
|---|---|---|
| Annotations and labels on a rule | Describe one alert | One rule |
| Notification templates | Shape the full message sent to a contact point | All alerts in a notification |
Templates Inside Alert Rules
Open an alert rule and scroll to the Annotations section. The summary and description fields accept template code between double curly braces.
Summary:
CPU is high on {{ $labels.instance }}
Description:
{{ $labels.instance }} in {{ $labels.region }} reached
{{ $values.B.Value }} percent CPU. Check the runbook.
Common Variables
| Variable | Contains |
|---|---|
| $labels | All labels of the alert instance |
| $labels.instance | The value of one label |
| $values.B.Value | The number produced by expression B |
| $value | A text summary of all expression values |
Format Numbers
Raw values often carry many decimals. The humanize function and the printf function tidy them.
{{ printf "%.1f" $values.B.Value }}% --> 91.3%
{{ humanize $values.B.Value }} --> 91.3
Notification Templates
A notification template shapes the complete message that reaches Slack, email, or another contact point. Create one under Alerting and Contact points, on the Notification Templates tab. Each template starts with define and ends with end.
{{ define "short.title" }}
{{ .Status | toUpper }}: {{ len .Alerts.Firing }} alert(s) firing
{{ end }}
Inside a notification template, the dot represents the whole notification. It holds the status, the list of alerts, and the labels shared by every alert.
Useful Fields
- .Status reads firing or resolved.
- .Alerts lists every alert in the notification.
- .Alerts.Firing lists only firing alerts.
- .Alerts.Resolved lists only resolved alerts.
- .CommonLabels holds labels shared by all alerts.
Loop Over Several Alerts
A single notification can bundle many alerts. The range keyword repeats a block once for each alert.
{{ define "alert.list" }}
{{ range .Alerts.Firing }}
Server: {{ .Labels.instance }}
Detail: {{ .Annotations.summary }}
{{ end }}
{{ end }}
Result with three alerts:
Server: web-01
Detail: CPU is high on web-01
Server: web-02
Detail: CPU is high on web-02
Server: web-03
Detail: CPU is high on web-03
Add Conditions
The if keyword shows text only when a rule holds.
{{ if eq .Status "firing" }}
Action needed now.
{{ else }}
The problem has cleared.
{{ end }}
Connect a Template to a Contact Point
Open the contact point settings and find the message or title field. Insert the template with the template keyword.
{{ template "short.title" . }}
{{ template "alert.list" . }}
The dot after the name passes the notification data into the template. Forget the dot and the template prints blanks.
A Complete Sample
The template below combines a title, a loop, and a condition into one readable message for a chat channel.
{{ define "team.message" }}
{{ if eq .Status "firing" }}ATTENTION{{ else }}RESOLVED{{ end }}
Alerts in this group: {{ len .Alerts }}
{{ range .Alerts }}
- {{ .Labels.alertname }} on {{ .Labels.instance }}
Summary: {{ .Annotations.summary }}
{{ end }}
{{ end }}
ATTENTION
Alerts in this group: 2
- HighCPU on web-01
Summary: CPU is high on web-01
- HighCPU on web-02
Summary: CPU is high on web-02
Readers see the status in the first line, the size of the problem in the second line, and one tidy entry per alert after that. A message built this way takes five seconds to scan.
Reuse Small Templates
Small templates keep your work tidy. Write one template for the title, one for the alert list, and one for the footer with a dashboard link. Combine them inside any contact point with the template keyword. One edit to the footer updates every message that uses it. Name templates with a dotted prefix, such as team.title, so the list stays easy to search.
Avoid Common Mistakes
- Close every if, range, and define block with its own end.
- Check label names for exact spelling, because a wrong name prints an empty value.
- Keep templates short, since some channels cut long messages.
- Avoid secrets and personal data in alert text.
Test Before You Trust
The template editor includes a preview. Use it to render your template against sample alert data. Check line breaks, missing values, and long text. Fix errors before the template reaches a live contact point.
Writing Guidelines
- Lead with what broke, add where it broke, and close with how bad it is.
- Keep the title under one line.
- Link to a dashboard or a fix guide.
- Skip jargon that new team members will not recognise.