Event Grid Monitoring and Diagnostics
Azure Event Grid runs silently in the background, routing events from sources to handlers. But when something goes wrong — a handler stops responding, events pile up, or deliveries fail — you need visibility into what is happening. Azure provides built-in monitoring tools that track every aspect of Event Grid behavior.
Why Monitoring Matters for Event Grid
Event Grid operates asynchronously. Unlike a direct API call where you immediately know if something failed, event delivery happens in the background. Without monitoring, a broken subscription could silently drop thousands of events before anyone notices.
Think of it like a courier company. You hand packages (events) to the courier (Event Grid) and trust them to deliver. Monitoring is the tracking system that tells you: package received, delivery attempted, delivery failed, or package returned to sender.
Key Metrics Available in Azure Monitor
Azure Monitor collects Event Grid metrics automatically. No configuration is needed to start collecting them.+------------------------------------------+ | Azure Monitor Metrics | +------------------------------------------+ | Published Events --> Events sent to | | the topic | | | | Matched Events --> Events that | | match a filter | | | | Delivered Events --> Events sent to | | a handler | | | | Failed Deliveries --> Handler rejected| | or unreachable | | | | Dead Lettered --> Moved to dead | | letter storage | +------------------------------------------+
Metric Descriptions
| Metric Name | What It Measures | Why It Matters |
|---|---|---|
| Published Events | Total events sent to a topic | Confirms sources are producing events |
| Matched Events | Events matching at least one subscription filter | Shows how many events reach the routing layer |
| Delivered Events | Events successfully sent to a handler | Confirms end-to-end delivery is working |
| Delivery Failed Events | Events that could not reach a handler | Identifies broken or misconfigured handlers |
| Dead Lettered Events | Events moved to dead letter storage | Shows events that exceeded all retry attempts |
| Dropped Events | Events with no matching subscription | Catches misconfigured filters or missing subscriptions |
| Destination Processing Duration | Time for handler to process an event | Identifies slow handlers causing timeouts |
Enabling Diagnostic Logs
Metrics give you numbers. Diagnostic logs give you details. Logs record every publish attempt, delivery attempt, and failure with timestamps and error codes.Steps to Enable Diagnostic Logs
Azure Portal
|
+--> Event Grid Topic or Domain
|
+--> Diagnostic Settings
|
+--> Add Diagnostic Setting
|
+--> Select Log Categories:
| - DeliveryFailures
| - PublishFailures
|
+--> Choose Destination:
- Log Analytics Workspace (recommended)
- Storage Account
- Event Hub
Log Analytics Workspace is the best destination because it supports querying with Kusto Query Language (KQL).
Reading Delivery Failure Logs with KQL
Once logs flow into Log Analytics, you query them directly. The table name for Event Grid logs is AzureDiagnostics.// Find all delivery failures in the last 24 hours AzureDiagnostics | where TimeGenerated > ago(24h) | where ResourceType == "EVENTGRID/TOPICS" | where OperationName == "DeliveryFailures" | project TimeGenerated, subject_s, errorCode_s, errorMessage_s | order by TimeGenerated desc
// Count failures by error code AzureDiagnostics | where OperationName == "DeliveryFailures" | summarize FailureCount = count() by errorCode_s | order by FailureCount desc
Common Error Codes in Delivery Failures
| Error Code | Meaning | Fix |
|---|---|---|
| 400 | Handler rejected the event (bad request) | Check handler code for validation errors |
| 401 | Handler authentication failed | Check webhook secret or managed identity permissions |
| 404 | Handler endpoint not found | Verify the subscription endpoint URL is correct |
| 408 | Handler timed out | Optimize handler code or increase timeout limit |
| 500 | Handler threw an internal error | Check handler logs for exceptions |
| 503 | Handler service unavailable | Check if handler service is running |
Setting Up Alerts
Viewing metrics manually is reactive. Alerts make monitoring proactive. Azure Monitor fires an alert when a metric crosses a threshold.
Useful Alert Rules for Event Grid
Alert 1: High Delivery Failure Rate +---------------------------------+ | Metric: Delivery Failed Events | | Operator: Greater Than | | Threshold: 10 per 5 minutes | | Action: Email + Slack webhook | +---------------------------------+ Alert 2: Dead Letter Events Growing +---------------------------------+ | Metric: Dead Lettered Events | | Operator: Greater Than | | Threshold: 1 per 5 minutes | | Action: PagerDuty notification | +---------------------------------+ Alert 3: No Events Published +---------------------------------+ | Metric: Published Events | | Operator: Less Than | | Threshold: 1 per 30 minutes | | Action: Email alert | | (detects silent source failure) | +---------------------------------+
Monitoring with Azure Event Grid Health
Azure Service Health shows platform-level incidents. If Event Grid itself has an outage in a region, Service Health reports it. Always check Service Health before assuming your application code has a bug.Access it at: Azure Portal → Service Health → Service Issues → Filter by Event Grid
End-to-End Monitoring Diagram
Event Source
|
| Publishes event
v
+-------------------+ Azure Monitor
| Event Grid |-------> Published Events metric
| Topic |
+--------+----------+
|
| Routes to subscriptions
v
+-------------------+ Azure Monitor
| Subscription |-------> Matched Events metric
| Filters Applied |
+--------+----------+
|
| Delivers to handler
v
+-------------------+ Azure Monitor
| Handler |-------> Delivered Events metric
| (Function, | Delivery Failed Events metric
| Logic App, |
| Webhook) |
+-------------------+
|
| On repeated failure
v
+-------------------+
| Dead Letter |-------> Dead Lettered Events metric
| Storage | + Diagnostic Logs
+-------------------+
Best Practices for Event Grid Monitoring
| Practice | Benefit |
|---|---|
| Always enable diagnostic logs for production topics | Provides full visibility into failures |
| Use Log Analytics for log destination | Enables powerful KQL queries |
| Set alerts on Delivery Failed Events | Catch broken handlers immediately |
| Set alerts on Dead Lettered Events | Detect events that exhaust retry budget |
| Monitor Published Events for source health | Detect silent failures in event producers |
| Build a dashboard in Azure Monitor Workbooks | Single view for all Event Grid health signals |
Azure Monitor Workbooks for Event Grid
Azure Monitor Workbooks let you build visual dashboards combining metrics and log queries. A good Event Grid workbook includes four panels: published events trend, delivery success rate, failure breakdown by error code, and dead letter count over time.Access Workbooks at: Azure Portal → Monitor → Workbooks → New