Event Hub with Azure Stream Analytics

Azure Event Hub collects millions of events per second. But raw events by themselves are just numbers and timestamps. Azure Stream Analytics transforms that raw stream into meaningful, real-time insights — without storing events first. Together, Event Hub and Stream Analytics form the core of real-time data processing pipelines.

What Is Azure Stream Analytics

Azure Stream Analytics is a fully managed, real-time analytics engine. It reads event streams, applies SQL-like queries, and produces output to destinations like databases, dashboards, or storage — all in real time as events flow through.

Think of it like a factory assembly line with a quality checker standing at the conveyor belt. The checker examines each item as it passes (stream analytics reads each event), applies rules (your query logic), and sends items meeting the criteria to a separate output box (filtered results to a destination).

+------------------+      Stream       +----------------------+
|   Event Hub      |  -------------->  |  Stream Analytics    |
|                  |                   |  Job                 |
| Millions of      |                   |                      |
| events/second    |                   | SQL Query runs       |
|                  |                   | continuously against |
+------------------+                   | the live stream      |
                                       +----------+-----------+
                                                  |
                              +-------------------+-------------------+
                              |                   |                   |
                              v                   v                   v
                       Power BI            Cosmos DB           Azure Storage
                       (Dashboard)         (Database)          (Cold Storage)

Core Components of the Integration

1. Input – Event Hub

The Event Hub acts as the input stream. Stream Analytics connects to the Event Hub using a consumer group. Dedicate a consumer group for Stream Analytics — do not share the $Default group.

2. Query – Stream Analytics SQL

You write a continuous SQL query that runs indefinitely against the incoming event stream. The query can filter, aggregate, join, or detect patterns.

3. Output – Any Destination

Query results go to an output. Stream Analytics supports Power BI, Azure SQL Database, Cosmos DB, Blob Storage, Azure Data Lake, Azure Synapse, Service Bus, Event Hub (another), and more.

Setting Up the Integration

Step 1: Create a Consumer Group for Stream Analytics
  Event Hub → Consumer Groups → Add "stream-analytics-cg"
Step 2: Create a Stream Analytics Job
  Azure Portal → Stream Analytics Jobs → + New
Step 3: Add Event Hub as Input
  Job → Inputs → + Add Stream Input → Event Hub
  Settings:
    - Namespace: myNamespace
    - Event Hub: myEventHub
    - Consumer Group: stream-analytics-cg
    - Alias: "EventInput"
Step 4: Add Output (e.g., Power BI)
  Job → Outputs → + Add → Power BI
  Alias: "PowerBIOutput"
  Dataset: RealTimeMetrics
Step 5: Write the Query
  Job → Query → Write SQL
Step 6: Start the Job
  Job → Start → Choose start time

Stream Analytics Query Language

Stream Analytics uses a SQL dialect with time-based extensions. The most important concept is the time window — a way to group events over a time period.

Simple Filter Query

-- Pass only high-temperature sensor readings
SELECT
    deviceId,
    temperature,
    timestamp
INTO
    [PowerBIOutput]
FROM
    [EventInput]
WHERE
    temperature > 80

Tumbling Window – Aggregate Per Time Slot

A tumbling window groups events into non-overlapping fixed-size time slots.

-- Count orders every 1 minute
SELECT
    COUNT(*) AS OrderCount,
    SUM(amount) AS TotalRevenue,
    System.Timestamp() AS WindowEnd
INTO
    [CosmosDBOutput]
FROM
    [EventInput]
GROUP BY
    TumblingWindow(minute, 1)

Time: 00:00 ---- 00:01 ---- 00:02 ---- 00:03
       [Window 1] [Window 2] [Window 3]
Each window independently:
  Window 1: 43 orders, $8,200
  Window 2: 61 orders, $12,400
  Window 3: 28 orders, $5,100

Sliding Window – Running Average

A sliding window emits a result each time an event arrives, looking back a fixed duration.

-- Average CPU per device over the last 5 minutes (sliding)
SELECT
    deviceId,
    AVG(cpuUsage) AS AvgCPU,
    System.Timestamp() AS EventTime
INTO
    [AlertOutput]
FROM
    [EventInput]
GROUP BY
    deviceId,
    SlidingWindow(minute, 5)
HAVING
    AVG(cpuUsage) > 90

This query fires an alert whenever any device's 5-minute average CPU exceeds 90%, evaluated at every incoming event.

Hopping Window – Overlapping Aggregations

A hopping window has a fixed size but advances in smaller steps, creating overlapping windows.

-- 10-minute window, advancing every 2 minutes
SELECT
    COUNT(*) AS ErrorCount,
    System.Timestamp() AS ReportTime
INTO
    [StorageOutput]
FROM
    [EventInput]
WHERE
    eventType = 'ERROR'
GROUP BY
    HoppingWindow(minute, 10, 2)

Window 1: covers 00:00 – 00:10
Window 2: covers 00:02 – 00:12
Window 3: covers 00:04 – 00:14
(overlapping coverage for smoother trend lines)

Real-World Architecture: E-Commerce Real-Time Dashboard

Web App / Mobile App
  → Customers place orders
  → Each order fires an event to Event Hub
Event Hub "orders-hub"
  Partitions: 8
  Consumer Group for Analytics: "stream-analytics-cg"
Stream Analytics Job "OrderAnalytics"
  Input:  Event Hub "orders-hub"
  Query 1: Order count per minute → Power BI (live dashboard)
  Query 2: Revenue by region per 5 minutes → Cosmos DB
  Query 3: Failed payments in last 2 minutes > 10 → Alert Service Bus
Outputs:
  Power BI → Operations dashboard (visible to warehouse team)
  Cosmos DB → Reporting API
  Service Bus → Fraud alert handler

Joining Event Streams with Reference Data

Stream Analytics can join a live event stream with a static reference dataset stored in Azure Blob Storage.

-- Enrich telemetry events with device location from reference data
SELECT
    t.deviceId,
    t.temperature,
    r.location,
    r.building
INTO
    [EnrichedOutput]
FROM
    [EventInput] t
JOIN
    [DeviceReferenceData] r ON t.deviceId = r.deviceId

Event stream:          Reference data (Blob):
deviceId: "dev-001"    deviceId  | location  | building
temperature: 72.5  --> "dev-001" | "Floor 3" | "HQ East"
Enriched output:
deviceId: "dev-001", temperature: 72.5, location: "Floor 3", building: "HQ East"

Scaling Stream Analytics Jobs

Stream Analytics jobs scale using Streaming Units (SUs). Each SU represents a portion of processing memory and CPU.

Streaming Units	Suitable For
1–3 SUs	Low-volume testing and development
6–18 SUs	Moderate production workloads
24–192 SUs	High-throughput enterprise pipelines

Match the SU count to your Event Hub partition count for maximum parallelism. A job with 6 SUs on an Event Hub with 6 partitions achieves full parallel processing.

Monitoring the Stream Analytics Job

Metric	What to Watch
Input Events	Confirms events are flowing from Event Hub
Output Events	Confirms query is producing results
Watermark Delay	How far behind real-time the job is processing
SU % Utilization	If near 100%, add more Streaming Units
Runtime Errors	Query errors or output write failures

Summary

Azure Stream Analytics transforms the raw event stream from Event Hub into actionable insights in real time. A continuously running SQL query filters, aggregates, and enriches events as they flow. Tumbling, sliding, and hopping windows enable time-based aggregations without any storage layer. Stream Analytics outputs results to Power BI, databases, storage, or other services simultaneously. Matching Streaming Units to partition count maximizes processing parallelism.