Event Hub Schema Registry

When multiple producers send events to an Event Hub, consumers need to know what format and structure each event follows. Without a shared agreement on structure, consumers must guess or hardcode assumptions that break when producers change their event format. Azure Event Hub Schema Registry solves this problem by storing event schemas centrally and linking each event to its schema version.

What Is a Schema Registry

A schema registry is a shared catalog of event structure definitions. Producers register a schema before sending events. Consumers look up the schema to understand how to read incoming events. Both sides agree on the contract without direct communication.

Think of it like a filing cabinet at a government office. Every form has a form number (schema ID). Citizens fill out form 4001-B (producer). The clerk knows that form 4001-B has specific fields in a specific layout (schema). No one needs to explain the structure each time because the registry holds it.

+------------------+         +---------------------+
|   Producer App   |         |   Schema Registry   |
|                  | ------> |                     |
| 1. Register or   |         | Schema: OrderEvent  |
|    look up schema|         | Version: 1          |
| 2. Serialize     | <------ | Schema ID: abc123   |
|    event using   |         +---------------------+
|    schema        |
| 3. Embed schema  |
|    ID in event   |
+--------+---------+
         |
         | Event with schema ID embedded
         v
+------------------+
|    Event Hub     |
+--------+---------+
         |
         v
+------------------+         +---------------------+
|   Consumer App   |         |   Schema Registry   |
|                  | ------> |                     |
| 1. Read event    |         | Fetch schema abc123 |
| 2. Extract       | <------ | Returns schema def  |
|    schema ID     |         +---------------------+
| 3. Deserialize   |
|    using schema  |
+------------------+

Schema Formats Supported

Azure Event Hub Schema Registry supports Apache Avro schema format as the primary format. JSON Schema support is also available.

Apache Avro

Avro is a binary serialization format. Events serialized with Avro are compact (smaller than JSON) and fast to serialize and deserialize. Avro schemas define field names, types, and optional default values.

Example Avro Schema for an OrderEvent:
{
  "type": "record",
  "name": "OrderEvent",
  "namespace": "com.myapp.events",
  "fields": [
    { "name": "orderId",    "type": "string" },
    { "name": "customerId", "type": "string" },
    { "name": "amount",     "type": "double" },
    { "name": "currency",   "type": "string", "default": "USD" },
    { "name": "timestamp",  "type": "long" }
  ]
}

Schema Groups

Schemas in the registry are organized into schema groups. A schema group is a logical container for related schemas.

Schema Registry
  |
  +-- Schema Group: "ecommerce"
  |     |-- OrderEvent (versions 1, 2, 3)
  |     |-- PaymentEvent (versions 1, 2)
  |     |-- ShipmentEvent (version 1)
  |
  +-- Schema Group: "iot-telemetry"
  |     |-- TemperatureReading (version 1)
  |     |-- PressureReading (version 1)
  |
  +-- Schema Group: "user-activity"
        |-- LoginEvent (versions 1, 2)
        |-- LogoutEvent (version 1)

Compatibility Modes per Schema Group

Each schema group enforces a compatibility mode. This controls what changes producers are allowed to make when creating a new schema version.

Compatibility Mode	What It Allows	Best For
None	Any change allowed between versions	Development environments
Backward	New schema can read data written with old schema	Consumers upgrade before producers
Forward	Old schema can read data written with new schema	Producers upgrade before consumers
Full	Both backward and forward compatible	Safest, most restrictive

Schema Versioning in Practice

Producers evolve their event structure over time. Schema versioning tracks each change while maintaining compatibility rules.

OrderEvent Version 1:
{
  "orderId": "string",
  "amount": "double"
}
Producer adds a new optional field → creates Version 2:
{
  "orderId": "string",
  "amount": "double",
  "currency": { "type": "string", "default": "USD" }  ← new field with default
}
Backward compatible: consumers on V1 schema can read V2 events
  → they simply ignore the unknown "currency" field or use the default

Creating a Schema Registry and Schema Group

Via Azure Portal

Azure Portal
  → Event Hub Namespace
    → Schema Registry (left menu)
      → + Schema Group
        → Name: "ecommerce"
        → Serialization Type: Avro
        → Compatibility: Backward
        → Create

Via Azure CLI

az eventhubs namespace schema-group create \
  --resource-group myResourceGroup \
  --namespace-name myNamespace \
  --name ecommerce \
  --schema-compatibility Backward \
  --schema-type Avro

Registering a Schema

Via Azure SDK (C#)

var client = new SchemaRegistryClient(
    "myNamespace.servicebus.windows.net",
    new DefaultAzureCredential());
string schemaContent = @"
{
  ""type"": ""record"",
  ""name"": ""OrderEvent"",
  ""fields"": [
    { ""name"": ""orderId"",    ""type"": ""string"" },
    { ""name"": ""amount"",     ""type"": ""double"" }
  ]
}";
SchemaProperties props = await client.RegisterSchemaAsync(
    "ecommerce",           // group name
    "OrderEvent",          // schema name
    schemaContent,
    SchemaFormat.Avro);
Console.WriteLine($"Schema ID: {props.Id}");

The schema ID returned is a short GUID that gets embedded into event payloads to link each event to its schema.

Producing Events with Schema Registry

// Create an encoder using the schema registry
var encoder = new EventDataEncoder(schemaRegistryClient, "ecommerce");
// Create the event object matching the schema
var order = new GenericRecord(schema)
{
    ["orderId"] = "ORD-9001",
    ["amount"] = 149.99
};
// Encode encodes the object AND embeds the schema ID
EventData eventData = await encoder.EncodeAsync(order, "OrderEvent");
// Send to Event Hub
await producerClient.SendAsync(new[] { eventData });

The encoded event payload is compact binary (Avro) with the schema ID prepended as a small header.

Consuming Events with Schema Registry

// Create a decoder using the same schema registry
var decoder = new EventDataDecoder(schemaRegistryClient);
// When an event arrives:
processor.ProcessEventAsync += async (args) =>
{
    // Decoder reads schema ID from event, fetches schema, deserializes
    GenericRecord record = await decoder.DecodeAsync<GenericRecord>(args.Data);
    string orderId = record["orderId"].ToString();
    double amount  = (double)record["amount"];
    Console.WriteLine($"Order {orderId}: ${amount}");
    await args.UpdateCheckpointAsync();
};

The decoder automatically fetches the schema from the registry using the embedded schema ID. Consumers do not need to hardcode schemas.

Benefits of Schema Registry

Benefit	Explanation
Schema enforcement	Producers cannot send structurally invalid events
Smaller events	Avro binary is far more compact than JSON
Backward/forward compatibility	Schema evolution does not break existing consumers
Self-describing events	Schema ID in each event tells consumers how to read it
Centralized governance	One place to view all event contracts across all teams
Reduced coupling	Producers and consumers agree via registry, not direct code sharing

Schema Registry Availability by Tier

Tier	Schema Registry Available
Basic	No
Standard	Yes
Premium	Yes
Dedicated	Yes

Summary

Schema Registry provides a central catalog for event structure definitions. Producers register schemas before sending. Consumers fetch schemas to deserialize events correctly. Schema groups organize related schemas and enforce compatibility rules. Avro serialization reduces event size significantly compared to JSON. Versioning lets schemas evolve over time without breaking existing consumers. Schema Registry is available on Standard, Premium, and Dedicated tiers of Azure Event Hub.

Previous lesson

Back to course

Next lesson