DynamoDB Streams
DynamoDB Streams captures every change made to items in a table and makes that change available to other systems in near-real-time. When an item is created, updated, or deleted, the stream records what happened and when. You can use this data to trigger automated actions, synchronize other systems, or build analytics pipelines.
A Simple Analogy
Think of DynamoDB Streams as a CCTV recording of your database. Every change — every insert, update, and delete — is recorded in sequence. Other systems can watch this recording and react to specific events. The table itself is not affected by who watches the stream.
What Streams Record
Each change to a table item creates a stream record. Each stream record captures:
- The event type: INSERT, MODIFY, or REMOVE
- The item's primary key
- Optionally: the full item before the change, after the change, or both
- A sequence number (ensures order of processing)
- A timestamp
Stream View Types
When you enable Streams, you choose what data each stream record contains:
| View Type | What It Includes | Use Case |
|---|---|---|
| KEYS_ONLY | Only the primary key of the changed item | You only need to know which item changed |
| NEW_IMAGE | The full item after the change | Sync the latest state to another system |
| OLD_IMAGE | The full item before the change | Audit logs, undo operations |
| NEW_AND_OLD_IMAGES | Both before and after states | Compute what changed (diff); full audit trail |
How Streams Work Internally
TABLE: Orders ↕ (every write generates a stream record) STREAM SHARD 1: Records for Partition A items STREAM SHARD 2: Records for Partition B items STREAM SHARD 3: Records for Partition C items ↕ Consumer (e.g., AWS Lambda, Kinesis Data Streams) ↕ Action: Send email, update search index, write to data warehouse
Streams are organized into shards. Each shard maps to a partition in the base table. DynamoDB creates new shards as the table partitions grow. Stream records within a shard are always in time order for items in that partition.
Stream Record Retention
Stream records are available for exactly 24 hours after they are created. After 24 hours, the records expire and are deleted from the stream. If your consumer falls behind by more than 24 hours, it will miss records.
Using Lambda to Process Streams
The most common pattern is to trigger an AWS Lambda function whenever new stream records appear. Lambda is a serverless function that runs your code in response to events. DynamoDB automatically invokes Lambda as soon as a new batch of stream records is available.
Example: New User Welcome Email
Trigger: DynamoDB Stream on Users table
Lambda function logic:
For each stream record:
IF EventName = "INSERT":
Send welcome email to NewImage.Email
IF EventName = "MODIFY" AND OldImage.Plan != NewImage.Plan:
Send plan-change confirmation email
Application writes new user → Stream fires → Lambda sends email
All automatically, without any polling or scheduled jobs
DynamoDB Streams vs Kinesis Data Streams for DynamoDB
AWS offers a second option for streaming DynamoDB changes: Kinesis Data Streams for DynamoDB. This sends the same change data to an Amazon Kinesis stream instead of a native DynamoDB stream.
| Feature | DynamoDB Streams | Kinesis Data Streams |
|---|---|---|
| Retention period | 24 hours | Up to 365 days |
| Consumers | Lambda (native), DynamoDB Streams SDK | Lambda, Kinesis consumers, analytics tools |
| Additional cost | Reads from stream billed separately | Kinesis stream costs apply |
| Best for | Simple event-driven workflows | Long retention, fan-out, advanced analytics |
Common Use Cases for DynamoDB Streams
1. Cross-Region Replication
DynamoDB Global Tables uses Streams internally to replicate changes across AWS regions. Every write in Mumbai appears in Singapore within seconds via streams.
2. Updating a Search Index
DynamoDB Table: Products ↓ stream record on every change Lambda: Reads stream → Writes to Amazon OpenSearch ↓ Search Engine: Always up-to-date product index
3. Real-Time Aggregation
Count new orders per minute, track total revenue in real time, or update a leaderboard score without querying the full table.
4. Audit Trail
Every change to sensitive data (financial records, medical data) is captured in the stream with OLD_AND_NEW_IMAGES for a full before-and-after audit trail.
Enabling Streams
You enable Streams at the table level from the AWS Console, CLI, or SDK. You can turn it on or off at any time. Enabling Streams on an existing table with data does not backfill historical changes — the stream starts capturing changes from the moment you enable it.
Summary
- DynamoDB Streams captures every INSERT, MODIFY, and REMOVE event on a table.
- Stream records are available for 24 hours.
- Use Lambda triggers to react to stream events in near real time.
- Choose a view type (KEYS_ONLY, NEW_IMAGE, OLD_IMAGE, NEW_AND_OLD_IMAGES) based on how much data your consumer needs.
- Use Kinesis Data Streams for DynamoDB when you need longer retention or fan-out to multiple consumers.