Spring Boot Kafka Messaging

Large systems split work across many services. An order service must tell the payment service, the stock service, and the email service that a new order exists. Direct calls tie these services tightly together, and one slow partner slows everyone. Apache Kafka solves the problem with messages. A service publishes an event once, and every interested service reads it in its own time.

Spring for Apache Kafka connects Spring Boot applications to Kafka with a template for sending and an annotation for receiving.

The Newspaper Delivery Idea

A newspaper company prints one edition and delivers copies to every subscriber. The printer never visits each home and never waits for readers to finish. Subscribers read when they have time, and new subscribers join without changing the printing process. Kafka works like that delivery system. Producers publish messages, and consumers subscribe to read them.

 Order service            Kafka                  Consumers
 (producer)           +-----------+
     |  publish       |  topic:   | ---->  Payment service
     +------------->  |  orders   | ---->  Stock service
                      +-----------+ ---->  Email service

Key Terms

Term	Meaning
Broker	A Kafka server that stores messages
Topic	A named channel, similar to a newspaper title
Partition	A slice of a topic that allows parallel reading
Producer	An application that sends messages
Consumer	An application that reads messages
Consumer group	A team of consumers that share the work of one topic
Offset	The position of a message inside a partition

How Partitions and Groups Work

 Topic: orders (3 partitions)

 Partition 0: [m1][m4][m7] ----> Consumer A  \
 Partition 1: [m2][m5][m8] ----> Consumer B   > group: stock-service
 Partition 2: [m3][m6][m9] ----> Consumer C  /

Each partition goes to one consumer inside a group. Adding consumers to the group spreads the load. A second group, such as email-service, receives its own full copy of every message.

Starting Kafka Locally

Docker runs a single-node broker for development.

services:
  kafka:
    image: apache/kafka:latest
    ports:
      - "9092:9092"

Launch it with docker compose up, and the broker listens on localhost:9092.

Adding the Dependency

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>

Configuring the Application

spring.kafka.bootstrap-servers=localhost:9092

spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.springframework.kafka.support.serializer.JsonSerializer

spring.kafka.consumer.group-id=stock-service
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.springframework.kafka.support.serializer.JsonDeserializer
spring.kafka.consumer.properties.spring.json.trusted.packages=com.estudy.shop.event

Serializers convert Java objects into bytes for storage, and deserializers reverse the process. The trusted packages property tells the JSON deserializer which classes it may create, which blocks unexpected types.

Creating a Topic

@Configuration
public class KafkaTopicConfig {

    @Bean
    public NewTopic ordersTopic() {
        return TopicBuilder.name("orders").partitions(3).replicas(1).build();
    }
}

Defining the Event

public record OrderCreatedEvent(Long orderId, String customerEmail, double total) {
}

Producing Messages

@Service
public class OrderEventProducer {

    private final KafkaTemplate<String, OrderCreatedEvent> kafkaTemplate;

    public OrderEventProducer(KafkaTemplate<String, OrderCreatedEvent> kafkaTemplate) {
        this.kafkaTemplate = kafkaTemplate;
    }

    public void publish(OrderCreatedEvent event) {
        kafkaTemplate.send("orders", String.valueOf(event.orderId()), event);
    }
}

The second argument is the message key. Messages with the same key land in the same partition, which keeps events for one order in order.

Consuming Messages

@Service
public class StockListener {

    private static final Logger log = LoggerFactory.getLogger(StockListener.class);

    @KafkaListener(topics = "orders", groupId = "stock-service")
    public void onOrderCreated(OrderCreatedEvent event) {
        log.info("Reserving stock for order {}", event.orderId());
        // reduce stock here
    }
}

Spring starts a background listener when the application boots. Every new message triggers the method.

The Full Flow

 POST /api/orders
      |
 OrderController --> OrderService saves order
      |
 OrderEventProducer publishes "orders" event
      |
 Kafka stores the event
      |
 StockListener, PaymentListener, EmailListener each react

Handling Failures

• Make listeners idempotent. A message can arrive twice, so processing the same order twice must cause no harm.
• Configure retries with a back-off delay for temporary failures.
• Send messages that keep failing to a dead letter topic for later review.
• Log the topic, partition, and offset of every failure.

When Kafka Fits

Good Fit	Weak Fit
Event streams between several services	A single small application
High message volume	One-off background jobs (use @Async)
Replaying past events	Immediate answers that a caller must wait for

Best Practices

• Name topics after business events, such as orders or payments.
• Keep messages small and include only the data consumers need.
• Choose the partition count based on expected parallel consumers.
• Version event classes carefully, because many services depend on them.
• Monitor consumer lag to detect slow consumers early.

Kafka decouples services and lets systems grow without tight wiring. Spring Boot hides most of the plumbing, so developers focus on the events themselves.