Apache Kafka Tutorial

By Dhiraj , 20 March, 2018 103K

In this tutorial, we will be developing a sample apache kafka java application using maven. We will be configuring apache kafka and zookeeper in our local machine and create a test topic with multiple partitions in a kafka broker.We will have a seperate consumer and producer defined in java that will produce message to the topic and also consume message from it.We will also take a look into how to produce messages to multiple partitions of a single topic and how those messages are consumed by consumer group....

By Dhiraj , 23 March, 2018 69K

In this tutorial series, we will be discussing about how to stream log4j application logs to apache Kafka using maven artifact kafka-log4j-appender.To keep application logging configuration simple, we will be doing spring boot configurations and stream log4j logs to apache Kafka....

By Dhiraj , 27 March, 2018 60K

In this tutorial, we will be setting up apache Kafka, logstash and elasticsearch to stream log4j logs directly to Kafka from a web application and visualize the logs in Kibana dashboard.Here, the application logs that is streamed to Kafka will be consumed by logstash and pushed to elasticsearch. In short, we will be setting up ELK stack to work with Kafka....

By Dhiraj , 12 April, 2018 42K

In my last article, we created a sample Java and Apache Kafka subscriber and producer example.In this article, we will be using spring boot 2 feature to develop a sample Kafka subscriber and producer application.We will take a look at the use of KafkaTemplate to send messages to Kafka topics, @KafkaListener annotation to listen to those messages and @SendTo annotation to forward messages to a specified topic.We will also take a look at how to produce messages to multiple partitions of a single topic and how those messages are consumed by a consumer group along with custom message converters in spring boot....