Install Apache Kafka on Ubuntu

Apache Kafka is a message broker service like ActiveMQ and RabbitMQ. You can learn more about Apache Kafka at https://kafka.apache.org  In this article, I will show you how to install Apache Kafka and verify that it’s working on Ubuntu 17.10 Artful Aardvark. Let’s get started.

Installing Apache Kafka

First update the package repository cache of your Ubuntu server with the following command:

The package repository cache should be updated.

Apache Kafka depends on Java. You can install OpenJDK 8 on Ubuntu 17.10 from the official package repository.

Run the following command to install OpenJDK 8 on Ubuntu 17.10:

Now press ‘y’ and then press <Enter> to continue.

OpenJDK 8 should be installed.

Now you have to install zookeeper. It is available in the official package repository of Ubuntu.

To install zookeeper, run the following command:

Now press ‘y’ and then press <Enter> to continue.

zookeeper should be installed.

You can run the following command to check whether zookeeper is running:

As you can see from the screenshot below, zookeeper is running.

If it’s not running on your machine for some reason, you can start zookeeper with the following command:

You should add zookeeper to the system startup. So it will start automatically on boot.

Run the following command to add zookeeper to the system startup:

zookeeper should be added to the system startup as you can see from the screenshot below.

I need netstat command which is provided by the net-tools package.

Now install net-tools package with the following command:

net-tools package should be installed.

Now you can run the following command to check whether zookeeper is running on port 2181.

As you can see it is running on port 2181.

Now run the following command to navigate to the ~/Downloads/ directory in your users HOME directory:

Now you have to Download Apache Kafka. At the time of writing, the latest version of Apache Kafka is v1.0.0.

Run the following command to download Apache Kafka 1.0.0 with wget:

Apache Kafka compressed archive should be downloaded.

As you can see from the output of ls command, the filename of Apache Kafka archive is kafka_2.12-1.0.0.tgz

Now create a directory Kafka/ in the /opt directory with the following command:

A directory /opt/Kafka should be created. This is where I will extract the archive.

Now extract the Apache Kafka archive in the /opt/Kafka directory with the following command:

The archive should be extracted.

You can run the following command to check the directory name:

Note that directory name.

Now open /etc/profile with the following command:

Add the following lines to the end of the file and save it.

Now open the ~/.bashrc file with the following command:

Add the marked line to the end of the file as save it.

Now restart your computer with the following command:

Once your computer starts, you can check whether the environment variables are added as follows:

Now for making our life easier, make a symbolic link of Kafka server.properties file as follows:

As you can see from the screenshot below, the symbolic link is created.

Now you can start Apache Kafka server with the following command:

Apache Kafka server should be started.

Testing Apache Kafka Server

You can create a test Topic testing on Apache Kafka server with the following command:

A testing topic should be created as shown in the screenshot below.

Now run the following command to use Kafka Producer API to send some message to the testing topic:

Once you press <Enter> you should see a new arrow (>) sign as shown in the screenshot below.

Just type in something and press <Enter> to start a new line. I typed in 3 lines of texts.

Now you can use the Kafka Consumer API to fetch the messages/lines from the testing topic with the following command:

You should be able to see the messages or lines you have written using the Producer API as marked in the screenshot below.

If you write a new message using the Producer API, it should also be displayed instantly on the Consumer side as shown in the screenshot below.

So that’s how you install Apache Kafka and verify its working on Ubuntu 17.10 Artful Aardvark. Thanks for reading this article.