Basic Apache Kafka CLI Commands

The Kafka CLI refers to a set of tools and scripts that provide a command-line interface to an Apache Kafka cluster. The CLI tools allow us to interact with the Kafka cluster and perform administrative tasks such as creating and managing topics, sending and receiving messages, and monitoring the cluster’s health and performance.

We can also use the Kafka CLI to test, debug, and perform the audits on the cluster. We can also use them to perform the tasks on remote clusters with remote access tools such as SSH.

This tutorial provides you with some of the fundamental and basic commands for the Kafka CLI tools. This can make you better understand your system and improve efficiency without a heavy learning curve.

Kafka CLI Tools Location

Although the location of the Kafka CLI tools may differ depending on the installation method and system, they are located in the install_dir/bin directory by default. The install_dir refers to the directory where the Apache Kafka is installed.

Kafka Start Services Commands

The following commands are useful to start the various Kafka services. To start the Zookeeper service, run the following command:

bin/zookeeper-server-start.sh config/zookeeper.properties

To start the Kafka server, run the command:

bin/kafka-server-start.sh config/server.properties

Commands to Manage Your Kafka Topics

The following are some of the most common commands that you can use to manage your Kafka topics.

To create a Kafka topic, use the following command:

bin/kafka-topics.sh --create --topic <topic_name> --bootstrap-server <broker_address> --partitions <number_of_partitions> --replication-factor <number_of_replicas>

To view the details of a given Kafka topics, use the following command:

bin/kafka-topics.sh --bootstrap-server=<broker_address> --describe --topic <topic_name>

To list the topics that are available in the cluster, use the following command:

bin/kafka-topics.sh --bootstrap-server=<broker_address> --list

Kafka Publisher Commands

Once you create a topic, you can write the events using the “kafka-console-producer.sh” utilities.

To write the events to a given Kafka topic, use the following command:

bin/kafka-console-producer.sh --topic <topic_name> --bootstrap-server=<broker_address>

You can then start writing the events to the topic. Once done, you can send a SIGKILL signal to the input buffer by pressing CTRL + C.

Other options that are accepted by the “Kafka-console-producer.sh” utility include the following:

–compression-code – It specifies the message compression format such as none, gzip, snappy, or zstd.
–producer – This allows you to specify the producer properties.
–request-required-acks – This option enables you to set the acks settings.

Kafka Consumer Commands

To read the messages from a given topic, you can use the “kafka-console-consumer.sh” tool. The command is shown in the following:

bin/kafka-console-consumer.sh --topic <topic_name> --from-beginning --bootstrap-server=<broker_address>

Kafka Alter Topic

To add the partitions to an existing topic, you can use the following command:

bin/kafka-topics.sh --zookeeper=localhost:2181 --alter --topic <topic_name> --partitions <number_of_partitions>

Once you are done with a topic, you can delete it using the following command:

bin/kafka-topics.sh –bootstrap-server=<broker_address> --delete --topic <topic_name>

Kafka Consumer Group

To create a new consumer group, use the following command:

bin/kafka-consumer-groups.sh --bootstrap-server <broker_address>--create --group <topic_name> --topic <topic_name>

To stop the kafka consumer group, use the following command:

bin/kafka-consumer-groups.sh --bootstrap-server <broker_address> --group <group_name> --reset-offsets --to-offset 0 --execute

This command stops the consumer group “my-group” and resets its offsets to 0.

To list the consumer groups, use the following command:

bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list

To delete a specific consumer group, use the following command:

bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group <group_name> --delete

Conclusion

This post outlined some of the most fundamental and practical commands that you can use to work with the CLI commands in Apache Kafka. You can print this list to allow you to quickly reference the commands with ease.