Debugging - Unable to connect to Kafka Server

Here are the most common causes why your application is not able to connect to Kafka.

1. The kafka bootstrap server port is not pingable from your client machine.

Solution: Check if you can telnet to the Kafka port from your client machine (from where you are trying to access Kafka) using the following command.
telnet <ip> <port>
Example: telnet 10.10.10.2 9092
If you are not able to telnet i.e. if your process is stuck without any response or a timeout has occurred means that the port is not accessible from your client. If successful, you will get an escape character ‘]’
So, ensure that you have the iptables rules set to following on the machine where your Kafka is running
iptables -A INPUT -p tcp --dport 9092 -j ACCEPT

2. Kafka is not running

Solution: Check if Kafka is running on the machine. Of course, this could be the most common cause and very rare to miss. You can check this by simply doing a netstat on the Kafka broker port

Example: netstat -alnp | grep :9092 where 9092 is the port where your Kafka broker is running.
If there is a process binding to the port, then Kafka is running.

Alternatively, you can also check using jps
Example: jps | grep SupportedKafka (SupportedKafka corresponds to the Kafka broker)

3. Advertised listeners

Advertised listeners are the listeners which the Kafka server exposes for the clients to connect to. The advertised listeners are nothing but the IP:Port combinations which can be accessible by the clients.

Kafka takes these advertised listeners from the server.properties file.

Solution: If you are running Kafka on a machine and client on another machine, then you need to set some advertised listeners in your kafka server.properties. The server.properties file is typically located under etc/kafka/server.properties folder of your apache-kafka folder.

In this file, add

advertised.listeners=plaintext://<kafka_server_ip>:<port>

Note that the kafka_server_ip corresponds to the IP of the machine where kafka is running on. This IP address should also be pingable from your client. You can also give the DNS name (or) hostname of the device that is pingable from the client.

Another alternate way is to add your Kafka server’s hostname (which you can get by doing cat /etc/hostname) in the hosts file of your client mapped to the IP address of your Kafka server.

For example, if the hostname is gowtham and IP is 10.10.10.10 then you should be adding

10.10.10.10 gowtham

to your hosts file (situated in /etc/hosts (linux), C:\Windows\System32\drivers\etc\hosts (windows))
Note that you would need root permissions (linux) and administrator permissions (windows) to edit this file.

​