google-cloud-platformapache-kafka

Cannot connect to kafka hosted in cloud from spring boot application


The kafka virual machine address in the Google cloud platform is xx.xx.xxx.xxx.

From my local spring boot app that usually connects to the localhost:9092 of the local machine's kafka server, I changed it to the GCP's virtual machine's ip as xx.xx.xxx.xxx:9092

But the server start up spits out the warning

2020-04-05 15:30:41.356  WARN 7968 --- [| adminclient-4] org.apache.kafka.clients.NetworkClient   : [AdminClient clientId=adminclient-4] Connection to node -1 could not be established. Broker may not be available.

and eventually times out. Should there be a different way to connect to cloud kafka from springboot application?


Solution

  • The following properties are added

    advertised.host.name in server.properties to public IP address, metadata.broker.list in producer.properties to public IP address, host.name to 0.0.0.0.

    These property files will be inside config folder. A restart is necessary.

    And it solved the problem. For broker/bootstrap.sever, the public IP address is used in the application after adding the above mentioned properties.