apache-kafkakuberneteskubectlminikubekcat

Kafka deployment on minikube


Hi i am new to kubernets, i am using minikube single node cluster for local development and testing.

Host: Ubuntu 16.04 LTS.
Minikube: Virtual box running minikube cluster

My requirement is i need to deploy kafka and zookeeper on minikube and should be used to produce or consume messages. I followed this link and successfully deployed it on minikube its details are below

$ kubectl get services
NAME            CLUSTER-IP   EXTERNAL-IP   PORT(S)                      AGE
kafka-service   10.0.0.15    <pending>     9092:30244/TCP               46m
kubernetes      10.0.0.1     <none>        443/TCP                      53m
zoo1            10.0.0.43    <none>        2181/TCP,2888/TCP,3888/TCP   50m
zoo2            10.0.0.226   <none>        2181/TCP,2888/TCP,3888/TCP   50m
zoo3            10.0.0.6     <none>        2181/TCP,2888/TCP,3888/TCP   50m

$ kubectl get pods
NAME                                      READY     STATUS    RESTARTS   AGE
kafka-deployment-3583985961-f2301         1/1       Running   0          48m
zookeeper-deployment-1-1598963595-vgx1l   1/1       Running   0          52m
zookeeper-deployment-2-2038841231-tdsff   1/1       Running   0          52m
zookeeper-deployment-3-2478718867-5vjcj   1/1       Running   0          52m

$ kubectl describe service kafka-service
Name:           kafka-service
Namespace:      default
Labels:         app=kafka
Annotations:    <none>
Selector:       app=kafka
Type:           LoadBalancer
IP:             10.0.0.15
Port:           kafka-port  9092/TCP
NodePort:       kafka-port  30244/TCP
Endpoints:      172.17.0.7:9092
Session Affinity:   None
Events:         <none>

and i have set KAFKA_ADVERTISED_HOST_NAME to minikube ip(192.168.99.100). Now for message producer i am using $cat textfile.log | kafkacat -b $(minikube ip):30244 -t mytopic its not publishing the message giving below message

% Auto-selecting Producer mode (use -P or -C to override)
% Delivery failed for message: Local: Message timed out

can any one help how to publish and consume message.


Solution

  • Going off the previous answer and your comment here is very basic sample code for a Kafka service with type=Nodeport.

    apiVersion: v1
    kind: Service
    metadata:
      name: kafka-service
      labels:
        name: kafka
    spec:
      ports:
      - port: 9092
        name: kafka-port
        protocol: TCP
      selector:
        app: kafka
      type: NodePort