ubuntuapache-kafkainit.d

How to automatically start Kafka upon system startup in Ubuntu?


Does Kafka have an official way (e.g. an init.d script) to start Kafka on system start up?

The only official method to start Kafka I have seen is:

nohup ~/kafka/bin/kafka-server-start.sh ~/kafka/config/server.properties > ~/kafka/kafka.log 2>&1 &

I have tried to use a @reboot task in crontab -e however it did not start Kafka. Some people have also written custom init.d

There are also custom init.d scripts available (e.g. one, two, three) however they are all different and I am not familiar enough with init.d to understand which one, if any to implement.

How to start Kafka on system startup?


Solution

  • Here's how I configure Kafka to start automatically on Ubuntu 14.04:

    sudo su
    cp -R ~/kafka_2.11-0.10.0.1 /opt
    ln -s /opt/kafka_2.11-0.10.0.1 /opt/kafka
    

    Copy the following init script to /etc/init.d/kafka:

    DAEMON_PATH=/opt/kafka/
    PATH=$PATH:$DAEMON_PATH/bin
    
    # See how we were called.
    case "$1" in
      start)
            # Start daemon.
            echo "Starting Zookeeper";
            nohup $DAEMON_PATH/bin/zookeeper-server-start.sh -daemon /$DAEMON_PATH/config/zookeeper.properties 2> /dev/null && \
            echo "Starting Kafka";
            nohup $DAEMON_PATH/bin/kafka-server-start.sh -daemon /$DAEMON_PATH/config/server.properties 2> /dev/null
            ;;
      stop)
            # Stop daemons.
            echo "Shutting down Zookeeper";
            pid=`ps ax | grep -i 'org.apache.zookeeper.server' | grep -v grep | awk '{print $1}'`
            if [ -n "$pid" ]
              then
              kill -9 $pid
            else
              echo "Zookeeper was not Running"
            fi
            echo "Shutting down Kafka";
            pid=`ps ax | grep -i 'kafka.Kafka' | grep -v grep | awk '{print $1}'`
            if [ -n "$pid" ]
              then
              kill -9 $pid
            else
              echo "Kafka was not Running"
            fi
            ;;
      restart)
            $0 stop
            sleep 2
            $0 start
            ;;
      status)
            pid=`ps ax | grep -i 'org.apache.zookeeper.server' | grep -v grep | awk '{print $1}'`
            if [ -n "$pid" ]
              then
              echo "Zookeeper is Running as PID: $pid"
            else
              echo "Zookeeper is not Running"
            fi
            pid=`ps ax | grep -i 'kafka.Kafka' | grep -v grep | awk '{print $1}'`
            if [ -n "$pid" ]
              then
              echo "Kafka is Running as PID: $pid"
            else
              echo "Kafka is not Running"
            fi
            ;;
      *)
            echo "Usage: $0 {start|stop|restart|status}"
            exit 1
    esac
    
    exit 0
    

    Make the kafka service with these commands:

    chmod 755 /etc/init.d/kafka
    update-rc.d kafka defaults
    

    Now you should be able to start and stop the kafka service like this:

    sudo service kafka start
    sudo service kafka status
    sudo service kafka stop
    

    If you want to remove the Kafka service later, run update-rc.d -f kafka remove.