windowsapache-kafka

How to Set Up Apache Kafka 2.13-4.0.0 on Windows 11?

The Problem

When starting Apache Kafka server on Windows, I encountered the error:

C:\Users\me\Software\Apache\Kafka\kafka_2.13-4.0.0>bin\windows\kafka-server-start.bat config\server.properties
The input line is too long.
The syntax of the command is incorrect.

C:\Users\me\Software\Apache\Kafka\kafka_2.13-4.0.0>

I resolved this by moving my Kafka directory to a shorter path, C:\kafka. However, I'd like to know if there's a way to keep a longer Kafka installation directory path, as I'd like to organize my software files into a dedicated directory.

Then, I tried to start Kafka server again, but encountered another error:

[2025-06-19 19:07:30,915] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
java.lang.RuntimeException: No readable meta.properties files found.
        at org.apache.kafka.metadata.properties.MetaPropertiesEnsemble.verify(MetaPropertiesEnsemble.java:480) ~[kafka-metadata-4.0.0.jar:?]
        at kafka.server.KafkaRaftServer$.initializeLogDirs(KafkaRaftServer.scala:141) ~[kafka_2.13-4.0.0.jar:?]
        at kafka.server.KafkaRaftServer.<init>(KafkaRaftServer.scala:56) ~[kafka_2.13-4.0.0.jar:?]
        at kafka.Kafka$.buildServer(Kafka.scala:68) ~[kafka_2.13-4.0.0.jar:?]
        at kafka.Kafka$.main(Kafka.scala:75) [kafka_2.13-4.0.0.jar:?]
        at kafka.Kafka.main(Kafka.scala) [kafka_2.13-4.0.0.jar:?]

Also, I received the warning:

DEPRECATED: A Log4j 1.x configuration file has been detected, which is no longer recommended.
To use a Log4j 2.x configuration, please see https://logging.apache.org/log4j/2.x/migrate-from-log4j1.html#Log4j2ConfigurationFormat for details about Log4j configuration file migration.

Version information:
- Windows 11
- Apache Kafka 2.13-4.0.0

How This Differs from Related Questions

Apache Kafka 2.13-4.0.0 marks a full transition to KRaft mode, removing ZooKeeper entirely. Most existing posts and answers involve using Kafka with Zookeeper, which makes them much less relevant to version 4.0.0. I believe that this question will serve as a helpful entry point for others facing similar problems with this newer setup for finding modern, Windows-specific solutions that are otherwise hard to find.

What I've Tried

I set up Kafka according to the instructions in https://www.geeksforgeeks.org/installation-guide/how-to-install-and-run-apache-kafka-on-windows/.

I changed log.dirs property in kafka/config/server.properties to:

log.dirs=C:/kafka/kafka-logs

I started Kafka server by running the following command in command prompt:

bin\windows\kafka-server-start.bat config\server.properties

To resolve the no readable meta.properties files found error, I followed the answer in https://stackoverflow.com/a/78756656/23512133, but then encountered another error:

2025-06-19T11:43:40.683429100Z main ERROR Reconfiguration failed: No configuration found for 'c387f44' at 'null' in 'null'
y6cBC6pfR8eo9tT0-e35Qw
Solution

  • How to Set Up Apache Kafka 4.0.0 for Windows

    Note: in some of the paths written below, I will be writing the path to your Kakfa installation directory as kafka\. Replace it with the path where you placed your Kafka installation directory (e.g., C:\kafka).

    1. Prerequisites

    You need to have Java Development Kit (JDK) installed. Ensure that your JDK's bin directory (e.g., C:\java\bin) has been added to your PATH environment variable.

    2. Download and Install Kafka 4.0.0

    This section provides instructions for downloading and installing Kafka on Windows.

    1. Download Kafka from https://kafka.apache.org/downloads. Look for the section that matches the version number of interest, then choose the binary download.
    2. Extract the downloaded files and place your Kafka directory in a convenient location, like C:\kafka.

    3. Edit kafka-run-class.bat to Fix Errors

    This section provides instructions for editing kafka-run-class.bat (in kafka\bin\windows\) to prevent the input line is too long error and the DEPRECATED: A Log4j 1.x configuration file has been detected warning.

    Consider creating a backup file kafka-run-class.bat.backup before proceeding.

    1. If you have placed your Kakfa installation directory in a path longer than C:\kafka, you would most likely need to edit kafka-run-class.bat to prevent the input line is too long error:

      1. In kafka-run-class.bat, replace the following lines (originally at lines 92-95):

        rem Classpath addition for release
for %%i in ("%BASE_DIR%\libs\*") do (
   call :concat "%%i"
)

        With the following lines:

        rem Classpath addition for release
call :concat "%BASE_DIR%\libs\*;"

      2. Restart Command Prompt if it was open.

    2. To prevent the DEPRECATED: A Log4j 1.x configuration file has been detected warning:

      In kafka-run-class.bat, replace the following lines (originally at lines 117-123):

      rem Log4j settings
IF ["%KAFKA_LOG4J_OPTS%"] EQU [""] (
    set KAFKA_LOG4J_OPTS=-Dlog4j2.configurationFile=file:%BASE_DIR%/config/tools-log4j2.yaml
) ELSE (
    rem Check if Log4j 1.x configuration options are present in KAFKA_LOG4J_OPTS
    echo %KAFKA_LOG4J_OPTS% | findstr /r /c:"log4j\.[^ ]*(\.properties|\.xml)$" >nul
    IF %ERRORLEVEL% == 0 (

      With:

      rem Log4j settings
setlocal enabledelayedexpansion
IF ["%KAFKA_LOG4J_OPTS%"] EQU [""] (
  set KAFKA_LOG4J_OPTS=-Dlog4j2.configurationFile=file:%BASE_DIR%/config/tools-log4j2.yaml
) ELSE (
  rem Check if Log4j 1.x configuration options are present in KAFKA_LOG4J_OPTS
  echo %KAFKA_LOG4J_OPTS% | findstr /r /c:"log4j\.[^ ]*(\.properties|\.xml)$" >nul
  IF !ERRORLEVEL! == 0 (

      Note the key changes:

      1. Added setlocal enabledelayedexpansion
      2. Changed %ERRORLEVEL% to !ERRORLEVEL!

      Additional information:

      • There is an error in the logic due to the order of expansion of environment variables in batch files.
      • Environment variables surrounded by % are expanded when the line is parsed, not when it's executed.
      • This means that while %ERRORLEVEL% is being changed dynamically at runtime, it does not expand to the updated value.
      • %ERRORLEVEL% was expected to expand to 1 due to the command echo %KAFKA_LOG4J_OPTS% | findstr /r /c:"log4j\.[^ ]*(\.properties|\.xml)$" >nul not finding a match
      • However, %ERRORLEVEL% expands to 0 instead of 1. %ERRORLEVEL% == 0 wrongly evaluates to true, causing the code in the IF !ERRORLEVEL! == 0 block to run, which includes printing the DEPRECATED: A Log4j 1.x configuration file has been detected warning.
      • To fix this issue, delayed expansion must be used, as shown in the instructions above.

    4. Configure Kafka for Running in KRaft Mode

    This section provides instructions for setting the log.dirs property in server.properties (in kafka\config\).

    This section also provides instructions for setting the controller.quorum.voters property in server.properties and formatting the storage directory for running Kafka in KRaft mode, to prevent the reconfiguration failed: no configuration found and no readable meta.properties files found errors.

    Consider creating a backup file server.properties.backup before proceeding.

    1. In server.properties, replace the following line (originally at line 73):

      log.dirs=/tmp/kraft-combined-logs

      With the following line:

      log.dirs=path/to/kafka/kraft-combined-logs

      Replace path/to/kafka/ with the path to your Kafka installation directory. Use "/" instead of "\" in the path to avoid escape issues and ensure compatibility.

    2. In server.properties, add the following lines to the bottom of the "Server Basics" section (originally at line 16 to 25):

      # Define the controller quorum voters for KRaft mode
controller.quorum.voters=1@localhost:9093

      This is for a single-node Kafka cluster. For a multi-node Kafka cluster, list multiple entries like:

      controller.quorum.voters=1@host1:9093,2@host2:9093,3@host3:9093

    3. In Command Prompt, temporarily set the KAFKA_LOG4J_OPTS environment variable to prevent the reconfiguration failed: no configuration found error by running the command:

      set "KAFKA_LOG4J_OPTS=-Dlog4j.configurationFile=file:///path/to/kafka/config/log4j2.yaml"

      Replace path/to/kafka/ with the path to your Kafka installation directory. Use "/" instead of "\" in the path to avoid escape issues and ensure compatibility.

    4. In Command Prompt, change directory to your Kafka installation directory, then generate a unique cluster ID by running the command:

      bin\windows\kafka-storage.bat random-uuid

    5. In Command Prompt, use the generated cluster ID to format your Kafka storage directory to prevent the no readable meta.properties files found error by running the command:

      bin\windows\kafka-storage.bat format -t <generated UUID> -c config\server.properties

      Replace <generated UUID> with the ID generated in step 4.

    5. Start Kafka

    This section provides instructions to start Kafka and verify that it is working correctly.

    1. In Command Prompt, change directory to your Kafka installation directory, then start Kafka by running the command:

      bin\windows\kafka-server-start.bat config\server.properties

    2. Launch a second Command Prompt window and verify that your Kafka server is working by following these steps:

      1. Change directory to your Kafka installation directory, then temporarily set the KAFKA_LOG4J_OPTS environment variable by running the command:

        set "KAFKA_LOG4J_OPTS=-Dlog4j.configurationFile=file:///path/to/kafka/config/log4j2.yaml"

        Replace path/to/kafka/ with the path to your Kafka installation directory. Use "/" instead of "\" in the path to avoid escape issues and ensure compatibility.

      2. Create a topic called test-topic by running the command:

        bin\windows\kafka-topics.bat --create --topic test-topic --bootstrap-server localhost:9092

      3. Produce a test message to test-topic:
        Start a producer for test-topic by running the command:

        bin\windows\kafka-console-producer.bat --topic test-topic --bootstrap-server localhost:9092

        Write a test message and press Enter.
        Press Ctrl+C to stop the producer.

      4. Consume a message from test-topic:
        Start a consumer for test-topic by running the command:

        bin\windows\kafka-console-consumer.bat --topic test-topic --from-beginning --bootstrap-server localhost:9092

        You should see the test message that you entered earlier being printed out.

      5. If you successfully produced and consumed a test message, Kafka is working correctly.