Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Kafka 1.0 stops with FATAL SHUTDOWN error. Logs directory failed

I have just upgraded to Kafka 1.0 and zookeeper 3.4.10.At first, it all started fine. Stand - alone producer and consumer worked as expected. After I've ran my code for about 10 minutes, Kafka fails with this error:

[2017-11-07 16:48:01,304] INFO Stopping serving logs in dir C:\Kafka\kafka_2.12-1.0.0\kafka-logs (kafka.log.LogManager)

[2017-11-07 16:48:01,320] FATAL Shutdown broker because all log dirs in C:\Kafka\kafka_2.12-1.0.0\kafka-logs have failed (kafka.log.LogManager)

I have reinstalled and reconfigured Kafka 1.0 again, the same thing happened. If I try to restart, the same error occurs.

Deleting log files helps to start Kafka, but it fails again after the short run.

I have been running 0.10.2 version for a long while, and never encountered anything like this, it was very stable over the long periods of time.

I have tried to find a solution and followed instructions in the documentation.

This is not yet a production environment, it is fairly simple setup, one producer, one consumer reading from one topic.

I am not sure if this could have anything to do with zookeeper.

**Update: ** the issue has been posted at Apache JIRA board The consensus so far seems to be that it is a Windows issue.

like image 201
TeilaRei Avatar asked Nov 07 '17 22:11

TeilaRei


3 Answers

Ran into this issue as well, and only clearing the kafka-logs did not work. You'll also have to clear zookeeper.

Steps to resolve:

  1. Make sure to stop zookeeper.
  2. Take a look at your server.properties file and locate the logs directory under the following entry.

    Example:
    log.dirs=/tmp/kafka-logs/
    
  3. Delete the log directory and its contents. Kafka will recreate the directory once it's started again.

  4. Take a look at the zookeeper.properties file and locate the data directory under the following entry.

    Example:
    dataDir=/tmp/zookeeper
    
  5. Delete the data directory and its contents. Zookeeper will recreate the directory once it's started again.

  6. Start zookeeper.

    <KAFKA_HOME>bin/zookeeper-server-start.sh -daemon <KAFKA_HOME>config/zookeeper.properties
    
  7. Start the kakfa broker.

    <KAFKA_HOME>bin/kafka-server-start.sh -daemon <KAFKA_HOME>config/server.properties
    
  8. Verify the broker has started with no issues by looking at the logs/kafkaServer.out log file.

like image 192
nukalov Avatar answered Nov 04 '22 12:11

nukalov


I've tried all the solutions like

  • Clearing Kafka Logs and Zookeeper Data (issue reoccurred after creating new topic)
  • Changing log.dirs path from forward slash "/" to backward slash "\" (like log.dirs=C:\kafka_2.12-2.1.1\data\kafka ) folder named C:\kafka_2.12-2.1.1\kafka_2.12-2.1.1datakafka was created and the issue did stop and the issue was resolved.

Finally I found this link, you'll get it if you google kafka log.dirs windows

This is on Dzone you'll get it if you google kafka log.dirs windows

like image 38
Ujjwal Pathak Avatar answered Nov 04 '22 12:11

Ujjwal Pathak


Just clean the logs in C:\Kafka\kafka_2.12-1.0.0\kafka-logs and restart kafka

like image 2
nick318 Avatar answered Nov 04 '22 12:11

nick318