apache-sparkubuntupysparkflume-ng

Deleted Apache Flume's flume.log, now errors are no longer logged after recreating


My flume.log file was getting enormous so I foolishly deleted it. It's absolute path was

/etc/apache-flume-1.8.0-bin/logs/flume.log 

I then recreated it (ie: touch flume.log) in the same directory. However nothing gets appended to it anymore after running a Pyspark program. It belongs to root, and I am running the pyspark code as root.

I also checked /etc/apache-flume-1.8.0-bin/conf/log4j.properties and ensured the path was correct (which it is). I also tried changing the path to my project directory to no avail.

I tried deleting the file (to see if it would be automatically created), and I tried creating an empty file first as well. Still no error logs.

Is there a way to fix this?

I am on Ubuntu 18.04, Apache Flume 1.8.0, Pyspark 2.2.2


Solution

  • I ended up having to uninstall Flume, and then reinstall Flume. Unfortunately I could not find any other way to get it working.