My flume.log file was getting enormous so I foolishly deleted it. It's absolute path was
/etc/apache-flume-1.8.0-bin/logs/flume.log
I then recreated it (ie: touch flume.log) in the same directory. However nothing gets appended to it anymore after running a Pyspark program. It belongs to root, and I am running the pyspark code as root.
I also checked /etc/apache-flume-1.8.0-bin/conf/log4j.properties and ensured the path was correct (which it is). I also tried changing the path to my project directory to no avail.
I tried deleting the file (to see if it would be automatically created), and I tried creating an empty file first as well. Still no error logs.
Is there a way to fix this?
I am on Ubuntu 18.04, Apache Flume 1.8.0, Pyspark 2.2.2
I ended up having to uninstall Flume, and then reinstall Flume. Unfortunately I could not find any other way to get it working.