hadoophdfsalluxio

why alluxio files keep a state of TO_BE_PERSISTED all the time


I have deployed an Alluxio cluster based on HDFS cluster. When I use Alluxio Native Java Api to write some files to Alluxio with setting writetype ASYNC_THROUGH, the files (even just having 1G) seem not be written to HDFS, keep a state of TO_BE_PERSISTED all the time after few days.


Solution

  • I find the answer. I find the log "ERROR DefaultAsyncPersistHandler - Not all the blocks of file /test/sample_data/order_detail_titile_2.0.txt stored on the same worker", When using ASYNC_THROUGH, Alluxio request all file blockers in one worker. There is no problem where I use CACHE_THROUGH