Why do files need time to synchronize after writing in writeType THROUGH in Alluxio?
When I write file in directory mounted by alluxio-fuse using writeType THROUGH. I find that it takes 2-3 minutes to synchronize files. Why do files need time to synchronize?
Following is mount direactory. write time : 15:40. after sync: 15:43
Solution
When writing to Alluxio with THROUGH writeType, Alluxio will first update its metadata to show files with zero bytes (like the 9.txt in your first image). When the file is successfully written to Alluxio Under FileSystem, Alluxio will update its metadata to show the actual size of this file (9.txt show its actual size as 209715200 bytes).
The 2 to 3 minutes is the time that Alluxio writes data to under filesystem.
Thanks,
- Trino with Spilling and Alluxio Caching makes a large number of requests to the Object Storage
- Mount HDFS Data on Alluxio
- How to check if folder is cached in Alluxio?
- Why do mtime and atime need to be updated?
- Will journal be lost if there is no time to flush to the journal writer before jvm hangs?
- Is hadoop3.1.1 not supported?
- Is there any way I can avoid specifying the port in the alluxio fs uri?
- Is the s3 connection configurable with role arn alone?
- Set Default Block Size for Specific Path
- setting up Alluxio cluster, path points to Java 8 but Alluxio thinks it's java 7?
- difference between WORKER_EVICTOR and WORKER_BLOCK_ANNOTATOR
- How can I maintain a list of constant masters and workers under conf/masters and conf/workers in a managed Scaling cluster?
- How do I get Alluxio POSIX to run with version 3 libfuse?
- Hive metastore with alluxio storage in parquet data type problem
- why alluxio files keep a state of TO_BE_PERSISTED all the time
- Master not able to start everytime I restart my alluxio machine
- How to set master address and 19998 port in Alluxio 2.0 java api?
- Why do files need time to synchronize after writing in writeType THROUGH in Alluxio?
- Why does UfsSyncPathCache.java:68 parameter not work in Alluxio?
- How to monitor the status of standby masters in Alluxio?
- Unable to access Alluxio File System API in IDE
- Timeout to read from Alluxio
- Channel is closed while reading from Alluxio using Presto
- Difference between Alluxio(Tachyon) and Tungsten in Spark?
- Resources/Documentation on how does the failover process work for the Spark Driver (and its YARN Container) in yarn-cluster mode
- Error in setting up Tachyon on S3 under filesystem
- Is Tachyon by default implemented by the RDD's in Apache Spark?
- Test Spark with Tachyon
- Tachyon Doesn't Seem to be Aware of Available Memory
- Tachyon on Dataproc Master Replication Error