I built custom functions for sequential resampling, clipping, and writing raster stacks from CHIRPS, Modis, and ERA5. After considering that I work over an extensive area and long temporal period (which causes huge rasters), I've set the working and temporary directories to E:// which have 770 GB. However, after performing the function (twice), my C:// decreased drastically from 172 to 15 GB.
Any idea to modify the function? or I was wrong in the first place?
# Load necessary libraries
library(terra)
library(progress)
setwd("E:/KLHK/KHG/1_Riau/R")
# Set the temporary directory for R's file usage
tempdir_custom <- "E:/KLHK/KHG/1_Riau/R"
# Ensure the temporary directory exists
if (!dir.exists(tempdir_custom)) {
dir.create(tempdir_custom, recursive = TRUE)
}
message("Temporary directory set to:", tempdir_custom)
# Sequential resampling function with memory management and flexible resolution
sequential_resample <- function(raster, final_resolution = 10, method = "cubicspline") {
current_res <- res(raster)
resampled_raster <- raster
# Halve the resolution until the largest side reaches <= 2x final resolution
while (max(current_res) > 2 * final_resolution) {
target_res <- current_res / 2
temp_target <- rast(ext(raster), res = target_res, crs = crs(raster))
resampled_raster <- resample(resampled_raster, temp_target, method = method)
current_res <- res(resampled_raster)
}
# Final resampling to the exact final_resolution value
final_target <- rast(ext(raster), res = c(final_resolution, final_resolution), crs = crs(raster))
resampled_raster <- resample(resampled_raster, final_target, method = method)
return(resampled_raster)
}
# Function to resample, clip, stack, and write raster data
resample_clip_stack_write <- function(
raster_data,
output_directory,
tempdir_custom,
final_resolution = 10,
extent = NULL,
method = "cubicspline",
stack_name = NULL
) {
# Ensure the output directory exists
if (!dir.exists(output_directory)) {
dir.create(output_directory, recursive = TRUE)
message("Created directory:", output_directory)
} else {
message("Directory already exists:", output_directory)
}
# Set up a temporary directory for intermediate operations
tempdir(tempdir_custom)
message("Temporary directory set to:", tempdir_custom)
# Initialize an empty list to store resampled and clipped raster layers
resampled_layers <- list()
# Iterate over each layer in the raster stack
for (i in 1:nlyr(raster_data)) {
message(paste("Processing layer", i, "of", nlyr(raster_data)))
# Access the layer by index
raster_layer <- raster_data[[i]]
# Apply sequential resampling
resampled_layer <- sequential_resample(raster_layer, final_resolution, method)
# Clip the resampled layer if an extent is provided
if (!is.null(extent)) {
message("Clipping layer to extent...")
if (inherits(extent, "SpatVector")) {
resampled_layer <- crop(resampled_layer, extent)
} else if (inherits(extent, "SpatRaster")) {
resampled_layer <- mask(resampled_layer, extent)
}
message("Clipping complete.")
}
# Store the processed layer
resampled_layers[[i]] <- resampled_layer
}
# Combine all processed layers into a single SpatRaster stack
stacked_raster <- rast(resampled_layers)
message("All layers combined into a single SpatRaster stack.")
# Determine the output file name
output_file <- file.path(
output_directory,
ifelse(is.null(stack_name), "Processed_Stack.tif", paste0(stack_name, ".tif"))
)
# Write the resulting stack to disk
writeRaster(stacked_raster, output_file, overwrite = TRUE)
message("Stacked raster written to:", output_file)
# Return the stacked raster as an in-memory SpatRaster object
return(stacked_raster)
}
# Define AOI or raster for clipping
aoi_Riau <- vect("E:/KLHK/KHG/1_Riau/1_input/AOI/PHU_bound_10km.shp") # Vector extent (AOI)
# Set the output directory and other parameters
output_res_riau <- "E:/KLHK/KHG/1_Riau/3_olah2/1_CHRIPS"
# Call the function to process rasters
res_Riau_precip_2021 <- resample_clip_stack_write(
raster_data = Riau_precip_2021,
output_directory = output_res_riau,
tempdir_custom = tempdir_custom,
final_resolution = 10, # Target resolution
extent = aoi_Riau, # Shapefile AOI or raster for clipping
method = "cubicspline",
stack_name = "Riau_Precip_2021_Stack" # Optional name for the resulting stack
)
thank to Chris, I clarified the background.
the code example above is for resampling, clipping, and writing CHIRPS stacks (precipitation) only. The initial Riau_precip_2021 stack (containing 24 rasters) only accounts for 264 KB in my E:// disk with the resulting resampling stack is 18.9 GB. this is not my concern since I already prepared E:// as primary storage for all resampled stacks.
find R still uses C://Users/.../AppData/Local/Temp to save their temporary rasters (yes it is the problem) before it stacked and written to my E:/. I want the temporary folder to be moved to this path:
tempdir_custom <- "E:/KLHK/KHG/1_Riau/R"
Any idea ?
You seem to think that this changes the R tempdir:
# Set up a temporary directory for intermediate operations
tempdir(tempdir_custom)
But it does not. tempdir
only returns the temporary directory path. I think you can only set the tempdir path before starting R (via a configuration file).
The good news is that, since your interest is in temporary files created by "terra", you can do
terraOptions(tempdir=tempdir_custom)