Changing R Temporary Files

I built custom functions for sequential resampling, clipping, and writing raster stacks from CHIRPS, Modis, and ERA5. After considering that I work over an extensive area and long temporal period (which causes huge rasters), I've set the working and temporary directories to E:// which have 770 GB. However, after performing the function (twice), my C:// decreased drastically from 172 to 15 GB.

Any idea to modify the function? or I was wrong in the first place?

# Load necessary libraries
library(terra)
library(progress)

setwd("E:/KLHK/KHG/1_Riau/R")

# Set the temporary directory for R's file usage
tempdir_custom <- "E:/KLHK/KHG/1_Riau/R"
# Ensure the temporary directory exists
if (!dir.exists(tempdir_custom)) {
  dir.create(tempdir_custom, recursive = TRUE)
}
message("Temporary directory set to:", tempdir_custom)

# Sequential resampling function with memory management and flexible resolution
sequential_resample <- function(raster, final_resolution = 10, method = "cubicspline") {
  current_res <- res(raster)
  resampled_raster <- raster  
  # Halve the resolution until the largest side reaches <= 2x final resolution
  while (max(current_res) > 2 * final_resolution) {
    target_res <- current_res / 2
    temp_target <- rast(ext(raster), res = target_res, crs = crs(raster))
    resampled_raster <- resample(resampled_raster, temp_target, method = method)
    current_res <- res(resampled_raster)
  }    
  # Final resampling to the exact final_resolution value
  final_target <- rast(ext(raster), res = c(final_resolution, final_resolution), crs = crs(raster))
  resampled_raster <- resample(resampled_raster, final_target, method = method)    
  return(resampled_raster)
}

# Function to resample, clip, stack, and write raster data
resample_clip_stack_write <- function(
  raster_data, 
  output_directory, 
  tempdir_custom, 
  final_resolution = 10, 
  extent = NULL, 
  method = "cubicspline",
  stack_name = NULL
) {
  # Ensure the output directory exists
  if (!dir.exists(output_directory)) {
    dir.create(output_directory, recursive = TRUE)
    message("Created directory:", output_directory)
  } else {
    message("Directory already exists:", output_directory)
  }  
  # Set up a temporary directory for intermediate operations
  tempdir(tempdir_custom)
  message("Temporary directory set to:", tempdir_custom)  
  # Initialize an empty list to store resampled and clipped raster layers
  resampled_layers <- list()  
  # Iterate over each layer in the raster stack
  for (i in 1:nlyr(raster_data)) {
    message(paste("Processing layer", i, "of", nlyr(raster_data)))    
    # Access the layer by index
    raster_layer <- raster_data[[i]]    
    # Apply sequential resampling
    resampled_layer <- sequential_resample(raster_layer, final_resolution, method)    
    # Clip the resampled layer if an extent is provided
    if (!is.null(extent)) {
      message("Clipping layer to extent...")
      if (inherits(extent, "SpatVector")) {
        resampled_layer <- crop(resampled_layer, extent)
      } else if (inherits(extent, "SpatRaster")) {
        resampled_layer <- mask(resampled_layer, extent)
      }
      message("Clipping complete.")
    }    
    # Store the processed layer
    resampled_layers[[i]] <- resampled_layer
  }  
  # Combine all processed layers into a single SpatRaster stack
  stacked_raster <- rast(resampled_layers)
  message("All layers combined into a single SpatRaster stack.")  
  # Determine the output file name
  output_file <- file.path(
    output_directory, 
    ifelse(is.null(stack_name), "Processed_Stack.tif", paste0(stack_name, ".tif"))
  )  
  # Write the resulting stack to disk
  writeRaster(stacked_raster, output_file, overwrite = TRUE)
  message("Stacked raster written to:", output_file)  
  # Return the stacked raster as an in-memory SpatRaster object
  return(stacked_raster)
}

# Define AOI or raster for clipping
aoi_Riau <- vect("E:/KLHK/KHG/1_Riau/1_input/AOI/PHU_bound_10km.shp")  # Vector extent (AOI)

# Set the output directory and other parameters
output_res_riau <- "E:/KLHK/KHG/1_Riau/3_olah2/1_CHRIPS"

# Call the function to process rasters
res_Riau_precip_2021 <- resample_clip_stack_write(
  raster_data = Riau_precip_2021,
  output_directory = output_res_riau,
  tempdir_custom = tempdir_custom,
  final_resolution = 10,  # Target resolution
  extent = aoi_Riau,  # Shapefile AOI or raster for clipping
  method = "cubicspline",
  stack_name = "Riau_Precip_2021_Stack"  # Optional name for the resulting stack
)

thank to Chris, I clarified the background.

the code example above is for resampling, clipping, and writing CHIRPS stacks (precipitation) only. The initial Riau_precip_2021 stack (containing 24 rasters) only accounts for 264 KB in my E:// disk with the resulting resampling stack is 18.9 GB. this is not my concern since I already prepared E:// as primary storage for all resampled stacks.

find R still uses C://Users/.../AppData/Local/Temp to save their temporary rasters (yes it is the problem) before it stacked and written to my E:/. I want the temporary folder to be moved to this path:

tempdir_custom <- "E:/KLHK/KHG/1_Riau/R"

Any idea ?

Solution

You seem to think that this changes the R tempdir:

# Set up a temporary directory for intermediate operations
  tempdir(tempdir_custom)

But it does not. tempdir only returns the temporary directory path. I think you can only set the tempdir path before starting R (via a configuration file).

The good news is that, since your interest is in temporary files created by "terra", you can do

terraOptions(tempdir=tempdir_custom)