I have a file named data_[2022-10-03:2022-10-23].csv.gzip
in S3, inside a bucket and folder named s3://<bucket_name>/data/cache/
I am attempting to delete this file using S3FS. When I attempt to do so with:
filename = 'data_[2022-10-03:2022-10-23].csv.gzip'
data_filepath = 's3://<bucket_name>/data/cache'
fs.rm(f"{data_filepath}/{filename}")
But the rm call throws exception:
error('bad character range 2-1 at position 54')
It seems it is thinking this is a regex.
I attempted escaping it with:
filename = filename.replace('[', '\\[').replace(']', '\\]').replace('-', '\\-')
but I get the same error:
('bad character range \\\\-1 at position 57',)
I also tried re.escape()
:
fs.rm(f"{data_filepath}/{re.escape(filename)}")
But I get:
error('bad character range \\\\-1 at position 57')
The same thing.
I tried the delete method:
fs.delete(f"{data_filepath}/{filename}")
Same error.
I tried accessing the underlying boto3 and deleting directly:
bucket_name = '<bucket_name>'
path = 'data/cache'
filename = 'data_[2022-10-03:2022-10-23].csv.gzip'
fs.s3.delete_object(Bucket=bucket_name, Key=f'{path}/{filename}')
This method executes with no errors, but nothing is deleted.
How can I delete this file?
Tested this with boto3
as below to see it deleting the file as it is:
import boto3
def s3_delete():
fs = boto3.client('s3', aws_access_key_id="########", aws_secret_access_key="########")
bucket_name = '<bucket-name>'
path = 'data/cache'
filename = 'data_[2022-10-03:2022-10-23].csv.gzip'
fs.delete_object(Bucket=bucket_name, Key=f'{path}/{filename}')
s3_delete()