AWS Lambda + S3: Moving/Renaming a folder in S3 using Lambda

I need to be able to move/rename a folder inside S3 using a lambda function written in python. I have seen that it is possible to copy and then delete a file using the boto3 library but I cannot find a method of renaming a folder.

Example: bucket:/folder/ -> bucket:/renamedfolder/

What you have seen is correct, currently AWS does not support renaming an object in S3. To rename an object in S3, you need to copy the object to a new location with the desired name and then delete the original object.

Folder structure in S3:

S3 is an object storage service, not a file system. The concept of "folders" in S3 is an abstraction provided by the AWS Management Console to help users organize their objects visually. However, these "folders" don't actually exist as separate entities within S3.

For example, if you have an object with the path my-bucket/my-folder/my-file.csv, the full name (or key) of the object is my-folder/my-file.csv. The my-folder/ part is simply a prefix in the object's key. S3 organizes objects based on these prefixes, and in the S3 console, this is represented as a folder structure for ease of use. Nevertheless, S3 only sees it as an object with the key (file name) my-folder/my-file.csv but the folder does not exist as a separate it just part of the object's name.

Conclusion: to rename a folder you have to rename all objects with that prefix

Example code:

EDIT thanks to comment from @jarmond code is adapted to hold over 1000 keys. As mention in the comments s3.list_objects_v2 only returns the first 1000 objects. If you have a prefix with more than 1000 keys you should consider this fact.

import boto3

s3 = boto3.client('s3')

def rename_prefix_in_many_objects(bucket_name, old_prefix, new_prefix, objects_keys):
    for obj in objects_keys :
        old_key = obj['Key']
        new_key = old_key.replace(old_prefix, new_prefix, 1)
        rename_s3_object(bucket_name, old_key, new_key)

def rename_s3_object(bucket_name, old_key, new_key):
    s3.copy_object(Bucket=bucket_name, 
                   CopySource={'Bucket': bucket_name, 'Key': old_key}, 
                   Key=new_key)
    s3.delete_object(Bucket=bucket_name, Key=old_key)

def rename_prefix_until_complete(bucket_name, old_prefix, new_prefix):
    response = s3.list_objects_v2(
        Bucket=bucket_name, Prefix=old_prefix
    )

    # This for loop is because by default aws only returns 1000 files
    while response["IsTruncated"]:
        rename_prefix_in_many_objects(bucket_name, old_prefix, new_prefix, response['Contents'])
        response = s3.list_objects_v2(
            Bucket=bucket_name,
            Prefix=old_prefix,
            ContinuationToken=response["NextContinuationToken"],
        )

bucket_name = "bucket"
old_prefix = "folder/"
new_prefix = "renamedfolder/"

s3_keys = rename_prefix_until_complete(bucket_name, old_prefix, new_prefix)