Lambda Function to Delete Object after interrogation of file, triggered by S3 Create event not deleting file

I have a Lambda function that is triggered by S3 Put when a file gets uploaded to the bucket. After interrogating the file, I move to delete the file if it fails some checks - in my test I am just checking the extension on the key. I noticed that when the function runs via the S3 trigger, the file is remaining in the bucket sometimes with delete markers. If I run the function using the Test button afterwards, it deletes every time. Also if I remove the S3 trigger altogether, upload a file, then run the Lambda function using Test button with the event data associated with my upload, it removes the file from the bucket every time. What I surmise from all this, is possibly a timing issue, so I even added a 15 second delay. Still it does not seem to work. Here is one block of code I have tried:

        # Extract bucket name, object key and version
        bucket_name = record['s3']['bucket']['name']
        object_key = record['s3']['object']['key']
        version_id = record['s3']['object']['versionId']

       # Slight pause before deleting all versions
       time.sleep(15) 

       print(f"File {object_key} has an invalid extension. Deleting it from bucket {bucket_name}.") 
       
       # First delete the object - will probably add a delete marker
       response = s3_client.delete_object(Bucket=bucket_name, Key=object_key, VersionId=version_id)
       print(f"Deleted {object_key} from {bucket_name} for version {version_id}.")
       print("Actual Response follows:")
       print(response)

       # Slight pause before deleting all versions
       time.sleep(3)

       # Call function to delete all versions since a versioned bucket
       delete_all_versions_new(bucket_name, object_key)

The function delete_all_versions_new(bucket_name, object_key) looks like this. I added this when the first delete which included version wasn't working.

       def delete_all_versions_new(bucket_name, object_key):
         """Delete all versions of an object from a versioned S3 bucket."""
         s3_client = boto3.client('s3')
         try:
           # List all versions of the object
           versions = s3_client.list_object_versions(Bucket=bucket_name, Prefix=object_key)
           if 'Versions' in versions:
               for version in versions['Versions']:
                  s3_client.delete_object(Bucket=bucket_name, Key=object_key, VersionId=version['VersionId'])
                  print(f"Deleted version {version['VersionId']} of {object_key} from {bucket_name}.")
           if 'DeleteMarkers' in versions:
               for marker in versions['DeleteMarkers']:
                 s3_client.delete_object(Bucket=bucket_name, Key=object_key, VersionId=marker['VersionId'])
                 print(f"Deleted delete marker {marker['VersionId']} of {object_key} from {bucket_name}.")
        except Exception as e:
            print(f"Failed to delete all versions of {object_key} from {bucket_name}: {e}")

Anybody run into this issue? Any suggestions are very much appreciated. Thank you for reviewing!

Update - here is log output:

File RSTMT/ASP_SUBMITTER/Q2+2023/DUA-Blank.pdf has an invalid extension. Deleting it from bucket after pause. Deleted KEY/ASP_SUBMITTER/Q2+2023/DUA-Blank.pdf from for version wp3CRB_gJwsCPGpDJQ9Bjvyus5N3d6V2. Actual Response follows: {'ResponseMetadata': {'RequestId': 'S0Y16H5YZCNCHJW6', 'HostId': 'xxxx', 'HTTPStatusCode': 204, 'HTTPHeaders': {'x-amz-id-2': '', 'x-amz-request-id': '', 'date': 'Fri, 09 May 2025 02:54:26 GMT', 'x-amz-version-id': '', 'server': 'AmazonS3'}, 'RetryAttempts': 0}, 'VersionId': 'wp3CRB_gJwsCPGpDJQ9Bjvyus5N3d6V2'} END RequestId: ...

Update on this issue. The S3 trigger seemed to have a timing issue. I was able to resolve this by using Cloudwatch EventBridge to monitor S3 CreateObject event and use that as a trigger. Once I removed the S3 trigger, changed to this trigger (and adjusted code for the changes to JSON event), it works perfectly now.