[SOLVED] minio.error.S3Error: S3 operation failed; code: AccessDenied with root user access key

minio.error.S3Error: S3 operation failed; code: AccessDenied with root user access key

I have docker cluster with several containers such as minio as file storage and backend on django and it roughly looks like this:

  files:
    image: minio/minio
    container_name: files
    env_file: .env
    command: server /data --console-address ":9001"
    healthcheck:
      test: curl -I ${MINIO_EXTERNAL_ENDPOINT}/minio/health/live
      interval: 5s
      timeout: 5s
      retries: 5
    volumes:
      - s3:/data
    ports:
      - 9000:9000
      - 9001:9001

  backend:
    build: ./backend
    container_name: backend
    env_file: .env
    volumes:
      - ./scripts/backend:/app/scripts
    healthcheck:
      test: curl --fail ${BACKEND_HC} || exit 1
      interval: 10s
      timeout: 5s
      retries: 5
    ports:
      - 8000:8000
    depends_on:
      files:
        condition: service_healthy

Everything was OK until today, when my SSD died and i was forced to use the older one. It has only 119 Gb of "real" space so i did Troubleshoot -> Purge Data in docker desktop (curse you, wsl/docker). And then cloned my project expecting it to start as usual. And then i was hit with the error in title. Here is useful part of stack trace:

backend  | [2024-11-13 15:50:41 +0700] [13] [INFO] Worker exiting (pid: 13)
backend  | [2024-11-13 15:50:41 +0700] [14] [ERROR] Exception in worker process
backend  | Traceback (most recent call last):

...

backend  |     application = get_wsgi_application()
backend  |                   ^^^^^^^^^^^^^^^^^^^^^^
backend  |   File "/usr/local/lib/python3.11/site-packages/django/core/wsgi.py", line 12, in get_wsgi_applicati
on
backend  |     django.setup(set_prefix=False)
backend  |   File "/usr/local/lib/python3.11/site-packages/django/__init__.py", line 24, in setup
backend  |     apps.populate(settings.INSTALLED_APPS)
backend  |   File "/usr/local/lib/python3.11/site-packages/django/apps/registry.py", line 124, in populate     
backend  |     app_config.ready()
backend  |   File "/usr/local/lib/python3.11/site-packages/django_minio_backend/apps.py", line 31, in ready    
backend  |     mbs = MinioBackendStatic()
backend  |           ^^^^^^^^^^^^^^^^^^^^
backend  |   File "/usr/local/lib/python3.11/site-packages/django_minio_backend/models.py", line 482, in __init
__

backend  |     self.check_bucket_existence()  # make sure the `MINIO_STATIC_FILES_BUCKET` exists
backend  |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend  |   File "/usr/local/lib/python3.11/site-packages/django_minio_backend/models.py", line 408, in check_
bucket_existence
backend  |     if not self.client.bucket_exists(self.bucket):
backend  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend  |   File "/usr/local/lib/python3.11/site-packages/minio/api.py", line 696, in bucket_exists
backend  |     self._execute("HEAD", bucket_name)
backend  |   File "/usr/local/lib/python3.11/site-packages/minio/api.py", line 440, in _execute
backend  |     return self._url_open(
backend  |            ^^^^^^^^^^^^^^^
backend  |   File "/usr/local/lib/python3.11/site-packages/minio/api.py", line 423, in _url_open
backend  |     raise response_error
backend  | minio.error.S3Error: S3 operation failed; code: AccessDenied, message: Access denied, resource: /loc
al-static, request_id: 18077B04951EF538, host_id: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e
3e8, bucket_name: local-static

As you can see, minio tries to verify if bucket local-static exists but fails to do so. The only credentials i use are root user access/secret keys so there is no way i would hit access denied. I never used any policies so there is nothing to configure hence i have no clue what to fix

Solution

Okay, i just rage-deleted the project and purged all data in docker, then cloned and built it again and this problem was gone.