dockernginxdevopsdocker-registry

docker private registry with nginx reverse proxy freeze when pushing


I setup my private registry using docker registry official image, when try to push my own build image to my docker registry subdomain (cloudflare) it freeze at some layer but the behavior is odds, when i follow this step on do it's work just fine

digital ocean step 6

the image size of ubuntu is around 102MB my nestjs api is 380MB

OS: Ubuntu 24.04 on VMWare (Private Datacenter)

The push refers to repository [example.example.co.th/registry/api-yarn]
e7451737ce6d: Pushed
41627bf62206: Pushing [==================================================>]  254.1MB
bd143d6edbf5: Pushed
4503746488c2: Pushed
173e955be7d1: Pushed
c694e1c012a5: Pushed
77a628713662: Pushing [==================================================>]  128.5MB
5af4f8f59b76: Pushed
received unexpected HTTP status: 524

this is my registry docker compose

services:
  registry:
    image: registry:latest
    restart: always
    ports:
      - 5000:5000
    environment:
      REGISTRY_AUTH: htpasswd
      REGISTRY_AUTH_HTPASSWD_REALM: Registry
      REGISTRY_AUTH_HTPASSWD_PATH: /auth/registry.password
      REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data
    volumes:
      - registrydata:/data
      - ./auth:/auth
    networks:
      - docker-registry-net

networks:
  docker-registry-net:
    driver: bridge
    name: docker-registry-net

volumes:
  registrydata:
    driver: local
    name: registry-data

and this my nginx config

server {
  listen 80;
  listen 443 ssl http2;
  ssl_certificate /home/justdev/ssl/just-car/cert.pem;
  ssl_certificate_key /home/justdev/ssl/just-car/key.pem;
  ssl_client_certificate /home/justdev/ssl/ca.crt;
  client_max_body_size 16g;
  proxy_read_timeout 900;
  server_name example.example.com;

  location /portainer/ {
    proxy_pass http://localhost:9000/;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_buffering off;
    client_max_body_size 0;
  }

  location /registry/ {
    proxy_pass http://localhost:5000/;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_buffering off;
    client_max_body_size 6G;
  }

  location /v2/ {
    proxy_pass http://localhost:5000/v2/;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_buffering off;
    client_max_body_size 6G;
  }
}

and this is my /etc/nginx/nginx.conf

user www-data;
worker_processes auto;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
        # multi_accept on;
}

http {

        ##
        # Basic Settings
        ##
        client_max_body_size 0;
        sendfile on;
        tcp_nopush on;
        types_hash_max_size 2048;
        # server_tokens off;

        server_names_hash_bucket_size 128;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;

        ##
        # Gzip Settings
        ##

        gzip on;

        # gzip_vary on;
        # gzip_proxied any;
        # gzip_comp_level 6;
        # gzip_buffers 16 8k;
        # gzip_http_version 1.1;
        # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /home/example/nginx-config/*.conf;
        include /etc/nginx/sites-enabled/*;
}
registry-1  | time="2024-06-27T20:49:24.874519035Z" level=error msg="response completed with error" auth.user.name= err.code="blob upload invalid" err.detail="invalid secret" err.message="blob upload invalid" go.version=go1.20.8 http.request.contenttype="application/octet-stream" http.request.host=docker-registry.justcar.co.th http.request.id=66002120-0659-4ccc-9c3e-c2959e520dd3 http.request.method=PATCH http.request.remoteaddr= http.request.uri="/v2/registry/lmsapi-yarn/blobs/uploads/5f0b7dab-c3e5-418f-ab37-2fce7b0fbdaa?_state=An2E8QMRBgaBRDNyT5sB4BGCRNxyNG7wIPS6dIt7_JZ7Ik5hbWUiOiJyZWdpc3RyeS9sbXNhcGkteWFybiIsIlVVSUQiOiI1ZjBiN2RhYi1jM2U1LTQxOGYtYWIzNy0yZmNlN2IwZmJkYWEiLCJPZmZzZXQiOjAsIlN0YXJ0ZWRBdCI6IjIwMjQtMDYtMjdUMjA6NDQ6MzcuMzE5MzU4MzY0WiJ9" http.request.useragent="docker/26.1.4 go/go1.21.11 git-commit/de5c9cf kernel/5.15.153.1-microsoft-standard-WSL2 os/linux arch/amd64 UpstreamClient(Docker-Client/26.1.4 \(windows\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=4.008346ms http.response.status=404 http.response.written=88 vars.name="registry/lmsapi-yarn" vars.uuid=5f0b7dab-c3e5-418f-ab37-2fce7b0fbdaa 

I want to be able to push my image into my private docker registry


Solution

  • removing proxy_buffering off; should fix the issue. like:

    location /v2/ {
        proxy_pass http://localhost:5000/v2/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        client_max_body_size 6G;
    }