jsonwordpressnginx

nginx sub_filter not working on some uncompressed json (wordpress, elementor)


I've now spent entirely too many hours/days googling this, I've almost waterboarded chatgpt for answers, so now I need to ask for some input.

I have a nginx reverse proxy in front of a php-fpm server (wordpress). I'm using sub_filter to change some values in responses.

This is working for some json responses, but not others.

Here are relevant parts of the nginx.conf:

# Reverse proxy
server {
    ... # certificates, logging etc. removed for simplicity

    root /var/www/html;
    index index.php index.html index.htm;
    
    ...

    location / {
        gzip off;

        ... # proxy_set_header stuff removed

        proxy_set_header Accept-Encoding "";

        proxy_pass http://127.0.0.1:8082;

        sub_filter_once off;
        sub_filter_types *;
        sub_filter "target_vlue" "replacement_value";
    }
}

# Application http
server {
    ... # listen, logging etc removed

    location ~ \.php$ {
            gzip off;
            try_files $uri =404;
            fastcgi_split_path_info ^(.+\.php)(/.+)$;
            fastcgi_index index.php;
            include fastcgi_params;
            fastcgi_param SCRIPT_FILENAME /var/www/html$fastcgi_script_name;
            fastcgi_param PATH_INFO $fastcgi_path_info;

            fastcgi_pass test-wordpress:9000; #its docker
    }
}

I have the double server setup for reasons - should not be relevant here.

Notice the gzip off; and Accept-Encoding "";

In php-pfm the compression is turned off:

9dedac8fdba4:/var/www/html# php -i |grep zlib
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value

I checked that the Content-Type header in the response is application/json; charset=utf-8 , and I've checked that application/json is part of the sub_filter types configuration.

Further debugging

The json that sub_filter does not work in is produced by Wordpress (plugin). Its big but here is a snippet:

{
  "success": true,
  "data": {
    "responses": {
      "app_site_editor_template_types": {
        "success": true,
        "code": 200,
        "data": [
          {
            "type": "header",
            ...
            "urls": {
               ...
              "thumbnail": "https://<TARGET_VALUE>/wp-content/plugins/elementor/assets/images/app/site-editor/header.svg", // <-- Here target_value is not changed by subfiler
              "EXTRA_DEBUG_TEST": "DEADBEEF"   // <--- My test, also not changed
            }
          },
          ...

As the comment says the "TARGET_VALUE" is not changed by sub_filter as hoped. If I change sub_filter to replace "DEADBEEF", thats also not working. So its not the value itself. I've edited the plugin .php to inject the extra "EXTRA_DEBUG_TEST" part of the json.

If I put a simple part of this JSON in a .php file and request that through the same server setup, then the values are replaced as expected:

{
  "status": "success",
  "message": "This is a simple JSON response",
  "items": [
    1,
    2,
    3,
    4,
    5
  ],
  "url": "https://<TARGET_VALUE>", // <-- sub_filter catches and changes this
  "url2": "https://<TARGET_VALUE>/wp-content/plugins/elementor/assets/images/app/site-editor/header.svg", // <-- sub_filter catches and changes this
}

Note: The second json is the entire debug-json used.

Anyone have any idea whats going on here?

Edit: Its not the json. I edited the plugin .php to return a really simple json object:

{
    "success": true,
    "data": {
        "responses": [ "https://<TARGET_VALUE>/wp-admin" ]
    }
}

sub_filter still not replacing the target_value.


Solution

  • What a wonderful developer approach :) Still even this can be solved using nginx gunzip module as follows:

    # Reverse proxy
    server {
        ... # certificates, logging etc. removed for simplicity
    
        root /var/www/html;
        index index.php index.html index.htm;
        
        ...
    
        location / {
            # gzip off; <-- no need for this directive
    
            ... # proxy_set_header stuff removed
    
            proxy_set_header Accept-Encoding ""; # this one is important
    
            proxy_pass http://127.0.0.1:8082;
    
            sub_filter_once off;
            sub_filter_types application/json;
            sub_filter "target_value" "replacement_value";
        }
    }
    
    # Application http
    server {
        ... # listen, logging etc removed
    
        location ~ \.php$ {
            # gzip off; <-- no need for this directive
            gunzip on; # this is a key point for this config to work
    
            try_files $uri =404;
    
            # fastcgi_split_path_info ^(.+\.php)(/.+)$; <-- this directive
            # makes no sense in "location ~ \.php$ { ... }"
            # (in opposite to something like "location ~ \.php(?:$|/) { ... }")
    
            fastcgi_index index.php;
            include fastcgi_params;
            fastcgi_param SCRIPT_FILENAME /var/www/html$fastcgi_script_name;
    
            # fastcgi_param PATH_INFO $fastcgi_path_info; <-- this one
            # makes no sense here as well
    
            fastcgi_pass test-wordpress:9000; #its docker
        }
    }