I've now spent entirely too many hours/days googling this, I've almost waterboarded chatgpt for answers, so now I need to ask for some input.
I have a nginx reverse proxy in front of a php-fpm server (wordpress). I'm using sub_filter to change some values in responses.
This is working for some json responses, but not others.
Here are relevant parts of the nginx.conf:
# Reverse proxy
server {
... # certificates, logging etc. removed for simplicity
root /var/www/html;
index index.php index.html index.htm;
...
location / {
gzip off;
... # proxy_set_header stuff removed
proxy_set_header Accept-Encoding "";
proxy_pass http://127.0.0.1:8082;
sub_filter_once off;
sub_filter_types *;
sub_filter "target_vlue" "replacement_value";
}
}
# Application http
server {
... # listen, logging etc removed
location ~ \.php$ {
gzip off;
try_files $uri =404;
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME /var/www/html$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_pass test-wordpress:9000; #its docker
}
}
I have the double server setup for reasons - should not be relevant here.
Notice the gzip off; and Accept-Encoding "";
In php-pfm the compression is turned off:
9dedac8fdba4:/var/www/html# php -i |grep zlib
zlib.output_compression => Off => Off
zlib.output_compression_level => -1 => -1
zlib.output_handler => no value => no value
I checked that the Content-Type header in the response is application/json; charset=utf-8 , and I've checked that application/json is part of the sub_filter types configuration.
Further debugging
The json that sub_filter does not work in is produced by Wordpress (plugin). Its big but here is a snippet:
{
"success": true,
"data": {
"responses": {
"app_site_editor_template_types": {
"success": true,
"code": 200,
"data": [
{
"type": "header",
...
"urls": {
...
"thumbnail": "https://<TARGET_VALUE>/wp-content/plugins/elementor/assets/images/app/site-editor/header.svg", // <-- Here target_value is not changed by subfiler
"EXTRA_DEBUG_TEST": "DEADBEEF" // <--- My test, also not changed
}
},
...
As the comment says the "TARGET_VALUE" is not changed by sub_filter as hoped. If I change sub_filter to replace "DEADBEEF", thats also not working. So its not the value itself. I've edited the plugin .php to inject the extra "EXTRA_DEBUG_TEST" part of the json.
If I put a simple part of this JSON in a .php file and request that through the same server setup, then the values are replaced as expected:
{
"status": "success",
"message": "This is a simple JSON response",
"items": [
1,
2,
3,
4,
5
],
"url": "https://<TARGET_VALUE>", // <-- sub_filter catches and changes this
"url2": "https://<TARGET_VALUE>/wp-content/plugins/elementor/assets/images/app/site-editor/header.svg", // <-- sub_filter catches and changes this
}
Note: The second json is the entire debug-json used.
Anyone have any idea whats going on here?
Edit: Its not the json. I edited the plugin .php to return a really simple json object:
{
"success": true,
"data": {
"responses": [ "https://<TARGET_VALUE>/wp-admin" ]
}
}
sub_filter still not replacing the target_value.
What a wonderful developer approach :) Still even this can be solved using nginx gunzip
module as follows:
# Reverse proxy
server {
... # certificates, logging etc. removed for simplicity
root /var/www/html;
index index.php index.html index.htm;
...
location / {
# gzip off; <-- no need for this directive
... # proxy_set_header stuff removed
proxy_set_header Accept-Encoding ""; # this one is important
proxy_pass http://127.0.0.1:8082;
sub_filter_once off;
sub_filter_types application/json;
sub_filter "target_value" "replacement_value";
}
}
# Application http
server {
... # listen, logging etc removed
location ~ \.php$ {
# gzip off; <-- no need for this directive
gunzip on; # this is a key point for this config to work
try_files $uri =404;
# fastcgi_split_path_info ^(.+\.php)(/.+)$; <-- this directive
# makes no sense in "location ~ \.php$ { ... }"
# (in opposite to something like "location ~ \.php(?:$|/) { ... }")
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME /var/www/html$fastcgi_script_name;
# fastcgi_param PATH_INFO $fastcgi_path_info; <-- this one
# makes no sense here as well
fastcgi_pass test-wordpress:9000; #its docker
}
}