Nginx Fast CGI cache ON error_page 404

Question

I would like to cache (fast_cgi) 404 response.

error_page 404 = /url_rewriting.php;

In my url_rewriting.php I generate image with php:

if(strpos($_SERVER['REQUEST_URI'], 'render/framed/file') !== FALSE) {
    $urlBlocks = ['VR', 'sizePixels', 'image', 'ver', 'frame', 'borderSize', 'mat', 'matSize', 'maxSize', 'frameGlass', 'minSize'];
    foreach($urlBlocks as $oneBlock) {
        if($pos = array_search($oneBlock, $urlParts)) {
            if(isset($urlParts[($pos+1)]) && $urlParts[($pos+1)] != '') {
                $_GET[$oneBlock] = urldecode($urlParts[($pos+1)]);
            }
        }    
    }

    chdir('include/php/render/framed');
    header('Status: 200 OK', false, 200);
    require ('include/php/render/framed/render_img.php');
}

By this way, I can have image src URL in the HTML like this :

https://mywebsite.com/include/php/render/framed/file/VR/1/size/300/image/U3dpwK/the-cat.jpg

which jpg file does not exist but is generated by PHP.

But I didn't find any way with Nginx to cache theses php generated images.

I tried this:

set $no_cache 0;

location ~ /render/ {
        include snippets/fastcgi-php.conf;
        #fastcgi_pass unix:/run/php/php7.0-fpm.sock;
        fastcgi_pass 127.0.0.1:9000;

        fastcgi_buffers 8 16k; # increase the buffer size for PHP-FTP
        fastcgi_buffer_size 32k; # increase the buffer size for PHP-FTP
        fastcgi_cache_key $scheme$host$request_uri$request_method;
        fastcgi_cache PROD;
        fastcgi_cache_valid any 20d;
        fastcgi_cache_valid 404 20d;
        fastcgi_cache_use_stale updating error timeout invalid_header http_500 http_503;
        fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
        fastcgi_hide_header "Set-Cookie";
        fastcgi_cache_bypass $no_cache;
        fastcgi_no_cache $no_cache;
        expires 10M;
        access_log off;
        add_header Cache-Control "public";
        add_header X-Cache-Status $upstream_cache_status;
}

It's working for image URL

https://mywebsite.com/include/php/render/framed/img.php?VR=1&size=300&image=U3pmwKi 

but not for image URL

https://mywebsite.com/include/php/render/framed/file/VR/1/size/300/image/U3dpwK/the-cat.jpg

Yet I have put fastcgi_cache_valid 404 20d;

So, how to cache scripts going on error_page 404 = /url_rewriting.php;?

EDIT

Here are 2 curl -I output:

Non working URL (For cache)

curl -I "https://mywebsite.com/include/php/render/framed/file/VR/1/sizePixels/300/image/SzDuehqyda%3D/ver//frame/black-e91-2/borderSize/1.70/mat/zkadhtcoz/matSize/10/maxSize/800/minSize/600/freedom.jpg"
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Fri, 04 May 2018 14:59:24 GMT
Content-Type: image/jpeg
Connection: keep-alive
Set-Cookie: Mywebsite=vmegg0qk7udtmkmcathd329kkp; expires=Sun, 03-Jun-2018 14:59:24 GMT; Max-Age=2592000; path=/
Cache-Control: private, max-age=31536000, pre-check=31536000
Pragma: private
Last-Modified: Sat, 01 Apr 2000 13:13:45 GMT
Content-transfer-encoding: binary
Expires: Sat, 08 Jun 19 15:59:24 +0100
Strict-Transport-Security: max-age=31536000

Working URL (For cache)

curl -I "https://mywebsite.com/include/php/render/framed/render_img.php?VR=1&sizePixels=360&image=SzDuehqyda%3D&ver=&frame=black-e91-2&borderSize=1.70&mat=zkadhtcoz&matSize=10&maxSize=800&minSize=600"
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Fri, 04 May 2018 14:59:59 GMT
Content-Type: image/jpeg
Connection: keep-alive
Cache-Control: max-age=25920000
Pragma: private
Last-Modified: Sat, 01 Apr 2000 13:13:45 GMT
Content-transfer-encoding: binary
Expires: Thu, 28 Feb 2019 14:59:59 GMT
Cache-Control: public
X-Cache-Status: HIT

EDIT2

Output of nginx -t

root@mywebsite-london-01:/var/www/mywebsite.com/prod# nginx -T
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
        # multi_accept on;
}

http {

        ##
        #  Adding GeoIP for Matomo
        ##
        geoip_country  /var/www/geoip/GeoIP.dat;
        geoip_city     /var/www/geoip/GeoLiteCity.dat;

        # Max File upload
        client_max_body_size 100m;

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # Logging Settings
        ##
        log_format compression '$remote_addr - $remote_user [$time_local] '
                               '"$request" $status $bytes_sent '
                               '"$http_referer" "$http_user_agent" "$gzip_ratio"';

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##

        #       gzip on;
        gzip on;
        gzip_disable "msie6";
        gzip_vary on;
        gzip_proxied any;
        gzip_comp_level 6;
        gzip_min_length 1100;
        gzip_buffers 16 8k;
        gzip_http_version 1.1;
        gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}


# configuration file /etc/nginx/modules-enabled/50-mod-http-geoip.conf:
load_module modules/ngx_http_geoip_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-image-filter.conf:
load_module modules/ngx_http_image_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-http-xslt-filter.conf:
load_module modules/ngx_http_xslt_filter_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-mail.conf:
load_module modules/ngx_mail_module.so;

# configuration file /etc/nginx/modules-enabled/50-mod-stream.conf:
load_module modules/ngx_stream_module.so;

# configuration file /etc/nginx/mime.types:

types {
    text/html                             html htm shtml;
    text/css                              css;
    text/xml                              xml;
    image/gif                             gif;
    image/jpeg                            jpeg jpg;
    application/javascript                js;
    application/atom+xml                  atom;
    application/rss+xml                   rss;

    text/mathml                           mml;
    text/plain                            txt;
    text/vnd.sun.j2me.app-descriptor      jad;
    text/vnd.wap.wml                      wml;
    text/x-component                      htc;

    image/png                             png;
    image/tiff                            tif tiff;
    image/vnd.wap.wbmp                    wbmp;
    image/x-icon                          ico;
    image/x-jng                           jng;
    image/x-ms-bmp                        bmp;
    image/svg+xml                         svg svgz;
    image/webp                            webp;

    application/font-woff                 woff;
    application/java-archive              jar war ear;
    application/json                      json;
    application/mac-binhex40              hqx;
    application/msword                    doc;
    application/pdf                       pdf;
    application/postscript                ps eps ai;
    application/rtf                       rtf;
    application/vnd.apple.mpegurl         m3u8;
    application/vnd.ms-excel              xls;
    application/vnd.ms-fontobject         eot;
    application/vnd.ms-powerpoint         ppt;
    application/vnd.wap.wmlc              wmlc;
    application/vnd.google-earth.kml+xml  kml;
    application/vnd.google-earth.kmz      kmz;
    application/x-7z-compressed           7z;
    application/x-cocoa                   cco;
    application/x-java-archive-diff       jardiff;
    application/x-java-jnlp-file          jnlp;
    application/x-makeself                run;
    application/x-perl                    pl pm;
    application/x-pilot                   prc pdb;
    application/x-rar-compressed          rar;
    application/x-redhat-package-manager  rpm;
    application/x-sea                     sea;
    application/x-shockwave-flash         swf;
    application/x-stuffit                 sit;
    application/x-tcl                     tcl tk;
    application/x-x509-ca-cert            der pem crt;
    application/x-xpinstall               xpi;
    application/xhtml+xml                 xhtml;
    application/xspf+xml                  xspf;
    application/zip                       zip;

    application/octet-stream              bin exe dll;
    application/octet-stream              deb;
    application/octet-stream              dmg;
    application/octet-stream              iso img;
    application/octet-stream              msi msp msm;

    application/vnd.openxmlformats-officedocument.wordprocessingml.document    docx;
    application/vnd.openxmlformats-officedocument.spreadsheetml.sheet          xlsx;
    application/vnd.openxmlformats-officedocument.presentationml.presentation  pptx;

    audio/midi                            mid midi kar;
    audio/mpeg                            mp3;
    audio/ogg                             ogg;
    audio/x-m4a                           m4a;
    audio/x-realaudio                     ra;

    video/3gpp                            3gpp 3gp;
    video/mp2t                            ts;
    video/mp4                             mp4;
    video/mpeg                            mpeg mpg;
    video/quicktime                       mov;
    video/webm                            webm;
    video/x-flv                           flv;
    video/x-m4v                           m4v;
    video/x-mng                           mng;
    video/x-ms-asf                        asx asf;
    video/x-ms-wmv                        wmv;
    video/x-msvideo                       avi;
}

# CACHE (Images rendering)
fastcgi_cache_path /var/www/cache/l7 levels=1:2 keys_zone=L7:1m max_size=100m inactive=20d;
fastcgi_cache_path /var/www/cache/prod levels=1:2 keys_zone=PROD:100m max_size=10000m inactive=30d; # For PROD

add_header X-Cache $upstream_cache_status; #To check what is that for

# SSL

ssl_certificate /etc/letsencrypt/live/mywebsite.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/mywebsite.com/privkey.pem; # managed by Certbot
ssl_session_cache shared:SSL:20m;
ssl_session_timeout 180m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers ECDH+AESGCM:ECDH+AES256:ECDH+AES128:DHE+AES128:!ADH:!AECDH:!MD5;
ssl_dhparam /etc/nginx/cert/dhparam.pem;
add_header Strict-Transport-Security "max-age=31536000" always;

server {
        listen [::]:443 ssl ipv6only=on; # managed by Certbot
        listen 443 ssl; # managed by Certbot

        server_name mywebsite.com www.mywebsite.com l7.mywebsite.com cdn.mywebsite.com dev.mywebsite.com;

        set $rootfolder "prod";
        set $ask_auth "Restricted Area";
        if ($host ~ "l7.mywebsite.com") {
                set $rootfolder "l7";
                set $ask_auth off;
        }

        root /var/www/mywebsite.com/$rootfolder;

        access_log /var/log/nginx/mywebsite.com-access.log compression buffer=32k;
        error_log /var/log/nginx/mywebsite.com-error.log;

        # Add index.php to the list if you are using PHP
        index index.php index.html index.htm index.nginx-debian.html;

        error_page 404 = /url_rewriting.php;

        set $no_cache 1;
        if ($request_uri ~* "render_img.php") {
            set $no_cache 0;
        }

        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                try_files $uri $uri/ =404;
                #auth_basic            $ask_auth;
                #auth_basic_user_file  .htpasswd;
        }

        set $no_cache 0;
        #location ~ render_img.php  {
        location ~ /render\/framed/ {
                include snippets/fastcgi-php.conf;
                #fastcgi_pass unix:/run/php/php7.0-fpm.sock;
                fastcgi_pass 127.0.0.1:9000;

                fastcgi_buffers 8 16k; # increase the buffer size for PHP-FTP
                fastcgi_buffer_size 32k; # increase the buffer size for PHP-FTP
                fastcgi_cache_key $scheme$host$request_uri$request_method;
                fastcgi_cache PROD;
                fastcgi_cache_valid any 20d;
                fastcgi_cache_valid 404      1d;
                #fastcgi_cache_valid any 20d;
                #fastcgi_cache_use_stale updating error timeout invalid_header http_500 http_503 http_404;
                fastcgi_cache_use_stale updating error timeout invalid_header http_500 http_503;
                fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
                fastcgi_hide_header "Set-Cookie";
                fastcgi_cache_bypass $no_cache;
                fastcgi_no_cache $no_cache;
                expires 10M;
                access_log off;
                add_header Cache-Control "public";
                add_header X-Cache-Status $upstream_cache_status;
        }

        location ~ \.php$ {
                include snippets/fastcgi-php.conf;
                #fastcgi_pass unix:/run/php/php7.0-fpm.sock;
                fastcgi_pass 127.0.0.1:9000;

                fastcgi_buffers 8 16k; # increase the buffer size for PHP-FTP
                fastcgi_buffer_size 32k; # increase the buffer size for PHP-FTP

                fastcgi_param GEOIP_ADDR $remote_addr;
                fastcgi_param GEOIP_COUNTRY_CODE $geoip_country_code;
                fastcgi_param GEOIP_COUNTRY_NAME $geoip_country_name;
                fastcgi_param GEOIP_REGION $geoip_region;
                fastcgi_param GEOIP_REGION_NAME $geoip_region_name;
                fastcgi_param GEOIP_CITY $geoip_city;
                fastcgi_param GEOIP_AREA_CODE $geoip_area_code;
                fastcgi_param GEOIP_LATITUDE $geoip_latitude;
                fastcgi_param GEOIP_LONGITUDE $geoip_longitude;
                fastcgi_param GEOIP_POSTAL_CODE $geoip_postal_code;

                if ($rootfolder = "l7") {
                        access_log /var/log/nginx/mywebsite_l7.com-access.log;
                }


        }

        location ~ /\.ht {
                deny all;
        }

}


server {
    if ($host = dev.mywebsite.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = cdn.mywebsite.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = l7.mywebsite.com) {
       return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = www.mywebsite.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = mywebsite.com) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    listen 80 default_server;
    listen [::]:80 default_server;

    server_name mywebsite.com www.mywebsite.com l7.mywebsite.com cdn.mywebsite.com dev.mywebsite.com;
    return 404; # managed by Certbot

}

# configuration file /etc/nginx/snippets/fastcgi-php.conf:
# regex to split $uri to $fastcgi_script_name and $fastcgi_path
fastcgi_split_path_info ^(.+\.php)(/.+)$;

# Check that the PHP script exists before passing it
try_files $fastcgi_script_name =404;

# Bypass the fact that try_files resets $fastcgi_path_info
# see: http://trac.nginx.org/nginx/ticket/321
set $path_info $fastcgi_path_info;
fastcgi_param PATH_INFO $path_info;

fastcgi_index index.php;
include fastcgi.conf;

# configuration file /etc/nginx/fastcgi.conf:

fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;

fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
fastcgi_param  REQUEST_URI        $request_uri;
fastcgi_param  DOCUMENT_URI       $document_uri;
fastcgi_param  DOCUMENT_ROOT      $document_root;
fastcgi_param  SERVER_PROTOCOL    $server_protocol;
fastcgi_param  REQUEST_SCHEME     $scheme;
fastcgi_param  HTTPS              $https if_not_empty;

fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
fastcgi_param  SERVER_SOFTWARE    nginx/$nginx_version;

fastcgi_param  REMOTE_ADDR        $remote_addr;
fastcgi_param  REMOTE_PORT        $remote_port;
fastcgi_param  SERVER_ADDR        $server_addr;
fastcgi_param  SERVER_PORT        $server_port;
fastcgi_param  SERVER_NAME        $server_name;

# PHP only, required if PHP was built with --enable-force-cgi-redirect
fastcgi_param  REDIRECT_STATUS    200;

SOLUTION

As said by Tarun Lalwani, the block priority and order matters. But also, when you want to cache 404 response, you have to check it on the filename of error_page:

location ~ (render_img.php|^/url_rewriting.php$) {
    fastcgi_cache...stuffs to cache...
}

We cannot check location ^~ /render/framed/ { as it's a 404 response which corresponds to /url_rewriting.php.

Answer 1

Your issue is that you need to give priority to your block of .php and not /render/framed/ when a .php is called in that location. It is still caught by the .php block

So you need to use below for your location block. You need to use

location ^~ /render/framed/ {
  ...
}

For understanding the same in details refer to

Nginx location priority