Django REST Framework pagination links do not use HTTPS

Question

I'm setting up pagination for a certain DRF endpoint that works well- however when deployed on my server, which uses HTTPS, the links to the next & previous pages are formed with http:// instead of https://. This causes the requests for next/previous pages to be blocked by the browser.

I've double checked that the initial request was issued, with HTTPS, and the 2nd answer to this question states that it should be using HTTPS in the formed URLs since the request came over HTTPS.

The first answer to that same question didn't help either- I added the X-Forwarded-Proto line to my nginx config and reloaded, to no avail.

The DRF docs mention that reverse() should behave as the base Django reverse, however it seems pretty clear that the initial request is HTTPS while the returned URL is HTTP.

Here are a couple screenshots that show the initial request (https://<domain>.com/api/leaderboard/):

With the response containing next: http://<domain>.com/api/leaderboard/?page=2):

I figured this would be a simple setting, but haven't been able to find anything after searching both this site and the DRF site.

This is my nginx configuration:

 location / {
    # proxy_pass http://127.0.0.1:9900;
    proxy_set_header X-Forwarded-Host $server_name;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-Proto $scheme;
    add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';

    root /opt/app/client/dist;
    index index.html index.htm;

}

This question contains a pretty detailed answer, but ultimately says that urls are formed with the same protocol as the request, which doesn't seem to be the case here. Do I need to set that Django SECURE_PROXY_SSL_HEADER? I wasn't sure, given the warning that it's potentially insecure.

Answer 1

Since this is the first SO post that comes up in google search result, I think it would be useful to share how I solved it. Mine has a Kubernetes flavor, but under the hood the logic is not so different than the others.

I'm using Kubernetes ingress controller sitting in front of Django, so things might be different to you, but I'll summarize my context so you can see if it's useful to you:

The Problem

So I got the same issue as OP, the pagination link from DRF is always http even if I'm hitting the api endpoint using https. You can see this more clear if you have setup the browser API which comes with the Django REST Framework (DRF), and go to the api root page. I see all links that's generated by DRF are all http regardless of what protocol I use when visiting the site.

My Context

The stack, by order from external world till Django:

1. Kuberenetes Nginx Ingress Controller (w/ SSL setup using letsencrypt)
2. Ingress rule points to Django service
3. Gunicorn starts the Django server
4. Django reads the settings.py and start serving requests

How I Debug and Found out the Cause

1. Dump nginx.conf of the ingress controller. In case you don't know how to to this: first figure out the pod name by kuberctl get pods -n <namespace of your ingress controller>, write that name down, then dump the file by kubectl exec -it -n ingress_controller_namespace ingress_controller_pod_name cat /etc/nginx/nginx.conf > nginx.conf.
2. Look at the nginx.conf, check if you have proxy_set_header X-Forwarded-Proto $scheme; in the location section for your website domain, or similar thing that sets the proxy header X-Forwarded-Proto properly. By default, the Kubernetes nginx ingress controller should already have this line (or the equivalent) for you.
3. Next the request is directed to Gunicorn. One gotcha is you will want to add --forwarded-allow-ips="*" to your gunicorn command, or if you know your nginx server IP, you can limit down to that one, so that gunicorn will forward the headers for you, otherwise gunicorn will strip off those headers. So you end up with command like gunicorn django_server.wsgi:application --forwarded-allow-ips="*" --workers=${PROPER_WORKER_NUM} --log-level info --bind 0.0.0.0:8001. You can have 4 workers by specifying workers=4, usually depends on the CPU you have on the server. The --log-level info is just for debug purpose, it's optional.
4. In Django, you will need to have SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https') in settings.py, which other answer has mentioned. You may wonder why the prefix HTTP_ while in nginx our header name is X-Forwarded-Proto. This is because WSGI will add the prefix to headers that it recognizes. This line basically says, if the header HTTP_X_FORWARDED_PROTO equals to the string "https", then Django recognizes the request as secure. This will affect several behaviors in Django, e.g. you will get request.is_secure == True, request.build_absolute_uri(None) == 'https://...', and most important, Django REST Framework pagination link will now use https! (As long as you indeed hit the api by https)

Alright, now you can test again. If DRF gives you https now - congrats. Or, if you are same as me, after trying above, still no luck - DRF still generating the damn http links. I want to share some tipbits when I was debugging like hell:

Print out the following values in Django, either by print(), if you have DEBUG=True and have access to the server log, or just pass them to template context and show them in a html page, if that makes it easier for you to test in a production environment where SSL is enabled.

• request.is_secure(): if you're failing to get DRF to use http, chances are you are getting False.
• request.META gives you all headers that Django received. Did the header HTTP_X_FORWARDED_PROTO show up? What value is it?
  • I want to share that this is the a-ha moment for me: the value of HTTP_X_FORWARDED_PROTO is https,https, which tells me that it somehow having duplicated values, and I probably have proxy_set_header set twice, and Bingo! By default K8 ingress controller already has this line, and since I added it again through a location-snippet, now I have two lines of proxy_set_header X-Forwarded-Proto $scheme; in total. After removing my snippet, DRF shows https and I was so so happy to get this done. It wasn't that straight forward because I thought proxy_set_header will overwrite and "set" the header value. But seems like it just keep appending.

Answer 2

Do I need to set that Django SECURE_PROXY_SSL_HEADER? I wasn't sure, given the warning that it's potentially insecure.

Yes you do. However, you need to take care about what's your doing. In particular making sure it drop the X-Forwarded-Proto from outside.