AWS ELB -> Backend Server over HTTPS with Self-Signed Certificate

Question

I already have HTTPS in place to terminate external HTTPS connections at my AWS ELB. I'm now attempting to secure my connections between my ELB and my backend NGINX servers on EC2 using HTTPS with a self-signed certificate. I've followed the documentation, but accessing the server over HTTPS results in a 408 HTTP timeout. I can't seem to get any debugging information to determine where things are failing.

• I've confirmed that the security groups permit connections between the ELB and NGINX on EC2.
• I've convirmed that the VPC allows traffic to be routed between the ELB and EC2 nodes (also HTTP works fine).
• I've confirmed that the HTTPS listener on the EC2 node is operating (I can hit it directly w/o going to the ELB.
• I have created an ELB policy of type PublicKeyPolicyType, and associated my public key.
• I have created an ELB policy of tyep BackendServerAuthenticationPolicyType, and associated it with the PublicKeyPolicyType.
• I have associated the BackendServerAuthenticationPolicyType with with the ELB.
• I have ensured that the SSLNegotiationPolicyType supports the algorithms and ciphers I have specified in my NGINX config.
• I see HTTP requests in my NGINX access logs, but not HTTPS requests.

Is there any way I can get any additional diagnostic information to test this?

Here is my ELB configuration:

$ aws elb describe-load-balancers --load-balancer-name <MY-ELB-NAME>

{
    "LoadBalancerDescriptions": [
        {
            "Subnets": [
                "<REDACTED>",
                "<REDACTED>",
                "<REDACTED>"
            ],
            "CanonicalHostedZoneNameID": "<REDACTED>",
            "VPCId": "<REDACTED>",
            "ListenerDescriptions": [
                {
                    "Listener": {
                        "InstancePort": 80,
                        "LoadBalancerPort": 80,
                        "Protocol": "HTTP",
                        "InstanceProtocol": "HTTP"
                    },
                    "PolicyNames": []
                },
                {
                    "Listener": {
                        "InstancePort": 443,
                        "SSLCertificateId": "<REDACTED>",
                        "LoadBalancerPort": 443,
                        "Protocol": "HTTPS",
                        "InstanceProtocol": "HTTPS"
                    },
                    "PolicyNames": [
                        "ELBSecurityPolicy-2015-05"
                    ]
                }
            ],
            "HealthCheck": {
                "HealthyThreshold": 2,
                "Interval": 30,
                "Target": "HTTP:80/health",
                "Timeout": 10,
                "UnhealthyThreshold": 2
            },
            "BackendServerDescriptions": [
                {
                    "InstancePort": 443,
                    "PolicyNames": [
                        "MyBackendServerAuthenticationPolicy"
                    ]
                }
            ],
            "Instances": [
                {
                    "InstanceId": "<REDACTED>"
                }
            ],
            "DNSName": "<REDACTED>.us-west-2.elb.amazonaws.com",
            "SecurityGroups": [
                "<GROUP_ID>"
            ],
            "Policies": {
                "LBCookieStickinessPolicies": [],
                "AppCookieStickinessPolicies": [],
                "OtherPolicies": [
                    "ELBSecurityPolicy-2015-05",
                    "MyBackendServerAuthenticationPolicy",
                    "MyPublicKeyPolicy"
                ]
            },
            "LoadBalancerName": "<MY-ELB-NAME>",
            "CreatedTime": "2016-03-23T20:58:49.490Z",
            "AvailabilityZones": [
                "us-west-2a",
                "us-west-2b",
                "us-west-2c"
            ],
            "Scheme": "internal",
            "SourceSecurityGroup": {
                "OwnerAlias": "<REDACTED>",
                "GroupName": "<GROUP_NAME>"
            }
        }
    ]
}

Here are my ELB policies:

$ aws elb describe-load-balancer-policies --load-balancer-name <MY-ELB-NAME>
{
    "PolicyDescriptions": [
        {
            "PolicyAttributeDescriptions": [
                {
                    "AttributeName": "Reference-Security-Policy",
                    "AttributeValue": "ELBSecurityPolicy-2015-05"
                },
                ...
                {
                    "AttributeName": "Protocol-TLSv1.2",
                    "AttributeValue": "true"
                },
                ...
                {
                    "AttributeName": "ECDHE-RSA-AES128-GCM-SHA256",
                    "AttributeValue": "true"
                },
                ...
            ],
            "PolicyName": "ELBSecurityPolicy-2015-05",
            "PolicyTypeName": "SSLNegotiationPolicyType"
        },
        {
            "PolicyAttributeDescriptions": [
                {
                    "AttributeName": "PublicKeyPolicyName",
                    "AttributeValue": "MyPublicKeyPolicy"
                }
            ],
            "PolicyName": "MyBackendServerAuthenticationPolicy",
            "PolicyTypeName": "BackendServerAuthenticationPolicyType"
        },
        {
            "PolicyAttributeDescriptions": [
                {
                    "AttributeName": "PublicKey",
                    "AttributeValue": "<REDACTED>"
                }
            ],
            "PolicyName": "MyPublicKeyPolicy",
            "PolicyTypeName": "PublicKeyPolicyType"
        }
    ]
}

Here is my NGINX config:

worker_processes 10;
worker_rlimit_nofile 8192;
events {
  worker_connections  4096;
}

error_log syslog:server=unix:/dev/log error;
pid       logs/nginx.pid;

http {
  default_type  application/octet-stream;

  log_subrequest on;
  access_log syslog:server=unix:/dev/log,severity=debug extended;

  tcp_nodelay    on;
  tcp_nopush     on;

  server_tokens off;

  upstream api {
    server localhost:8080;
  }

  server {
    listen 80 default_server;
    listen [::]:80 default_server;

    location / {
      # Redirect all other HTTP requests to HTTPS with a 301 Moved Permanently response.
      return 301 https://$host$request_uri;
    }
  }

  server {
    listen 443 ssl;
    listen [::]:443 ssl;

    ssl_certificate /path/to/ssl.crt;
    ssl_certificate_key /path/to/ssl.key;
    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;ECDHE

    # Diffie-Hellman parameter for DHE ciphersuites, recommended 2048 bits
    ssl_dhparam /path/to/dhparam.pem;

    # modern configuration. tweak to your needs.
    # See: https://mozilla.github.io/server-side-tls/ssl-config-generator/
    ssl_protocols TLSv1.2;
    ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256';
    ssl_prefer_server_ciphers on;

    add_header Strict-Transport-Security "max-age=15768000; includeSubDomains;";

    # Our main location to proxy everything else to the upstream
    # server, but with the added logic for enforcing HTTPS.
    location / {
      proxy_http_version 1.1;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_redirect off;
      proxy_next_upstream error;

      proxy_pass http://api;
    }
  }
}

I'm generating keys/certificates using the following commands:

$ openssl genrsa \
  -out /path/to/ssl.key 2048
$ openssl req \
  -sha256 \
  -new \
  -key /path/to/ssl.key \
  -out /path/to/ssl.csr
$ openssl x509 \
  -req \
  -days 365 \
  -in /path/to/ssl.csr \
  -signkey /path/to/ssl.key \
  -out /path/to/ssl.crt
$ openssl dhparam -out /path/to/dhparam.pem 2048

Answer 1

Adding some non-EC DHE ciphers to the NGINX config solved this for me. I've switched to the following config in the HTTPS listener in nginx.conf:

  # intermediate configuration. tweak to your needs.
  ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS';

I'd like to drop all non-EC DHE ciphers and only support ECDHE. I suspect this fixes the problem because I'm generating an RSA key/cert instead of an EC key/cert. If anyone knows how I can properly generate an EC key/cert, and then correctly extract the EC public key for upload to AWS, please improve upon my answer. I've attempted to generate an EC key/cert, but when I try to create the ELB public key policy, AWS reports it as an invalid public key.