Firefox 33.0 won't open a specific local application: Error code: sec_error_invalid_key

Question

Firefox upgraded to version 33.0 this morning. Since then, I cannot load a specific local application over HTTPS -- note that it uses a self-signed certificate. It displays the following error message:

The key does not support the requested operation. (Error code: sec_error_invalid_key)

I cannot see anything in Firebug. I restarted Firefox in safe mode, to make sure no add-on was the problem. I also cleaned my cache and cookies. The same application opens fine with Chrome, and Firefox can open other applications that use HTTPS with a self-signed certificate.

Any idea how I could troubleshoot this issue?

Edit: Mozilla has made several important changes to security in Firefox 33.0. Details can be found here.

In my particular situation, the self-signed certificate was blocked because it was deemed too weak:

RSA 512, 1000 and 1023-bit certificates are now blocked by Firefox since they are not sufficient for security. Most certificates currently being issued should have a 2048-bit key length.

Answer 1

I have encountered the same problem after upgrading to Firefox 33 with Tomato router. The key length is a problem here.

Tomato generates 512 bit long RSA key by default. However, Firefox 33 requires minimum 1024 bit key.

I had to manually generate a longer key in Tomato.

I did that following way:

1. Log in using ssh to the router.
2. cd /tmp
3. cp /usr/sbin/gencert.sh .
4. chmod +w gencert.sh

5. Edit the gencert.sh file you copied and change the following line:

   openssl req -new -out /tmp/cert.csr -config openssl.config -keyout /tmp/privkey.pem -newkey rsa:512 -passout pass:password

   into:

   openssl req -new -out /tmp/cert.csr -config openssl.config -keyout /tmp/privkey.pem -newkey rsa:1024 -passout pass:password

6. ./gencert.sh $(date +%s)
7. nvram unset https_crt_file
8. nvram commit
9. service httpd restart

Now httpd will use the new certificate. If you have "Save in NVRAM" checkbox enabled it will be saved in NVRAM and survive router reboot.

Do not check "Regenerate" checkbox, because automatically regenerated certificates are still 512 bit long.

If you remove your certificate from NVRAM, you must repeat procedure described above.

Firefox 34 and newer:

Starting from Firefox 34 you need to additionally enable SSL 3.0 support in Firefox configuration:

1. Enter about:config address in the URL bar.

2. Set the following options to 0:

   security.tls.version.fallback-limit
   security.tls.version.min