This post follows on from my previous post titled “Internet Explorer breaks with TLS1.2 and cert chains containing an MD5 hash“. It turns out that if a website’s certificate chain contains a SHA-512 hash then Internet Explorer fails too. In my previous post, I linked to an article describing the IE error when an MD5 hash is present in the certificate chain. It turns out that article has the necessary information to conclude that a SHA-512 signature would result in the same error.

Note that this error is presented if the web server presents a certificate containing an unsupported signature. If a server only presents a certificate signed with an acceptable signature (eg SHA-256) and presents no other certificates in the chain (acceptable or not signature algorithms) then things work. The problem only highlights itself when an unsupported signature algorithm is presented over the wire. So, you can have a CA’s older cert, which is using an MD5 hash, in your trusted roots and can access a site fine if the web server administrator does not pass the certificate chain. If the certificate chain is then passed over the wire the website will become inaccessible. Note that this is only the case over TLS1.2 connections.

Here is a screenshot of a section of the article I linked to showing the relevant information:

We can see the signature algorithms which IE will accept. Notice that SHA512 is missing, although it is listed within RFC5246 section 7.4.1.4.1 with a code of 6 and is apparently supported by Microsoft (obviously need KB968730 for Windows XP or Windows 2003). Given that you can import a root certificate into Internet Explorer with a SHA512 signature I find it odd that IE will not accept a certificate with a SHA512 signature over the wire.

The frustrating thing with this error is that no real error message is presented. Just a broken web page. Internet Explorer really should highlight this with an error message that indicates the problem – something as simple as “The certificate chain contains certificates with signature algorithms which we do not support”.

My recommendation is thus to not issue certificates to end users with MD5 or SHA512 signatures and to not setup CAs with a signature of SHA512. For new CA certificates a signature of SHA256 should be considered the minimum with SHA384 being recommended. I would also suggest using a 4096-bit RSA key if possible, although 2048-bit keys should be acceptible for a while yet.

EDIT: 2013/06/29 = Note that you can apparently enable the use of SHA512 with RSA certificates (and SHA512 with ECDSA certs too) by editing the registry at:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Cryptography\Configuration\Local\SSL\00010003\Functions (REG_MULTI_SZ)

Add: RSA/SHA512 (or ECDSA/SHA512) to the list of values.
I wonder why Microsoft decided to exclude them by default??