SSL Mutual authentication Fails

Question

When I try to establish a Mutual authentication between server and client, I am getting the errors below.

The timing between the server and client is exactly same and also the certs are proper in server and client. What is the cause for this error and what are the possible ways to occur this error?

  Thread-10, WRITE: TLSv1 Change Cipher Spec, length = 1
[Raw write]: length = 6
0000: 14 03 01 00 01 01                                  ......
*** Finished
verify_data:  { 130, 243, 38, 76, 253, 223, 88, 181, 223, 28, 110, 123 }
***
[write] MD5 and SHA1 hashes:  len = 16
0000: 14 00 00 0C 82 F3 26 4C   FD DF 58 B5 DF 1C 6E 7B  ......&L..X...n.
Padded plaintext before ENCRYPTION:  len = 48
0000: 14 00 00 0C 82 F3 26 4C   FD DF 58 B5 DF 1C 6E 7B  ......&L..X...n.
0010: F5 44 95 0D 5A D0 8E 6F   40 89 10 2D 00 5F BB CF  [email protected]._..
0020: 30 D1 C6 06 0B 0B 0B 0B   0B 0B 0B 0B 0B 0B 0B 0B  0...............
Thread-8, WRITE: TLSv1 Handshake, length = 48
Thread-8, waiting for close_notify or alert: state 1
[Raw read]: length = 5
0000: 15 03 01 00 02                                     .....
[Raw read]: length = 2
0000: 02 2E                                              ..
Thread-8, READ: TLSv1 Alert, length = 2
Thread-8, RECV TLSv1 ALERT:  fatal, certificate_unknown
%% Invalidated:  [Session-4, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA]
Thread-8, called closeSocket()
Thread-8, Exception while waiting for close javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
Thread-8, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
Thread-8, called close()
Thread-8, called closeInternal(true)

Your help is much appreciated.

Answer 1

Well, you say:

the certs are proper in server and client

But at least one of the systems disagrees:

Received fatal alert: certificate_unknown

This message means that one party (you don't say whether you are showing client-side or server-side logs) received an explicit alert message from the server, of class "fatal" and value 46 (0x2E, aka "certificate_unknown"). As specified in the TLS 1.0 standard:

certificate_unknown

    Some other (unspecified) issue arose in processing the
    certificate, rendering it unacceptable.

The "other" means that it is not a problem of the certificate being expired or revoked, or using an unsupported cryptographic algorithm. There can be many causes for a received certificate being "unacceptable", foremost of which being the impossibility of building a valid chain from an a priori known trusted root CA down to the certificate itself (normally, systems should send an "unknown_ca" alert in that case, but many do not have such precision in reporting errors with certificates).

To further investigate the issue, you should:

• Activate full logging on both client and server. You show logs from the system who received the alert message; you should have a look at the logs from the other system, who sent the alert message;

• Verify the configuration of both systems, in particular, for the one who sent the alert message (i.e. not the one which threw the "received fatal alert" exception, but the other one) what trust anchors it is using to validate the peer's certificate;

• Check what certificate chains the systems are actually sending. Since this occurs before the encryption is activated, the certificates ought to be visible to some network monitoring tool à la Wireshark. Notably, check the order: in SSL/TLS, the end-entity certificate comes first, then the CA which issued it, then the CA which issued that CA, and so on.

Remember too that client certificates make sense only as authentication, not as authorization. Even if the client sends a certificate that the server can validate with regards to a trusted root, this only tells to the server who is at the other end of the line. It is still up to the server to decide whether that specific client should be allowed to proceed or not. This normally entails mapping the client certificate to an identity or a role, e.g. by extracting some specific field from the certificate. This process is completely outside of the scope of TLS itself, but must still occur, so there is some software and, crucially, some configuration on the server side that you probably wish to check.

In the other direction (validation of the server's certificate by the client), the rules are specified in RFC 2818, at least in the context of HTTPS: the intended server name (as extracted from the target URL) should appear in the server's certificate. Outside of HTTPS, it is up the each client application to decide whether the certificate they got from the server is the right one, i.e. that the certificate is valid (properly issued by a trusted CA and so on), but also that the identified server is indeed the one the client should be talking to. There again, configuration.