I am testing some ECDHE_RSA ciphers on FIPS platform But the handshake is failing because of the cert and key I am using on Server in OpenSSL.

Same cert and Key are working fine if the platform is non-fips.

I am using a Load Balancer between the client and server. Here is the Configuration:

Using Curl on the client:

curl -v -o ssl_ecdhe.txt -tls1.2

Using Openssl on Server:

/usr/local/ssl/bin/openssl s_server -accept 443 -cert /root/2k.pem -key /root/2k.key -tls1_2 -named_curve secp384r1 -WWW -msg

Configuration on Load Balancer:

slb template server-ssl srvssl 
  cipher TLS1_ECDHE_RSA_AES_128_SHA 
  ec-name secp384r1 
  version 33 33 
slb server main-server 
  port 443 tcp 
slb service-group main-service-gp-ssl tcp 
  member main-server 443 
slb virtual-server main-vip 
  port 80 http 
    service-group main-service-gp-ssl 
    template server-ssl srvssl


Srv31(~)#openssl x509 -in 2k.pem -text -noout

        Version: 1 (0x0)
        Serial Number:
        Signature Algorithm: md5WithRSAEncryption
        Issuer: C=US, ST=California, L=SanJose, O=A1, OU=Regression, CN=Regression/emailAddress=regression@networks.com
            Not Before: Sep 29 21:56:06 2008 GMT
            Not After : Sep 27 21:56:06 2018 GMT
        Subject: C=US, ST=California, L=SanJose, O=A1, OU=Regression, CN=Regression/emailAddress=regression@networks.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Exponent: 65537 (0x10001)
    Signature Algorithm: md5WithRSAEncryption

Please help me with this.

closed as unclear what you're asking by Tobi Nary, StackzOfZtuff, TheJulyPlot, AJ Henderson, Matthew Aug 18 '17 at 15:08

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • Steffen, Thanks for pointing out the issue with the cert. – alpha Aug 18 '17 at 12:52
  • I found out that If I am using elliptic curves like secp256r1 and secp384r1 with ECDHE_RSA ciphers and with above-mentioned certs and keys then I am getting "empty reply from server message", not seen handshake failure anymore. If I am not using elliptic curves then Its working fine even with the 2k.pem. Please tell me how the elliptic curve works and how it affects the ciphers. Also please suggest me a command so that I can generate correct certs and keys that can work with the above configuration. – alpha Aug 18 '17 at 13:05

You don't describe exactly why the handshake is failing (error message) but claim that this is because of the certificate. An obvious problem with this certificate is the totally insecure signature algorithm MD5 which is usually not supported with TLS 1.2 clients anymore:

Signature Algorithm: md5WithRSAEncryption

Apart from that the certificate is self-signed, i.e. there is no trust path to a locally trusted certificate at the client.

Issuer: C=US, ST=California, L=SanJose, O=A1, OU=Regression, CN=Regression/emailAddress=regression@networks.com
Subject: C=US, ST=California, L=SanJose, O=A1, OU=Regression, CN=Regression/emailAddress=regression@networks.com

Also, the subject of the certificate does not match the URL which would also result in validation problems.

And finally this is a x509v1 certificate which compared to a x509v3 certificate has no information about usage restrictions, CRL and OCSP URL's, Subject Alternative Names etc.

  • It's not clear what 'FIPS platform' constitutes in this Q, but even before MD5 was actually broken it was prohibited (in jargon, un-Approved) for FIPS-140 usage simply because it didn't go through the NBS/NIST/CSE process. OTOH the signature algorithm on a selfsigned cert (including root) has no effect on security, and most implementations I've looked at don't even check it (OpenSSL only recently, in 1.1.0, added an option to do so). (Concur with the rest.) – dave_thompson_085 Aug 18 '17 at 11:18
  • Actually, FIPS platform is nothing, it's FIPS-140. I am testing ciphers on a Load Balancer so To check the FIPS compliant ciphers I need to enable FIPS on a Load Balancer. So I am referring it as a FIPS platform. Also, I can't use OpenSSL 1.1.0 because I don't have the authority to change the version. As far as I know, we need more than 2K size certs and keys for FIPS. Please correct me if I am wrong. Can you please share the command so that I can generate a new cert and key of size 2048. – alpha Aug 18 '17 at 12:46
  • @SteffenUllrich, I am testing these ciphers locally. I got it that I am using a wrong cert and key pair. Can you please share the command so that I am able to generate a correct cert and key pair. Actually, It's my first project and so I don't have that much knowledge about it. – alpha Aug 18 '17 at 13:06
  • @alpha: I have no idea what the requirements for your certificate are. self-signed or signed by public CA, CN, any extensions ... ? There are many resources on the internet which show you how to create certificates with openssl so use these once you know your requirements. – Steffen Ullrich Aug 18 '17 at 13:12
  • @SteffenUllrich, I have found a solution. Created a new cert and key pair using this command "openssl req -x509 -nodes -days 1826 -newkey rsa:2048 -keyout ec_supp_fips_rsa.key -out ec_supp_fips_rsa.crt" So basically, I needed "Signature Algorithm: sha1WithRSAEncryption". I think it has something to do with elliptic curves. Thanks for your help. – alpha Aug 18 '17 at 15:35

Not the answer you're looking for? Browse other questions tagged or ask your own question.