3

I am trying to set up a certificate chain for a lab server. I have created my own root CA, an intermediate CA and a server certificate. I supplied these certificates along with the server key to the openssl s_server command. When I run openssl s_client and connect to that server, openssl complains that there is a self-signed certificate in the chain.

When I connect to a public web server using s_client, however, not only does the server not send all of the certificates in the chain (just the intermediate parent certificate of the server certificate) but openssl doesn't complain about a self-signed certificate, let alone an incomplete certificate chain.

If I use s_server with a CA file containing just the server's parent intermediate certificate, s_client complains that it can't get the local issuer certificate. I never see this error with public web servers even though they don't send the entire certificate chain.

In none of these tests (using my own certificates or public web servers) I am using the -CApath, -CAfile or -verify options with the s_client command.

I don't know what I'm doing wrong. Why does s_client complain about my certificate chain even though I don't use -verify? Why is it complaining about my (I assume) root certificate being self-signed when all root certificates are self-signed?

For example, this is what I get with a public web server:

openssl s_client -showcerts -servername security.stackexchange.com -connect security.stackexchange.com:443
CONNECTED(00000004)
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = *.stackexchange.com
verify return:1
---

But using s_server with my full certificate chain, I get this:

openssl s_client -showcerts -servername server.domain.com -connect server.domain.com:443
CONNECTED(00000004)
depth=2 C = US, ST = State, L = City, O = Company, OU = Company CA
verify error:num=19:self signed certificate in certificate chain
---

Here are my certificates. And yes, I have the constraints CA=TRUE, Digital Signature and Certificate Sign set in my root CA.

openssl x509 -in root_ca.cert.pem -noout -text
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            bc:e0:9f:2a:5d:25:6e:8f
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=US, ST=State, L=City, O=Company, OU=Company CA
        Validity
            Not Before: May 30 22:35:50 2019 GMT
            Not After : May 25 22:35:50 2039 GMT
        Subject: C=US, ST=State, L=City, O=Company, OU=Company CA
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
.........
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                2A:0A:D6:EF:96:02:70:4F:89:7A:69:C5:3E:37:47:EE:B1:E1:92:C0
            X509v3 Authority Key Identifier:
                keyid:2A:0A:D6:EF:96:02:70:4F:89:7A:69:C5:3E:37:47:EE:B1:E1:92:C0

            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
    Signature Algorithm: sha256WithRSAEncryption
.........

root CA:

-----BEGIN CERTIFICATE-----
MIIDjDCCAnSgAwIBAgIJALzgnypdJW6PMA0GCSqGSIb3DQEBCwUAMFMxCzAJBgNV
BAYTAlVTMQ4wDAYDVQQIDAVTdGF0ZTENMAsGA1UEBwwEQ2l0eTEQMA4GA1UECgwH
Q29tcGFueTETMBEGA1UECwwKQ29tcGFueSBDQTAeFw0xOTA1MzAyMjM1NTBaFw0z
OTA1MjUyMjM1NTBaMFMxCzAJBgNVBAYTAlVTMQ4wDAYDVQQIDAVTdGF0ZTENMAsG
A1UEBwwEQ2l0eTEQMA4GA1UECgwHQ29tcGFueTETMBEGA1UECwwKQ29tcGFueSBD
QTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMXCAU2fb4zNVGDryN5f
H4BozBizvkSp71e5NsYa/LW4R/sEedj2k+1szT2rMRrhbkinYCxjx+qMtZ0ZTmBB
y/zbI7XsTaApLZ9f2BMYkfWWi81Plvdg/Z7Z9S5oW3Bnr5ZzhFnAQkVnL5vSbFsG
5dFJMuCUHdGwaAb6ebJCyBxJST3kEd8aog/sdGwH6NPdjel7oc9aCcfp7+Dy7T0g
ThE6vbO4qisTlw+dV+fJ2dGt11vDHc3VnHSaFbb7iuDTG3LeWgF9AhuEkf5uxHQX
zNyx3AkCL9W1keoTTZaIYkHwyTxv/ghrRgLURe8XhC9fpC3cE+wO1Tvf8nmsLEQx
mKMCAwEAAaNjMGEwHQYDVR0OBBYEFCoK1u+WAnBPiXppxT43R+6x4ZLAMB8GA1Ud
IwQYMBaAFCoK1u+WAnBPiXppxT43R+6x4ZLAMA8GA1UdEwEB/wQFMAMBAf8wDgYD
VR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IBAQAyoV6gqDAYWs/biVv/EZ52
QMjrF93gyx8xY7PKMCYOhnzl8qjro33mMjVJLzzvmtjSZ1DHhLk0kgqw2HJJ9kNY
tzXze9h97pWPvA/4g8wQa1Pc2xuVZ7ELxs5qk1Btkgqh3C2DwGU1Vkruch0wTjG+
r28UdvjVfRObg+qx7We7dRAqk3KjXUJvKCZMu0GBYuCrWFrMR6Xc1O47UiEbzzrC
lTeEP6iZKIZI8D1iasrQdjL4CCh3E5w97Hl/NHKPuxTVqs6AdOqDoCBwrQEajO4t
2qHzVBGvTI57PzPYnvc+0fG5n0vn1Dx5SWy7Dl4+51x5vx6tNbTjqhIJnzyuh1l3
-----END CERTIFICATE-----

intermediate CA:

-----BEGIN CERTIFICATE-----
MIIDmzCCAoOgAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwUzELMAkGA1UEBhMCVVMx
DjAMBgNVBAgMBVN0YXRlMQ0wCwYDVQQHDARDaXR5MRAwDgYDVQQKDAdDb21wYW55
MRMwEQYDVQQLDApDb21wYW55IENBMB4XDTE5MDUzMDIyMzYwMVoXDTI5MDUyNzIy
MzYwMVowZjELMAkGA1UEBhMCVVMxDjAMBgNVBAgMBVN0YXRlMRAwDgYDVQQKDAdD
b21wYW55MRMwEQYDVQQLDApDb21wYW55IENBMSAwHgYDVQQDDBdDb21wYW55IElu
dGVybWVkaWF0ZSBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMS3
OdbP2p2vzCVL7MMlSszxefWbsuw/F3t2pgqZivxTebMS8GGVhYfG39wKnJOKi/Q8
i4vUxi7RjQT/enpOmTixrG+IrGK9bg0cJg9tM2FHAMr7uGeoqIY0/02dAX1ffrBI
KP+U5jk6JhEwqG23xXg22TEtmFwYBclTZE5m11tF15FCJEV9r3ljxJlusXcdOEFn
KSrSwYyU352p+WcDuyUbqIeUDzvGpQqmndbEHM3GIo0uvxmkCXr1/Bc6QkhfAJMj
A8y+wvmyZFr1QRhX5R0udyHiwWzjgmSsZ8rDpOlcWKsmon07h+iV5EfEu/fiO1OT
ddW9xSR3oxBTyCErdVkCAwEAAaNmMGQwHQYDVR0OBBYEFHdi+4NsPyB3yTBZnsti
8QKQngr5MB8GA1UdIwQYMBaAFCoK1u+WAnBPiXppxT43R+6x4ZLAMBIGA1UdEwEB
/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IBAQBD
AeasspCf6jpj0+HyMJ9+GyF/5utKy8KOHhNMUOe+sExHvPWnEbtpKZ2gqxQpoNbz
7yFDitLPyEPk2CUNwfMW+QRfy+cqW7ci1mFc8xMo1WP2VPmojCGydZ+9IYUqQ5UE
DcMSSiGneeWhJ71oZwqE9GikPHZfLq6SEeHD3SI9+0vvv0zbcykH+yOvsaDIq+bl
G8rqX1hCs+00L6yBJZHEdz3rKviJgxT3eYkfekd65p92Ehqtzd/ZehqR5P8dM+h/
qnpZOJAvtVhOArP7GwyPcqapToXVEppFhS1nSlxDauTa1VrvzkIgHOzOR/v8EGyH
lublY3WfPEBZxdE3zKln
-----END CERTIFICATE-----

server certificate:

-----BEGIN CERTIFICATE-----
MIIEpDCCA4ygAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwZjELMAkGA1UEBhMCVVMx
DjAMBgNVBAgMBVN0YXRlMRAwDgYDVQQKDAdDb21wYW55MRMwEQYDVQQLDApDb21w
YW55IENBMSAwHgYDVQQDDBdDb21wYW55IEludGVybWVkaWF0ZSBDQTAeFw0xOTA1
MzAyMjM2MDhaFw0yNDA1MjgyMjM2MDhaMIGRMQswCQYDVQQGEwJVUzEOMAwGA1UE
CAwFU3RhdGUxDTALBgNVBAcMBENpdHkxEDAOBgNVBAoMB0NvbXBhbnkxEzARBgNV
BAsMCkNvbXBhbnkgQ0ExHjAcBgNVBAMMFXNlcnZuYW1lIDE5MDUzMDE3MzYwNDEc
MBoGCSqGSIb3DQEJARYNcm9vdEBzZXJ2bmFtZTCCASIwDQYJKoZIhvcNAQEBBQAD
ggEPADCCAQoCggEBAMDHXE+tXVz+vyj2gw8Fzy9QbNFj4Xgx9hUuvSnv5rDbdh1K
B0TtQ+JfGqD3onnRUZ6vOEe5m9yfc0Kw2QnyPbEwB3MKc61I0Ls3ve4I2v6auHS0
NDdZyUtTHGlF95XL9c+UqzaOn2eaHQM4VvhhVgDAswXR4Br6m+ID0cOCmS+YHWRG
HyvRKRdKn5w7MZ3CKJ7VDgYutREfw6lwoGvgWUTNDYbgLUY26AH0lhMrMjzrp1bx
SI+2nbliGBQd/81otWD9FjnbTqBIdDKQlTyrqHJWF0ZWCjZiVJuUC1hKzBySX59R
rD7gqARDeSbhlln6LGo31BsqwoYgvvXe/26pdxECAwEAAaOCAS4wggEqMAkGA1Ud
EwQCMAAwEQYJYIZIAYb4QgEBBAQDAgZAMDMGCWCGSAGG+EIBDQQmFiRPcGVuU1NM
IEdlbmVyYXRlZCBTZXJ2ZXIgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFGSrkvKc5psT
2b+CXmOo1CCICw5OMHwGA1UdIwR1MHOAFHdi+4NsPyB3yTBZnsti8QKQngr5oVek
VTBTMQswCQYDVQQGEwJVUzEOMAwGA1UECAwFU3RhdGUxDTALBgNVBAcMBENpdHkx
EDAOBgNVBAoMB0NvbXBhbnkxEzARBgNVBAsMCkNvbXBhbnkgQ0GCAhAAMA4GA1Ud
DwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDATATBgNVHREEDDAKgghzZXJ2
bmFtZTANBgkqhkiG9w0BAQsFAAOCAQEAwlOx4gPc7UPC/FRWtsjzeTNsXXXY4zLg
NnuBw1y3YCzInLV0w97QvK+QfZ6EANgou7Wl4q3k4SodkAL9GXnfecW2EN13geGa
SP+hPqT84jqSsk5mqOYhpyKZHjXTYPUYyQ9p0dIy/Dm9dNjOvHpefsK/DRHJPkyu
NS5oZE2CTmZwnNq6w5I/53Gd0dA54F2N3v169ygKCr6WlM95lEwZgZNTLNqyyZnO
P7Kbt4ufB7u76ky3X9YvAwSwqj4VzqAm4d/xKAilHEZYWlL2Mo6FqfuXij/avoUc
OoGt/A6yzFGLBrGEJf4iQR9Iwgwo3+yZqeMHykM/e44MwCVTA6MH1g==
-----END CERTIFICATE-----
  • 1
    s_client does verify the cert chain using default roots installed by your distro. By default it doesn't stop after verification failed because it's a test tool, but it does verify. If you specify -CAfile, does it work? The complaint about self-signed is because the domain's cert is issued by something not trusted by the trusted roots, not because the root itself is self-signed (as you noted, all roots are self-signed). – Z.T. May 30 at 23:50
1

First, it is quite correct for a TLS/SSL server to send the intermediate aka chain cert(s) but not the root cert; see rfc 5246 sec 7.4.2 or the slightly more verbose version in rfc 8446 sec 4.4.2.

As Z.T. mostly-correctly commented, -verify is the default for s_client (you don't need to specify it) and if you don't specify -CAfile and/or -CApath by default (except for bugs in some obsolete versions) it uses a default truststore, configured at compile time, which can include either a single file with concatenated PEM certs or a directory containing separate PEM files with names or links using truncated subject hashes usually created by c_rehash (add) or, as commented, manually determined with x509 -hash; see the man page for verify(1ssl) on your system or on the web. OpenSSL itself doesn't provide the root certs that go in such a truststore; if you are using a distro or other packaged version, the builder usually configures OpenSSL to use a set of root certs provided by the distro or package. (On distros I know, this distro-provided truststore is also used for other software including NSS, GNUtls, and Java; see below.) If you (or someone) built OpenSSL by hand, you have to do this yourself. Since you are not getting a verify error on public servers, your build presumably is using a truststore that has (at least some) public CA's preinstalled; you can see where this is with openssl version -d (plus the hardcoded names cert.pem and/or certs).

You can add your root or other anchor (see below) to this default truststore, or use -CAfile and/or -CApath to specify a custom one containing your own root(s) or anchor(s). On the distros I know, and probably others, you should NOT hand-modify the default files because they are automatically generated by a process that also sets the truststores used by other software such as NSS, GNUtls, and Java; these need to contain the same data (certs) but in different formats. Instead see e.g. man update-ca-trust for RedHat family or man update-ca-certificates for Debian family.

OpenSSL will use an intermediate (aka chain) cert or certs in the truststore to build the cert chain if needed, i.e. if not sent by the server (in violation of the RFC, but many do that), but historically it will only accept a chain -- either fully received from the server or (partly) built from the local truststore -- if it ends at a root that is in the local truststore. For recent versions, namely 1.0.2 up but only documented since 1.1.0, there is an option -partial_chain which does accept a chain ending in an intermediate (non-root) that is in the local truststore.

  • Thanks! I've figured out part of my confusion. Although my version of OpenSSL does not include a default truststore, at some point last year I must have done something that caused it to get copies of a large number of root-level certificates. They were all stored in /var/ssl/certs, but that directory does not exist by default on AIX. I renamed the directory and the openssl command then behaved consistently for anything I connected to--"verify error:num=20:unable to get local issuer certificate". Now I just need to figure out why it considers my certificate chain to be self-signed. – Steve Talmage Jun 2 at 3:43
  • It considers your root cert to be self-signed because it is; that's the definition of a root cert. If you configure the server to not send the root cert s_client will report verify error 20 instead of verify error 19, if you think that is somehow significant, but in either case the cert cannot be verified. What matters about a root is not that it is self-signed, which it always is, but whether it is trusted. – dave_thompson_085 Jun 2 at 6:12
  • 1
    I am aware that all root certificates are self-signed. I was just trying to figure out why s_client was complaining about mine and no one else's. After I discovered that a truststore actually existed on my system, I added my root certificate to it, used x509 -hash to get the hash value, created a symbolic link from the hash value to my root certificate, and s_client stopped complaining. Now I fully understand s_client's criteria for determining if a root certificate is to be trusted. Thanks for your help! – Steve Talmage Jun 3 at 21:12
  • As I said, the difference is that if the server sends a chain including the root, and the root is not trusted, it gives error 19. If the server sends a chain excluding the root, and the root is not trusted, it gives error 20. That difference is not in the cert, but in the server. If the root is in the truststore -- and being in the default dir with a hash name is one way, another is to put it in the default file -- verification succeeds, whether or not the server sends it. – dave_thompson_085 Jun 4 at 7:57
  • Right. I understood that part after you initially explained it. I just didn't realize that I had a truststore in the first place. – Steve Talmage Jun 5 at 16:45
0

Regarding the fullchain I generated mine for use with Apache2 using:

cat intermediateCA.crt rootCA.crt > fullchain.crt

Without testing your exact circumstances I cannot suggest more than the order the web server is reading the certificate chain.

For OpenSSL stating your rootCA being self-signed. This could be due to your rootCA not being installed into your certificate store on the OS (or even web browser). Installing a .crt file across operating systems can vary, Adding trusted root certificates to the server will explain how. Bear in mind that Mozilla Firefox uses its own certificate store.

When I first started being my own certificate authority, for similar purposes, I began issuing my own rootCA using a tutorial by Jaime Linux which may be more appropriate given you are issuing for a lab server, as the tutorial explains creating a flat database to track signed certificates.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.