An introduction to PKI, TLS and X.509, from the ground up.

Public Key Infrastructure (PKI) provides a framework of encryption and data communications standards used to secure communications over public networks. At the heart of PKI is a trust built among clients, servers and certificate authorities (CAs). This trust is established and propagated through the generation, exchange and verification of certificates.

This article focuses on understanding the certificates used to establish trust between clients and servers. These certificates are the most visible part of the PKI (especially when things break!), so understanding them will help to make sense of—and correct—many common errors.

As a brief introduction, imagine you want to connect to your bank to schedule a bill payment, but you want to ensure that your communication is secure. "Secure" in this context means not only that the content remains confidential, but also that the server with which you're communicating actually belongs to your bank.

Without protecting your information in transit, someone located between you and your bank could observe the credentials you use to log in to the server, your account information, or perhaps the parties to which your payments are being sent. Without being able to confirm the identity of the server, you might be surprised to learn that you are talking to an impostor (who now has access to your account information).

Transport layer security (TLS) is a suite of protocols used to negotiate a secured connection using PKI. TLS builds on the SSL standards of the late 1990s, and using it to secure client to server connections on the internet has become ubiquitous. Unfortunately, it remains one of the least understood technologies, with errors (often resulting from an incorrectly configured website) becoming a regular part of daily life. Because those errors are inconvenient, users regularly click through them without a second thought.

Understanding the X.509 certificate, which is fully defined in RFC 5280, is key to making sense of those errors. Unfortunately, these certificates have a well deserved reputation of being opaque and difficult to manage. With the multitude of formats used to encode them, this reputation is rightly deserved.

An X.509 certificate is a structured, binary record. This record consists of several key and value pairs. Keys represent field names, where values may be simple types (numbers, strings) to more complex structures (lists). The encoding from the key/value pairs to the structured binary record is done using a standard known as ASN.1 (Abstract Syntax Notation, One), which is a platform-agnostic encoding format.

Designed for efficient operation across a broad range of hosts, ASN.1 allows several ways to encode values using its "basic encoding rules" (BER). As you will see shortly, multiple encodings for the same data will not work for your certificates, so the X.509 standard designated a subset of BER as the "distinguished encoding rules" (DER). The use of DER ensures that there is exactly one way any value might be encoded.

A certificate may be distributed in this raw, binary DER format. Since binary data is not terminal- or email-friendly, the encodings defined in the Privacy Enhanced Mail (PEM) standard are commonly applied. Although PEM is largely an obsolete standard, it defines methods of encoding binary material in a text-safe format. Most important, it defines the base64 scheme for encoding binary data into text and specifies the use of encapsulation boundaries to signal the beginning and ending of encoded content. The use of the PEM format is generally preferred on Linux servers (more on that later in this article).

Certificates may be distributed in a multitude of other formats as well. What is important to understand is that regardless of the encoding format—DER, PEM, PFX or something else—all certificates are basically the same when they are decoded. Tools, such as OpenSSL, are able to read or convert any of these formats easily.

The certificate encodes two very important pieces of information: the server's public key and a digital signature that can be used to confirm the certificate's authenticity. Additionally, the certificate will include metadata used by the CA to track the certificate and provide guidelines on how the public key can be used.

Public key cryptography, also known as asymmetric key cryptography, provides a mechanism to establish a secured communication channel over an insecure network. Using the server's public key, the client and server are able to negotiate a shared symmetric key securely, which can be used to secure communications.

Encryption: Symmetric vs. Asymmetric

Two broad types of encryption are widely used to secure web traffic: symmetric key and asymmetric key.

In both types of encryption, the key refers to a passphrase (of sorts) required to encrypt or decrypt the data. Without the key, an entity attempting to read the data would be unable to read the encrypted content. Likewise, malicious content would be difficult to generate.

Symmetric key encryption uses the same key on both sides of the communication channel to encrypt or decrypt data. Symmetric key encryption is implemented in algorithms such as AES or DES. It has the advantage of being very fast with a low overhead, but a secure channel must exist between the two parties through which the key may be exchanged.

Asymmetric key encryption uses a pair of keys, known as a private key and a public key. These keys are different values. Data encrypted using the private key can be decrypted only using the public key. The reverse is also true: data encrypted with the public key can be decrypted only with the private key. Asymmetric encryption algorithms include algorithms such as RSA, DSA and ECDSA.

While symmetric keys have desirable properties for communication, they must be generated and exchanged in a secure manner. Asymmetric algorithms fit this niche, and you can use them as the foundation for building a secured channel.

But, how do you (as the client) know that the public key can be trusted as authentic? You can use the certificate authority (CA), a trusted third party, as a mediator of sorts. By submitting the certificate to be signed by the CA, the owner of the server gives consent for the certificate to be "authenticated" by the CA. If the client, who trusts that the CA has verified the server properly, can confirm that the signature is valid, the certificate can be trusted.

A hash, often using the SHA256 algorithm, is a digital fingerprint of the data. If you change a single bit in the data, the hash will change. By computing a hash over the DER-encoded public key section of the certificate and then signing the hash with its own private key, the CA is giving its stamp of approval on the certificate. This signed hash value is the signature appended to the certificate.

When the client receives the server's certificate, the client can create the hash over the same data in the certificate that was signed by the CA. If the client is able to decrypt the signature using the CA's public key and it matches the hash the client computed itself, the client can be certain that the server's certificate was endorsed by the CA.

The final piece of the puzzle is understanding that the client must trust the CA explicitly. This is done by adding the CA's public key to the client's "trusted key store". As the user of common web services, you don't need to think much about certificate authorities. Both your operating system and your web browser ship with curated lists of authorities, which have been pre-selected as trustworthy for you. Still, from time to time, it may be necessary to install a certificate from a non-standard CA.

Now that you understand the basics of how a certificate is put together, let's look at a real-world example. Using the following command, you can pull the certificate from the google.com domain:

$ openssl s_client -showcerts -connect google.com:443 </dev/null

This command connects to the remote server using TLS and dumps a large amount of information about the TLS handshake to your console. If you had not redirected standard input from /dev/null, the connection would remain open, allowing you to interact directly with the server on port 443 (you can close this connection explicitly with Ctrl-D).

Below is an (abbreviated) view of the output, which I explore through the remainder of this article:

$ openssl s_client -showcerts -connect google.com:443
 ↪</dev/null
CONNECTED(00000003)
depth=2 OU = GlobalSign Root CA - R2, O = GlobalSign, CN =
 ↪GlobalSign
verify return:1
depth=1 C = US, O = Google Trust Services, CN = Google
 ↪Internet Authority G3
verify return:1
depth=0 C = US, ST = California, L = Mountain View, O =
 ↪Google LLC, CN = *.google.com
verify return:1
---
Certificate chain
0 s:/C=US/ST=California/L=Mountain View/O=Google
 ↪LLC/CN=*.google.com
  i:/C=US/O=Google Trust Services/CN=Google Internet
 ↪Authority G3
-----BEGIN CERTIFICATE-----
MIIIgjCCB2qgAwIBAgIITFQTbb/xK/QwDQYJKoZIhvcNAQELBQAwVDELMAkGA1UE
BhMCVVMxHjAcBgNVBAoTFUdvb2dsZSBUcnVzdCBTZXJ2aWNlczElMCMGA1UEAxMc
R29vZ2xlIEludGVybmV0IEF1dGhvcml0eSBHMzAeFw0xODEwMzAxMzE1MDVaFw0x
        <... content omitted ...>
OTAxMjIxMzE1MDBaMGYxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlh
pvxysdjrJ8qfUyD0AY/Z8dCs1RQfx8SKbXuoML9e0X5uxRmeyjQ0s+BPJDIQG5b8
IGSRGSm8vWtg9vz/GDZIErtEO1kgXOslBBGL5NSCFpxkp1lh/Usi3nFzPcU6Fbvx
WMSdoZZKpgy5+6GGjYv/dEyEdnXYzg==
-----END CERTIFICATE-----
1 s:/C=US/O=Google Trust Services/CN=Google Internet
 ↪Authority G3
  i:/OU=GlobalSign Root CA - R2/O=GlobalSign/CN=GlobalSign
-----BEGIN CERTIFICATE-----
MIIEXDCCA0SgAwIBAgINAeOpMBz8cgY4P5pTHTANBgkqhkiG9w0BAQsFADBMMSAw
HgYDVQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMjETMBEGA1UEChMKR2xvYmFs
U2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjAeFw0xNzA2MTUwMDAwNDJaFw0yMTEy
        <... content omitted ...>
FIwsIONGl1p3A8CgxkqI/UAih3JaGOqcpcdaCIzkBaR9uYQ1X4k2Vg5APRLouzVy
7a8IVk6wuy6pm+T7HT4LY8ibS5FEZlfAFLSW8NwsVz9SBK2Vqn1N0PIMn5xA6NZV
c7o835DLAFshEWfC7TIe3g==
-----END CERTIFICATE-----
---
        <... content omitted ...>
---
DONE

This output shows that the server returned not one, but two PEM-encoded certificates. Each certificate is bracketed by -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- markers. Although they look intimidating, remember that they are nothing more than base64-encoded DER data, and you have tools that can look inside!

OpenSSL provides a little bit of context just above the BEGIN mark of each certificate with the s: (subject) and i: (issuer) tags. The subject tells you what server (or other entity) the certificate was generated for, while the issuer tells you which certificate authority signed the certificate. If the subject matches the server to which you are connecting and you trust the issuer, you can be on your way.

At this point, let's pause to note that the X.509 standard used to encode certificates descends from the same series of X.500 standards as LDAP. You can see some of this common lineage in the directory syntax used to identify the subject and issuer:

C=US,ST=California,L=Mountain View,O=Google LLC,CN=*.google.com

If you're not familiar with LDAP naming standards, it's most important to understand that the "CN" is the "common name" of the certificate owner, with the remainder of the name being used for organization within a directory.

You can decode these certificates using OpenSSL. Extract the first certificate (CN=*.google.com) into a file named "google_com.crt" and the second certificate (CN=Google Internet Authority G3) into a file named "google_authority_g3.crt". Include the BEGIN and END wrapping each certificate, but nothing more. When you're done, you should have two files that look like this:

$ cat google_com.crt
-----BEGIN CERTIFICATE-----
MIIIgjCCB2qgAwIBAgIITFQTbb/xK/QwDQYJKoZIhvcNAQELBQAwVDELMAkGA1UE
BhMCVVMxHjAcBgNVBAoTFUdvb2dsZSBUcnVzdCBTZXJ2aWNlczElMCMGA1UEAxMc
R29vZ2xlIEludGVybmV0IEF1dGhvcml0eSBHMzAeFw0xODEwMzAxMzE1MDVaFw0x
        <... content omitted ...>
OTAxMjIxMzE1MDBaMGYxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlh
pvxysdjrJ8qfUyD0AY/Z8dCs1RQfx8SKbXuoML9e0X5uxRmeyjQ0s+BPJDIQG5b8
IGSRGSm8vWtg9vz/GDZIErtEO1kgXOslBBGL5NSCFpxkp1lh/Usi3nFzPcU6Fbvx
WMSdoZZKpgy5+6GGjYv/dEyEdnXYzg==
-----END CERTIFICATE-----

$ cat google_authority_g3.crt
-----BEGIN CERTIFICATE-----
MIIEXDCCA0SgAwIBAgINAeOpMBz8cgY4P5pTHTANBgkqhkiG9w0BAQsFADBMMSAw
HgYDVQQLExdHbG9iYWxTaWduIFJvb3QgQ0EgLSBSMjETMBEGA1UEChMKR2xvYmFs
U2lnbjETMBEGA1UEAxMKR2xvYmFsU2lnbjAeFw0xNzA2MTUwMDAwNDJaFw0yMTEy
        <... content omitted ...>
FIwsIONGl1p3A8CgxkqI/UAih3JaGOqcpcdaCIzkBaR9uYQ1X4k2Vg5APRLouzVy
7a8IVk6wuy6pm+T7HT4LY8ibS5FEZlfAFLSW8NwsVz9SBK2Vqn1N0PIMn5xA6NZV
c7o835DLAFshEWfC7TIe3g==
-----END CERTIFICATE-----

Now you can decode the content of either certificate using the command:

$ openssl x509 -in google_com.crt -noout -text

Here's a slightly edited version of the output:

Certificate:
   Data:
       Version: 3 (0x2)
       Serial Number: 5500042407018834932 (0x4c54136dbff12bf4)
   Signature Algorithm: sha256WithRSAEncryption
       Issuer: C = US, O = Google Trust Services, CN = Google
 ↪Internet Authority G3
       Validity
           Not Before: Oct 30 13:15:05 2018 GMT
           Not After : Jan 22 13:15:00 2019 GMT
       Subject: C = US, ST = California, L = Mountain View,
 ↪O = Google LLC, CN = *.google.com
       Subject Public Key Info:
           Public Key Algorithm: rsaEncryption
               Public-Key: (2048 bit)
               Modulus:
                  00:d1:bf:94:10:1f:94:15:bd:6c:3b:83:97:49:29:
                  ad:08:63:18:11:1b:57:7d:4d:b3:3f:9c:cd:62:ed:
                  eb:4d:d2:6b:78:3f:3f:01:48:43:a8:81:b6:42:f6:
                           <... content omitted ...>
                  e1:e4:24:b8:21:c4:9e:e5:86:c6:73:45:4f:a8:6f:
                  e0:81:f3:4e:46:03:3d:e9:d2:01:5b:6f:57:3c:22:
                  d4:83
               Exponent: 65537 (0x10001)
       X509v3 extensions:
           X509v3 Extended Key Usage:
               TLS Web Server Authentication
           X509v3 Subject Alternative Name:
               DNS:*.google.com, DNS:*.android.com,
 ↪<... content omitted ...>
           X509v3 CRL Distribution Points:
               Full Name:
                 URI:http://crl.pki.goog/GTSGIAG3.crl

   Signature Algorithm: sha256WithRSAEncryption
        c7:57:a4:97:ad:32:e1:5f:10:53:05:ba:03:c4:cd:2e:11:c9:
        7d:36:a9:4c:16:a8:46:a1:5a:30:c4:4f:04:86:8d:8b:e1:95:
        24:34:62:94:48:b9:8a:3d:d2:d7:49:eb:a5:6c:59:72:c3:64:
                <... content omitted ...>
        16:9c:64:a7:59:61:fd:4b:22:de:71:73:3d:c5:3a:15:bb:f1:
        58:c4:9d:a1:96:4a:a6:0c:b9:fb:a1:86:8d:8b:ff:74:4c:84:
        76:75:d8:ce

This looks like progress—you can see the certificate provided by the google.com server in all of its glory! Let's visit the highlights.

Near the top of the certificate, you can see the serial number in the "Data" section. The certificate authority gives each certificate a unique serial number when it is generated. This allows certificates to be identified uniquely if there ever is a need to revoke them.

Two "Signature Algorithm" blocks follow the Data block. The first of these contains the server's public key information along with any extensions (options) enabled by the CA. The second block contains the signed hash generated by the CA.

Looking at the detail within the first "Signature Algorithm" block, you can see the subject, issuer and validity dates for the certificate. In general, the certificate may be used only on a server with a name matching the subject and within the dates specified. The "Issuer" (a CA) identifies the public key used to validate the signature in the second block. If this issuer's public key is in the trusted key store, the certificate is valid.

Perhaps most important, you can see the server's public key in this section. This is the key the client will use to encrypt content so that it can be decrypted only on the server. In this case, the server is providing a 2048-bit RSA key.

Below the server's public key, you'll find a block labeled "X509v3 extensions". The extensions in this section are set by the CA when the certificate is signed and can be used to enable (or restrict) the use of the certificate. The full details of these extensions are defined in RFC 5280, but I'll cover the highlights here.

You can see in the "X509v3 Extended Key Usage" section that the certificate is authorized for "TLS Web Server Authentication". This means that the certificate may be used to identify a web server positively. Other common uses that might be listed here include functioning as a CA (allowing the signing of certificates for other servers) or authorizing the certificate to be used as proof of a client's identity.

The "Subject Alternative Names" section is optional, but it may be used to list any additional web servers permitted to use this certificate. The SAN section allows servers to be listed using either DNS names or IP addresses. As you can see in the example, wild cards are supported. If the certificate's subject does not match the server name, but a match can be made here, the certificate is treated as if the subject were a match.

The final entry to look at in the X509v3 extensions block is the "X509v3 CRL Distribution Points". A CRL is a "certificate revocation list". These lists contain the serial numbers of any certificates issued by the CA that have been compromised, retired or (for some other reason) made invalid.

As a regular step in negotiating a TLS connection, the client should verify that the certificate presented by the server—or the serial numbers of a certificate used as a CA—does not appear on a CRL. Ideally, these serial numbers would be checked against a current version of the domain's CRL each time a certificate is verified. In practice, pulling an updated CRL for each connection adds a lot of overhead to the process. Most clients will use a cache, which is rarely up to date but avoids issues if the site hosting the CRL is unavailable.

The second "Signature Algorithm" block contains the signed hash of the DER-encoded X.509 data found in the first block. You can see that the SHA256 hash algorithm was used, and that an RSA private key was used to sign the hash. After being decoded with the CA's corresponding public key, this hash must match the value computed by the client for the same data.

This is where the use of DER for encoding is important: if there were multiple ways to encode the data in the certificate, as there might be using BER, the hash might assume several different values. By using DER, you guarantee that the values are encoded and decoded consistently into the same series of bytes. If a single byte changes, a different hash would be created and the verification of the signature could not be completed!

Certificates contain a lot of detail, and all of it must line up before a client can trust the certificate. It shouldn't be surprising that things fall apart as often as they do.

The most common errors with TLS connections include:

• Server name mismatch, meaning that the certificate's CN (or any entries in the subject alternative names section) doesn't match the host that presents it.
• A certificate has expired and needs to be re-issued by the CA.
• The root CA used to sign the certificate is not in the client's trusted key store.

With the knowledge you've gained about certificates in this article, the meanings and (perhaps more important) solutions to those problems should become more clear.

Resources

• Introduction to ASN.1
• RFC 5280 (X.509 standard)
• RFC 1421 (Privacy Enhanced Mail)
• OpenSSL man pages relating to x509 manipulation, specifically man x509 or man openssl-x509.
• OpenSSL man pages relating to secure client, specifically man s_client or man openssl-s_client.