Understanding HTTPS and the TLS Handshake

I use HTTPS every day, but for a long time I never really understood what happens behind it. These notes summarise the TLS handshake as I have come to understand it.

The problem with plain HTTP

Plain HTTP is transmitted in clear text, so the data can be read or tampered with at any node along the network path. HTTPS adds a TLS layer between HTTP and TCP to solve three problems: encryption (prevent eavesdropping), integrity (prevent tampering) and authentication (prevent impersonation).

Symmetric vs asymmetric encryption

• Symmetric encryption: the same key encrypts and decrypts. It is fast, but safely sharing the key with the other side is the hard part;
• Asymmetric encryption: a public/private key pair, where only the private key can decrypt what the public key encrypted. It is secure but slow.

TLS cleverly combines the two: it uses asymmetric encryption during the handshake to securely negotiate a symmetric key, and then all subsequent data is encrypted symmetrically — balancing security and performance.

The handshake, roughly

1. The client sends the list of cipher suites it supports (Client Hello);
2. The server picks a cipher suite and sends its certificate to the client (Server Hello);
3. The client verifies the certificate is trusted, then a session symmetric key is negotiated;
4. Once both sides confirm, all further communication is encrypted with that symmetric key.

Certificate chains and CAs

How does the client know the server's certificate is genuine? Through the certificate chain. The server certificate is signed by an intermediate CA, which is in turn signed by a root CA, and the root CA's certificate is pre-installed in the operating system/browser and trusted by default.

Verification walks up from the server certificate, level by level, until it reaches a locally trusted root. If any link does not match, the browser reports "certificate not trusted".

Summary

Once I understood the handshake, Caddy's automatic certificate issuance made much more sense: it is simply completing, on our behalf, the process of proving to a CA that we really control the domain (the ACME protocol), and then obtaining a trusted certificate.