When learning about how to use OpenSSL to create self-signed certs, it became clear to me that most of the information available online assumes you know what certificates are, and how they fit in HTTPS.
I decided to have a go at explaining HTTPS by explaining how communications are made secure, namely how Diffie-Hellman key exchange and digital certificates work. Here's my take on it.
HTTPS allows us to communicate securely over an insecure channel (for example over an open wifi network in an airport) where it is easy for someone to listen to all network traffic. The communications with HTTPS are secure because they are encrypted.
The communication is encrypted using symmetric key encryption. This particular type of encryption requires that both the sender and receiver have the same key (i.e. the same key is used to encrypt and decrypt the data).
But if the sender and the receiver need to send the key to each other through the insecure channel, doesn't this make this whole exercise pointless?
This can only work if the sender and receiver have a way of sending a secret key to each other even with someone listening to the communication (and without that someone being able to figure out what the secret key is).
This is where the Diffie Hellman key exchange comes into play.
Diffie-Hellman
Diffie-Hellman allows two parties to create a shared secret key without revealing it, while doing it "in the open", i.e. even if someone has access to all the information that the two parties exchange, they still won't be able to discover what the secret key is.
It sounds like magic, right? No, it's just math.
Here's a simple example of how it works. First two numbers are picked, they are usually called p and g. p is short for prime, g is short for generator.
A prime is a natural number greater than 1 that has no divisors other than 1 and itself.
A generator g for prime number p is a number such that g^x mod p for x between 1 and p-1 will "generate" all the numbers from 1 to p-1.
mod (short for modulo) is a function that returns the remainder of an exact division, for example 5 mod 3 is 2, because the exact division of 5 by 3 is 1 and the remainder of that is 2 (3 * 1 + 2 = 5).
Now that we know what mod is, here's an example of a generator for a prime. 3 is a generator for 7 because:
- 3^1 mod 7 = 3
- 3^2 mod 7 = 2
- 3^3 mod 7 = 6
- 3^4 mod 7 = 4
- 3^5 mod 7 = 5
- 3^6 mod 7 = 1
If we order the results we get: 1, 2, 3, 4, 5, 6, we can see that they are all the numbers from 1 to p-1, which makes g = 3 a generator for p = 7.
We know have all the pieces to perform Diffie-Hellman "by hand".
As an example, imagine Alice and Bob want to write letters to each other. Unfortunately someone is reading their correspondence, so they decide to use Diffie-Hellman to be able to communicate secretly.
Alice picks g and p, for example g = 3, and p = 7 and sends a letter with them to Bob.
Alice also picks a secret number less than p, for example 2.
Once Bob receives p and g, he picks a secret number less than p, for example 5.
Bob then computes this: g^secretBob mod p, in this case 3^5 mod 7 = 5. Bob sends this number to Alice, let's call this number numberBobSent.
At the same time Alice computes g^secretAlice mod p, in this case 3^2 mod 7 = 2 and sends it to Bob, let's call this number numberAliceSent.
Alice receives Bob's letter with number 5. She then computes numberBobSent^secretAlice mod p, in this case 5^2 mod 7 = 4.
Bob receives Alice's letter and does the same: numberAliceSent^secretBob mod p, in this case 2^5 mod 7 = 4.
Their shared secret is 4. Whoever read the letters could read p, g, numberAliceSent and numberBobSent. However, they don't know neither Alice nor Bob's secret numbers nor their shared secret. Alice and Bob can now use their shared secret to encrypt and decrypt their correspondence.
This works because for large primes it is prohibitively costly to figure out the shared secret with the information that an eavesdropper has access to (p, g, and numberBobSent and numberAliceSent).
Digital Certificates
We now know that web servers and browser clients can make use of Diffie-Hellman to establish a secret key to encrypt communications even when someone has access to all information that is exchanged.
This solves the problem of secure communications over an insecure medium, right? Well, not really.
Going back to Alice and Bob's example, imagine that whoever is reading their correspondence, let's say Eve, takes a more proactive role. Eve will grab Alice's letters and replace them with her own, and will do the same with Bob's.
She will be able to establish a secret key with both Alice and Bob. Alice will think she's corresponding with Bob, and Bob with Alice, while they are both corresponding with Eve.
She then just has to receive Alice's letters, decrypt them with the key she established with Alice, read them, encrypt them with the key she established with Bob and send them to him.
There's no way for Alice and Bob to know that their communication is not private.
This is what is called "A man in the middle attack".
So how can we solve this problem?
That's where digital certificates come into play, but before we need to talk about asymmetric key encryption.
Asymmetric Key Encryption
With asymmetric keys, there are two keys, a private and a public one. Both can encrypt and decrypt, but if you encrypt with one of them you can only decrypt with the other.
The private key is never supposed to be sent, while the public key can be freely sent to anyone. Because anything encrypted by the private key can only be decrypted with the public key, if someone sends you something encrypted and you can decrypt it with a public key, you know that only the holder of the private key could've encrypted what was sent.
A certificate is just a public key bundled together with information about the holder of the private key (e.g. the subject name which usually contains the website domain, e.g. blinkingcaret.com).
What makes a certificate trustworthy is that it is digitally signed by a certificate authority (an example would be Symantec, formerly Verisign). But first, what is a digital signature?
Digital Signature
The goal of a digital signature is to prove the identity of the sender of the information and that the information was not changed along the way.
A digital signature is performed in two steps. First a hash of what is to be signed is created (a hash function transforms an arbitrary amount of data into a fixed amount). Then it is encrypted using a private key. The encrypted hash is bundled together with the information. The encrypted hash is the signature.
When someone receives a piece of information with the encrypted hash they can create a hash of the information themselves (using the same type of hashing function, e.g. md5), use the public key to decrypt the signature and compare the two hashes. If they match it means that the information not only was not changed, it was also created by whomever the private key belongs to.
In case you are wondering why we don't just encrypt all the information instead of using a hash function, that's because the hash is much smaller and still has all the properties we need to guarantee that the information was not tampered with.
Certificate Authority
A certificate authority is an entity that signs certificates. To sign a certificate, a certificate authority needs a private key, and with that a public key, which also comes in a certificate.
The certificate authorities' certificates are special though. They are self-signed, which means they are signed with the private key that pairs with the public key in the certificate. Also, your computer's operating system came installed with a several certificate authorities' certificates.
When someone gives us a certificate, if that certificate was signed by a certificate authority we trust (any of the certificate authorities for which we have a certificate installed in our computer) we trust the certificate (that's when you get the green address bar your browser).
So, coming back to Alice and Bob's example, we should change the story here because the metaphor with the letters breaks a bit.
Imagine that Bob and Alice would deliver the letters to each other by hand.
Also, imagine that Alice and Bob had never met, they don't know how the other one looks like.
Although more complicated, a man in the middle attack is still possible in this scenario. Imagine someone, let's say Joe would make himself pass by Bob and meet Alice, and then someone else, for example Jane, would pass herself by Alice and meet Bob. John and Jane could coordinate between themselves so that even if Alice and Bob tried to use Diffie-Hellman they would end up talking to Jane and John without knowing it.
However, imagine now that Bob and Alice have id cards, issued by a government, and that are extremely difficult to forge.
Alice can ask Bob for his id card. The card has Bob's name and his picture, and is issued by a government that Alice knows is trustworthy, Alice is satisfied that Bob is whom he says he is. Bob can do the same in regards to Alice.
In HTTPS the id card is the certificate and the issuer is the certificate authority. Also, in HTTPS normally only the web server provides the certificate. However, although uncommon authentication of users with certificates is also possible.
I hope this gives you a better idea of how all these pieces fit together to enable HTTPS. Be sure to check my blog for more related content, for example how to create your own certificate authority, install it and create signed certificates using OpenSSL.
Top comments (16)
This is a really nice article. By way of a little history, the specification that defines certificates is X.509, and is part of the series that describes the OSI Directory - which ended up simplified as LDAP. A certificate contains, in effect, a bunch of LDAP attributes - and so it literally can contain Bob's photo via the jpegPhoto attribute. That's not likely to happen, though, because a Certification Authority ought to only sign a certificate when it can verify every attribute within it.
One thing you haven't mentioned is revocation and status checking... But maybe I should write something on that.
Which I've done: Licensed Revoked; Certificate Status Checking in PKIX.
I must say that your article has been an excellent read. Not only you talked about the basics like Diffie Hellman Key exchange but also related those basics with the bigger set of things like digital signatures and certificates.
Overwhelmed to see such a simplified explanation of such a complex topic.
Hello. Thanks for the interesting article! I have a question. Is it just a coincidence in this example that the 'numberAliceSent' is the same as 'secretAlice' and that 'numberBobSent' is the same as 'secretBob'?
Hi Henri,
Thanks :)
They may or may not be the same, it's just a coincidence that they are.
Well explained!!
I am having a doubt on which encryption type is actually used.
What I understand from the order you have written is that Asymmetric encryption (probably RSA) is used only for Certificates and Symmetric encryption (Diffie-Hellman) for all other communication.
Correct me if I'm wrong.
Certificates are used to prove that the website is whom it says it is. The rest of the communication is encrypted using symmetric key encryption.
Diffie-Hellman is not an encryption algorithm though, it's a key exchange algorithm.
Although the article only mentions Diffie-Hellman, there are other options to exchange a secret key.
An awesome article, but I still don't understand about
magic mathstuff. πVery interesting post, thanks !
Hey, really nice article. Can you explain how browser verifies the identity of a certificate if it's signed by a non root CA ?
Thanks for letting me know
Thanks for such a simple explanation of important aspect of web. Learnt new thing.
Great piece!
Just a heads up, I think "it was also created by however the private key belongs to." should be ", it was also created by whomever the private key belongs to."
Thanks for letting me know