Updated, 10/22/2018: Added proposal 1.a, with added HTTP Public Key Pinning
By now everyone should know that HTTPS is secure, and that HTTPS is important. Browsers even declare HTTP websites as "insecure" in this day and age. And yet, when you look at IoT devices, such as your Smart Home gadgets, they are commonly using HTTP. This means that they communicate without encryption.
Disclaimer: I'm not a security expert, so please bear with me if you find information that may appear to be not totally correct or incomplete. I'll be happy to receive your feedback in a comment and update the article!
The problem with IoT (transport-layer) security
IoT devices are often cheap and hence the engineering efforts need to be cheap and fast, too. Often there is just no budget or time to take security into consideration. So often even absolute security basics such as authentication are often not in place. There are more than enough gadgets out there that, when you have access to someone else's LAN, you can take complete control of.
This is not only a problem of no-name products from Asia, but even of devices from high-profile companies, such as the Google Home. There is inofficial documentation of the local Google Home API that the Google Home App uses. According to this documentation - and I hope that by now the API has been revoked and redesigned - at least at some point in time the communication was unencrypted (HTTP) and unauthenticated. Wow.
While not using authentication is something that you can - and must - blame any engineer for, you can not blame anyone for not using HTTPS on their IoT device. Why? Because it is plain impossible*. At least not when seeing HTTPS like it is designed to work. We'll see why this is the case later, but fact is that the mechanisms of secure communication via HTTPS are designed in a way that you can not easily use it for local communication. Ouch.
*) Yes I know, this is an exaggeration, but how else should I have got your attention? :-)
Mahmoud Al-Qudsi has written a nice blog-post on this.
Here's a shocking inline picture that should increase your blood pressure instantly:
Immediate security measures for all IoT developers
Before we deep-dive into HTTPS, there are a few very important security measures that every developer of IoT devices absolutely must follow:
- Use authentication for all your API endpoints.
- Use unique, strong passwords for each device, generated during the manufacturing process.
- Close all ports that are not used by your public IP using a firewall/iptables/netfilter.
- Don't expose your services that are written in whatever language with whatever HTTP server implementation directly - instead use a well-maintained, well-tested HTTP server as your ingress, such as nginx, which is available for ARM, too.
- Allow firmware updates by your users, or even automatically.
Benefits of using HTTPS in IoT
But why should you even consider using HTTPS? Well, first of all, if you suppose that your home network is a sacred place that only you have access to, you can feel safe with HTTP. But if you accept the fact that it will always be possible to gain access to more or less all of your LAN, you will feel safer when you know that even with that intrusion possible, without knowing the proper credentials or authorization your IoT devices are still safe.
One way that someone can have access to your LAN is a very old, very easy attack called "DNS rebinding", which we will talk about later for another reason. DNS rebinding allows an attacker to access vital components such as your router - and routers often have a lot of security flaws. Are you convinced, yet? So how can HTTPS help?
HTTPS prevents communication sniffing, data manipulation and offers verification of your peer. This should make you feel safer, since even when your router is compromised, your bank account login is still safe. Also, HTTPS verifies the identity of the server, so that you can be sure who you are communicating with, and in case of a crime happening, you know who to sue. Yay!
Let's remember these three important promises that HTTPS guarantees, since we will come back to them later:
1) Privacy: No communication sniffing
2) Integrity: No data manipulation
3) Identification: You know who you are talking to (or: whose software you are talking to)
The core problem
The core problem with HTTPS is that a fundamental component of it is the host name. Browsers compare the host name that you entered into the address bar with the host name that the server's certificate has been issued to. If these don't match, the browser will not connect to the server, except when your user wades through hardly visible links to finally add an exception for your device. And it will tell your users that it is highly insecure what he is currently doing along the way. Frightening!
And that's not all: Since November 1st 2015, this domain must not be an IPv4 or IPv6 address and must be a FQDN with a public top level domain. No certificates for 192.168.1.20 or raspberry.local!
In other scenarios, where no such control is available, such as built-in clients in other IoT gadgets like the Homee that can trigger arbitrary https endpoints (often used for IFTTT, but can point to your local device, too), communication will be plain impossible.
So, this bites you, when:
- You want to provide a secure web frontend hosted on your IoT device
- You want to provide a secure, local API on your IoT device
No solution in sight?
So we seem to be shit out of luck. A versatile solution for Transport Layer Security for local HTTPS connections seems to be impossible as of now. HTTPS has been designed with the public internet in mind, and does not contain mechanisms that handle local usage properly. But what options do we have to still secure our IoT communication? Spoiler: None of them work out of the box in all environments.
Solution #1: Ignore invalid hosts in SSL certificates
If you are lucky enough to build all clients yourself - i.e. when you build a mobile or desktop app to connect to your device or your devices connect to each other - you can still establish an HTTPS connection. Nearly all HTTPS client allow you to handle certificate errors in your code. Here's an example for Java. Just please make sure that you only ignore errors regarding the host name, and still let the connection fail when the certificate has other errors, such as when it has expired.
Pro: Works in offline scenarios and not a single byte leaves your local network.
Con: Only works when you control all clients. :-(
Solution #2: Use public domain names for private IPs
This is how Plex is doing it. You create dynamic domain names for all your devices, e.g. with a GUID in the subdomain or anything, and resolve them to your device's private IP. This requires some infrastructure to programmatically add DNS entries and create certificates for those. This is easily possible today with Cloud DNS providers from Google, Azure & Co. That's rather complex, but works perfectly - as long as your user's routers do not do DNS rebinding protection.
If you happen to only have customers that use routers that don't do DNS rebinding protection, or you control the local network infrastructures and internet access to resolve the domain name, or run a local DNS server, you are cool. This is a very viable and maybe the best solution then. If you don't control the network infrastructure, especially the router, the risk is high that DNS rebinding protection becomes more popular and your solution begins to fail for more and more users.
Pro: Except for the domain lookup, all traffic is local and marked safe - even in security sensitive browsers - and works in many scenarios without a hassle.
Con: Works flawlessly in most setups, but is very troublesome to the non-tech-savvy user that happens to use secure routers. :-(
Solution #3: Roll your own crypto! Well, not really
I've actually seen this done in production. You can rely on well-tested and proved, high quality JavaScript cryptography libraries that allow you to implement things like certificate validation (except the hostname, yo!) and encryption for the payload. There's even standardization for such a library under way in the W3C.
You should totally make sure that you have the proper knowledge to tackle this, or let your solution verify by someone that knows this kind of stuff. It's very easy to make mistakes.
Pro: It's pretty much guaranteed to work and can be very safe when you do it right.
Con: Your browser will still tell your users that communication is insecure, albeit all the effort. And you can not communicate with your device using standard clients, such as curl. :-(
Solution #4: Public reverse proxy
Crazy: To gain security that is trusted by all browsers and clients, you have to open your local network to the public internet! To get HTTPS for local devices, you could set up a publicly accessible reverse proxy that forwards incoming traffic to your local device through a secure tunnel. A simple home-grown setup could look like this:
1) Create a unique and not-easily-guessable subdomain for each of your IoT devices, such as 09460e83-1759-4fbd-afed-9ee4adf8b288.iot.example.com
2) Generate a certificate via Let's Encrypt for that domain and deploy it to your IoT device.
3) Set up a ssh connection from your IoT device that tunnels incoming traffic on the HTTPS port to the local web server:
ssh user@09460e83-1759-4fbd-afed-9ee4adf8b288.iot.example.com -R *:443:localhost:443
And zap, there you go, you have HTTPS to your local device.
There are services that make this even easier, such as https://ngrok.com/ and https://webhookrelay.com/.
Pro: Will be displayed as secure in all browsers.
Con: It does not work locally without internet access and exposes your device to the public internet. :-( But at least the access point cannot be found by guessing or scraping your service. Combined with very good passwords, this is not that bad, but it's still much more exposed than it ought to be.
Solution #5: Port forwarding and dynamic DNS
This is basically the same as Solution #4, but even less secure. Your device can connect to a dynamic DNS service (either built by yourself or you use one of the popular ones) and keep up a domain that points to the router's public IP address. Then it gets a certificate from Let's Encrypt for that domain, and via UPnP it can programmatically set up a port forwarding to itself. But heck, your device is now publicly accessible from the internet, and public IP enumeration will make it detectable by hackers.
Pro: Not that complex to setup up
Con: Very insecure. First, you don't want to use port forwarding at all, because it drills a whole in your local network that is otherwise cut off from the public net. Second, this certain setup makes your device publicly accessible by enumerating ISP's IP ranges, and it will be very likely to be targeted by script kiddies and crackers. Don't do this! :-(
Additionally - and I am glad that this is the case by now - most routers don't allow UPnP by default and it must be enabled manually.
How would a proper solution look like?
Well, the problem that we have here is trust. Your browser does not trust your device, because the browser can not verify that the device contains only software from someone trustworthy. On the public web, this is done through the hostname, because domains are registered to an entity that you can track down and stick your foot up their ass if they do something malicious. But how can you verify this with devices, that can be potentially spoofed?
It seems that Apple has a solution for this in it's HomeKit echosystem. The strongest solution seems to be possible with a special Authentication Coprocessor. But as of somewhen in 2017, a software authentication solution has been offered, too. But this solution is by no means open or publicly documented (as is typical for Apple), so I don't have a clue how it works. I bet it is pretty secure, though, and would very much like to know how it works!
Short of knowing Apple's secrets, I'd come up with my own proposals of how a secure system could be designed.
Proposal #1: Simply drop the host name check in HTTPS for private IPs
This would be a no-brainer for browsers, and IMHO not that bad for security. The full certificate is checked, like expiration date, chain of trust. But the contained domain is ignored. Instead, the user is shown the company that the certificate has been issued to, and asked whether she trusts this company. Similar to how desktop applications ask you whether the setup may run as admin/root. The security would be still very high - at least much higher than the current state of the art.
How well does this solution deliver on the three promises of HTTPS?
1) Privacy: The communication is encrypted and can not be sniffed, even when someone took complete control over your local network - except for the device you are connecting to.
2) Integrity: This holds true in the same way
3) Identification: This is not as strong as it is with internet domains. By typing in the domain, or clicking on a link, you already tell the browser beforehand who you trust. When browsing to an IP, you don't make verifyable assumptions about the identity of your peer. That's why I'd consider the one-time popup telling you the identity of whoever controls the device with the IP you entered, which is, IMHO, basically as strong as typing in a domain name.
Proposal #1.a: With added Public Key Pinning
This proposal is my favorite. Browsers act like described in Proposal #1, but with added HTTP Public Key Pinning of the most immediate certificate - it must not be any arbitrary certificate from the certificate chain, but the immediate certificate that verifies the very host that you are just connecting to. The browser will "pin" this certificate to this host - which may be an IP address - and warn the user when it connects to the same host, but receives a different certificate this time. This is how ssh has been doing it for ages. This means that it's considered secure enough for a remote shell that gives you potential root access to a device, so I guess it's secure enough to connect to IoT devices that are only accessible from your local network, via a usually limited API, too.
Proposal #2: Let the manufaturer validate the device's authenticity
This proposal is more sophisticated than #1, but more complex, too. As always, for a bit more security, you'll need exponentially more effort.
I think there is no general method to verify that a device has not been tinkered with. But, for sure, the manufacturer should be the instance that knows best how to check the device for authenticity. So why not let the manufacturer do this? Browsers could implement a mechanism following this scheme:
- Connect to an IoT device using HTTPS
- The browser reads the certificate and validates the usual stuff, like expiration date and chain of trust - but NOT the hostname
- The HTTPS response contains a randomly generated session id and a random token, and the certificate includes an URL to an online validation web service (used later)
- The device is expected to connect to that web service, and send it a random token for the given session id
- The manufacturer can choose to only accept this token after it did some kind of validation of the device's integrity
- Before any more communication is done, let the user verify that the manufacturer from the certificate is the institution that she expected and trusts. This is much like what desktop operating systems do when installing software. Asin Proposal #1, this is the important part that establishes the trust.
- When the user confirms, the browser contacts the validation web service (it must be HTTPS!), send it the session ID and read back the token. This token is compared to the one contained in the HTTPS response from the device
- Only when the tokens match, the connection is considered successful -As a bonus: The device's certificate and the validation service's certificate must be issued to the same company. This would further enhance the trustworthyness.
How well does this solution deliver on the three promises of HTTPS?
1) and 2) are the same as in the first proposal, and as strong as for public domains
3) Is a bit stronger in this proposal, because you couple the trusted, well-tested public domain verification with the local device verification. Plus, the manufacturer can opt to implement additional steps to verify that the device has not been tempered with. This would be pretty strong, IMHO not a bit less secure than public domain verification, but of course it'd be quite a hassle to implement. Plus: It requires an active internet connection for the device and the browser. (But again, the I in IoT stands for "Internet".)
I think, however, that the first proposal is trustworthy enough. You just want to verify that the device you are connecting with is the one you expect to. If this is the case, because you recognize the manufacturer's name, this should be good enough™, at least for my taste. And with the added Public Key Pinning of proposal 1.a, you get quite a solid security setup.
Conclusion
Fact is that local HTTPS connections will only be possible without additional tinkering after the standards have been changed. I think there is really a need that the IETF, browser developers and whoever else is needed, have a look at this issue and provide a solution to IoT manufacturers. It has always been common to offer local web pages as front-ends for IoT devices, and it will become even more popular. Because of the problems discussed in this post, which have become even more proplematic over the last months and years, currently nearly all manufacturers fall back to HTTP, to circumvent all the trouble. This is a real pity and should be changed ASAP!
Top comments (13)
I’ve run into a situation where we had devices sending payloads over HTTPS every 30 seconds over a cellular 3G connection. We wound up disabling HTTPS because the SSL overhead greatly increased payload size and latency. In our case the payload size meant the difference between a $100/month per device data cost vs a $3000/month per device cost. I wasn’t part of the original engineering team that chose HTTP for the transport otherwise I would have investigated messagepack over UDP or something more sensible. The problem isn’t that HTTPS isn’t baked in but that engineers are making poor protocol choices and security is an important part of that decision. In our case the data was ephemeral and had a short usefulness lifetime but your case may differ.
This sounds particularly bad. Was there a lot of dropped packets or did you find any other badness with 3G vs HTTPS?
These devices are installed in vehicles on rural roads so you can assume that we saw every conceivable kind of connectivity issue but bill shock was of the most importance to the client.
Countries have a bunch of satellites that transmit data unencrypted... not like it is all top secret, but TLS is not an option. There are so many different reasons why TLS breaks, and stock TCP can be horribly inefficient for different use cases.
Just a bit curious, since cellular ~3G transport is probably going to get a lot more common in the next few years.
What I learned by watching security videos and going to meetups:
As for the HTTPS overhead maybe the QUIC protocol will improve things.
Thank you for sharing, loved your article.
Hope to see more article on IOT as I am currently getting started with IOT ,also it has awared me about the security flaws in IOT device.
learned
Thanks for the feedback! Great to know that the article was helpful.
Do you mind sharing what kind of IoT project you're planning?
"I'm not a security expert, so please bear with me if you find information that may appear to be not totally correct or incomplete."
Assessing the validity of statements regarding security is fairly simple: assume that the first and all following assertions are false.
Great article. Very helpful. Another aspect for IoT/devices seems to be expiration period. A cert has a maximum life of just over 2 years. Once our device is fielded there no way for us to update the cert and the user won't want to manage that either. Is this a common problem? Are there common solutions?
We will be attacked by an army of fridges soon.
It's well after midnight and this party is still going! Great article. I am facing this issue with a product that is meant to form its own network when no others are available. Trust is the core of the issue and preventing is from using an otherwise beautiful, secure, cutting edge environment at every level.
Is anyone working on this problem? I would like to further explore ways to establish trust that don't depend on a public internet connection. Obviously mass adoption of any solution won't arrive in time for our project but it would be great to see this use case covered for future devs!
Hi Daniel, thanks for the mention :) I am the creator of Webhook Relay.
Recently I explored IoT area as well and thought about ways I could help other engineers/developers/admins. The main problem in running a public reverse proxy I guess is trust, therefore I added a new feature - TLS pass-through without the termination on the public server side.
Currently I only created an addon for Home Assistant (webhookrelay.com/v1/guide/home-aut...) which auto generates locally a DuckDNS certificate and can even do TLS termination on the device if the web server doesn't support it. It makes it quite easy to deploy.
However, ideally the web server that user runs should generate the certificates using Let's Encrypt DNS challenge so they could not trust proxy provider at all :)
Another useful feature would be doing IP whitelisting on the public server side, although it can become less useful if the user has dynamic IP (goes to a cafe and uses a wifi).
Also, when a device just needs to receive webhooks and doesn't need to respond, a safest way would be to use webhook forwarding (webhookrelay.com/v1/guide/webhook-...) where traffic is going only one-way and only to a specific URL path (for example: /v1/webhook). This way attacked would have to know exactly what kind of web server is on the other end + that HTTP handler would have to be exploitable.
This is also the solution described in Datto blog.heckel.xyz/2018/08/05/issuing...