DEV Community: Szilárd Pfeiffer

How Diffie-Hellman Key Exchange can Cause Availability Issues

Szilárd Pfeiffer — Mon, 16 Sep 2024 12:00:00 +0000

The Diffie-Hellman key exchange is a cryptographic protocol that allows parties to establish a shared secret over an insecure channel. The security of this key exchange is based on the difficulty of the Discrete Logarithm Problem (DLP) in a given group, such as the multiplicative group of integers modulo a prime number $p$ .

The Diffie-Hellman key exchange is computationally expensive due to the nature of the mathematical operations it involves, particularly the explained modular exponentiation with very large numbers Even with optimized algorithms like square-and-multiply or Montgomery multiplication, which can lead to significant computational effort. The arithmetic operations involved require high-precision arithmetic to handle large numbers, however, standard processors are not designed to handle such large integers directly, so multiple-precision arithmetic libraries are used,

Exploiting the computationally expensive nature of the Diffie-Hellman key exchange protocol an attacker can perform a denial-of-service (DoS) attack by sending arbitrary numbers as public keys to a target server forcing it to generate its public key, validate the peer's one, and compute the shared secret. All three operations require the computationally complex modular exponentiation which creates an asymmetric resource usage situation, which is the basis for overloading the server's CPU and rendering it unavailable. The attack is carried out with such a methodology called D(HE)at attack (CVE-2002-20001).

Mathematical Background

Basic Concepts

The Diffie-Hellman key exchange is based on the following key mathematical concepts:

Modular Arithmetic: This involves operations on numbers modulo some integer $p$ .
Primitive Root and Generator: For a prime number $p$ , a number $g$ is called a primitive root modulo $p$ (or generator) if every number between 1 and $p - 1$ can be expressed as $gk(modp)g^k \pmod p$ for some integer $k$ . In other words, the powers of $g$ modulo $p$ generate all numbers from 1 to $p - 1$ .
Discrete Logarithm Problem (DLP): Given $g$ , $p$ , and $ga(modp)g^a \pmod p$ , it is computationally hard to determine $a$ . This problem forms the basis of security for the Diffie-Hellman key exchange.

Public Key and Shared Secret Computation

Let's assume two parties -- Alice and Bob -- want to establish a shared secret. The followings explain how it can be
done using the Diffie-Hellman key exchange:

Public Parameters:
- Alice and Bob publicly agree on a large (usually 2048-8192 bit) prime number $p$ and a primitive root (generator) $g$ . These values are not secret and can be known to everyone.
Private Exponents:
- Alice chooses a private key $a$ , which is a random integer between 1 and $p - 1$ .
- Bob chooses a private key $b$ , which is also a random integer between 1 and $p - 1$ .
Compute Public Keys:
- Alice computes her public key $g^a \pmod p$ using her private key $a$
- Bob computes his public key $g^b \pmod p$ using his private key $b$
- Both $A$ and $B$ are shared over the insecure communication channel.
Compute Shared Secret:
- Alice computes the shared secret $B^a \pmod p$ using Bob's public key $B$ .
- Bob computes the shared secret $A^b \pmod p$ using Alice's public key $A$ .
- Both computations result in the same value, $g^{ab} \pmod p$ , which is the shared secret key.

Issues Cause the Performance Degradation

The situation is aggravated by the fact that to maintain security, the size of the prime $p$ and the private keys $a$ and $b$ must be large enough to prevent attacks, such as the discrete logarithm attack. The larger these numbers, the more secure the protocol, but also the more computationally demanding it becomes. In environments with limited computational resources (like IoT devices, embedded systems, or mobile devices), performing modular exponentiation with large numbers can lead to noticeable delays, higher power consumption, and increased resource usage.

Long Exponent (CVE-2022-40735)

The number of operations grows with the size of the private exponent. The time complexity of modular exponentiation is about $O(log⁡e)O(\log e)$ , where $e$ is the exponent. It means that the long exponent issue in the Diffie-Hellman key exchange relates to the length of the private exponent (the secret number chosen by each party). If the exponent is particularly long (e.g., 8192 bits), then performing this exponentiation operation can be computationally expensive and time-consuming, and may cause availability issues (CVE-2022-40735).

Guideline	2048	3072	4096	6144	8192
NIST SP 800-57 Part 1	224	256	292 $∗^*$	348 $∗^*$	394 $∗^*$
RFC 3526 Estimate 1	220	260	300	340	380
RFC 3526 Estimate 2	320	420	480	540	620
RFC 7919	225	275	325	375	400

* estimated value

According to the technical paper of the D(HE)at attack (Table 2) there have been different recommendations for short private exponent sizes for various prime number sizes for decades. The exponent sizes were determined by considering both security and performance, but there were cryptographic libraries, such as OpenSSL, and OpenJDK, that used long exponents (Table 4) considering only security, but not the performance risking damage to availability, being that the larger the exponent the more effective a D(HE)at attack. As the paper states:

"Only expanding the size of the private keys without proportionally increasing the modulus p does not yield stronger security, but causes significant performance degradation."

Unnecessary Public Key Validation (CVE-2024-41996)

Parties can agree on any public parameters (prime number $p$ and generator $g$ ) during the cryptographic handshake. However, certain combinations of these parameters may risk the small subgroup confinement attack. Parties can avoid the attack by validating the peer's public key according to NIST SP 800-56A Rev 3.. The validation requires modular exponentiation ( $\equiv A^{(p-1)/2} \bmod p$ ) whose resource requirement is similar to the public key generation when long exponent was used, as both sizes of the base ( $A$ ) and the exponent ( $(p - 1) /2$ ) is the same as the size of the prime ( $p$ ).

According to the special publication of NIST (Section 5.6.2.3.2) under certain conditions the validation can be skipped, without risking a small subgroup confinement attack. The conditions are met when parties use parameters defined in the related standards (RFC 7919, RFC 8268) as the prime parameters are always safe-primes, which is the case in TLS 1.3, SSH 2.0, and IKEv2. Cryptographic libraries, such as OpenSSL, perform the validation independently from the fact that the cryptographic protocol guarantees the use of safe primes, risking damage to availability (CVE-2024-41996), similarly to the long exponent attack.

"This routine shall only be used with ephemeral FFC public keys generated using the approved safe-prime groups when assurance of the partial validity of such keys is to be obtained ..."

Large key sizes by default

Parties can agree on any public key sizes (prime number $p$ ) during the cryptographic handshake. It means that a malicious client can force the server to use the largest key size enabled by acting as if it supports only the largest enabled one. As larger key sizes require more resources when computing and validating the public key than the smaller ones the default of the enabled Diffie-Hellman parameters in the case of cryptographic libraries becomes particularly important. When larger parameter sizes (e.g., 6144, or 8192 bit) are enabled by default an attack can be more important than only the smaller ones would be enabled as in many cases maintainers use the library default assuming that the defaults were selected with due care.

Security Strength	Symmetric Key Algorithm	FFC (DSA, DH, MQV)	IFC (RSA)	ECC (ECDSA, EdDSA, DH, MQV)
≤ 80	2TDEA	L=1024, N=160	k=1024	f=160–223
112	3TDEA	L=2048, N=224	k=2048	f=224–255
128	AES-128	L=3072, N=256	k=3072	f=256–383
192	AES-192	L=7680, N=384	k=7680	f=384–511
256	AES-256	L=15360, N=512	k=15360	f=512+

According to NIST SP 800-57 Part 1, Rev. 5 256 bits of security would require a 15360-bit key size, which is practically unfeasible for the majority of cryptographic libraries stated by the paper about D(HE)at attack (Table 12). It is also considerable that both 256 and 384 bits of security can be achieved with much better performance using elliptic-curve-based Diffie-Hellman key exchange (Table 14). It means that neither security nor performance considerations can justify enabling algorithms by default which may cause availability issues. However, the majority of the cryptographic libraries -- including OpenSSL -- enable the largest key size (8192-bit) during parameter negotiation (RFC 7919) according to D(HE)at the technical paper (Table 10).

Library	Maximum Prime Modulus Enabled by Default
OpenSSL	8192
GnuTLS	8192
NSS	8192
OpenJDK	8192
OracleJDK	8192
WolfSSL	2048

Summary

There are three operations during the Diffie-Hellman key exchange that require resource-intensive modular exponentiation:

public key generation
public key validation
shared secret calculation

A malicious client can trigger all three operations without any privileges or considerable resource requirement (D(HE)at attack -- CVE-2002-20001) due to the nature of the protocol, even though the effectiveness highly depends on implementation issues.

In the case of public key generation and shared secret calculation the resource requirement increases strongly when a long exponent is used, meaning that the potential of a successful attack also increases (CVE-2022-40735). This implementation issue becomes problematic, when the cryptographic protocol supports the parameter negotiation (e.g., TLS, SSH), meaning that a larger key size can be forced as presented in the technical paper about the D(HE)at attack (Table 7, Figure 11). System administrators can hardly reduce the impact of this implementation issue. The only exception is the Diffie-Hellman parameter file in the case of TLS, where exponent size can be declared.

The resource requirement of the public key validation is necessarily as high as the other two operations as the base, the exponent and the modules (prime) have the same large (2048-8192 bits) size. However, the validation is not needed if an appropriate modulus is used, but implementations may not take this fact into account (CVE-2024-41996), thus increasing the cost of the key exchange. A system administrator cannot mitigate the issue since the validation is done independently from the Diffie-Hellman parameters.

The default setting of the supported Diffie-Hellman parameter sizes is significant in the case of a cryptographic protocol where the parameters are negotiable. In several cases, default values are not overwritten in the application server configuration. Practically it means that the larger a cryptographic library enables the key size by default the more effective an attack can be. This is especially true when the library is affected by the mentioned issues, however, larger key sizes can be disabled in a server configuration.

Bitcoin account hijacking using OSINT techniques

Szilárd Pfeiffer — Thu, 16 Mar 2023 09:46:29 +0000

Researchers at Kudelski Security have managed to break Bitcoin and Ethereum wallets using a novel attack against one of the most popular asymmetric key algorithms of modern cryptography. Although Satoshi Nakamoto’s wallet was not among the 764 wallets they were able to break, it is quite worrying to see that a software issue can make such a modern cryptographic algorithm like ECDSA vulnerable. Analysis of data that comes purely from open sources could reveal a practical weakness of an algorithm that is the fundamental basis of secure internet communication, public key infrastructures, and cryptocurrency transactions.

Software Issue Behind the Scenes

As with many other times in the history of attacks against cryptographic algorithms, the cause of a successful attack is not a vulnerability in the algorithm itself, but the fact that poor-quality software used an algorithm with insufficient care. It is crucial to obtain “high-quality” random numbers during the calculation of many cryptographic operations. Such operations store passwords using salted hashes, the generation of cryptographic keys used to authenticate web servers on the internet, or an employee before access is given to the company’s private network using a virtual private network (VPN) service. The generation of digital signatures – which is necessary for the verification of the transactions – also requires a cryptographically strong random value. In the absence of strong random values, there would be a relation between the random and the private part of the signing key. This property could be exploited, and the private key can be acquired by an attacker. In the possession of the private key, any transaction related to the key can be verified, meaning that the balance of the wallet can be transferred.

Not a New Idea, but a New Method

Though the attack itself is novel, the idea behind the attack is not so new. The lack of high-quality random numbers has caused serious vulnerabilities in the past. In 2002, a researcher found an issue in the version of the OpenSSL distributed with Debian Linux and its derivatives that reduced the entropy of generated random values. The issue caused SSH, VPN, and X.509 keys generated by the affected versions of OpenSSL to become vulnerable. Services still used a key generated by the affected version of the code which meant they remained vulnerable until the regeneration of the key. In 2010, a group called _fail0verflow _compromised an ECDSA key Sony used to sign software for the PlayStation 3 game console. The root cause of the attack was almost the same as it is now, namely Sony did not use different random numbers when digitally signing different software. The novelty of the current attack is that it exploits the high-degree relationships among the random values (nonces) used to generate digital signatures. It means if the nonce was not generated by a cryptographically secure random number generator but using a weak pseudo-random number generator (PRNG), it would allow the attacker to retrieve the private part of the key used to generate the signatures.

Special Kind of Open-source Intelligence

Three prerequisites must be met to successfully retrieve the signer’s private key from signatures. The first one is that the random value used to generate the signature should come from a weak PRNG. The second one is to have a batch of consecutive digital signatures generated by using the same PRNG. The third one is that the signatures can be ordered, meaning that we know which signatures have been generated after which one. It could be assumed that there are implementations that use weak PRNGs as it is not basic knowledge whether random number generators are adequate for cryptographic purposes and some legacy software may follow older guidelines. The question is: where can an attacker find a consecutive and ordered list of signatures? The answer is: in several places. Digital signatures are part of each cryptographic protocol, so they can be collected from open sources. The best examples are cryptocurrencies, where the transfers are validated by digital signatures. These digital signatures must be published to allow each party to validate the transactions. Basically, cryptocurrency blockchains are huge collections of the necessary signatures. For instance, researchers could collect 763 million unique signatures from the Bitcoin blockchain which were generated by 424 million unique public keys. Most of the time a key was used to generate only a few signatures, but millions of keys generated at least four signatures, which is the minimum requirement of the attack.

Someone Got Ahead of the Researchers

Researchers ran their attack for two days and 19 hours with an estimated cost of USD 265, resulting in 762 unique broken wallets, but someone may have gotten ahead of them, as all the wallets had zero balance. The researchers suspect that these wallets have already been hacked in the past. If they had not been hacked, 484 BTC could have been stolen from these wallets, which means almost 12 million USD. However, 484 BTC was worth 31 million USD at Bitcoin’s peak. The question arises, where did the money go? The researchers obtained that the recipients were addressed by the latest transactions of the broken wallets. They identified 466 different recipient addresses, where the top 1 received 75 BTC, and the top 5 received 140 BTC, meaning more than USD 1.5 million, and almost USD 3 million, respectively. They counted 144 BTC in total, which is far from the theoretical 484 BTC, but it still sounds like a profitable business, worth the aforementioned USD 265 cost of the investigation. After the first transaction to the top address in 2018, several transactions were initiated from that address to several recipients for 0.5 or 1 BTC, although the account still had a balance of 63.5 BTC. Researchers also found public conversations about accounts that were swept exploiting repeated nonces. A forum member called johoe claimed that he had collected 135 addresses that can be compromised using that technique, and 82 had been compromised already at the time of his post. He also stated that he collected 7 BTC from the broken accounts. He was willing to send the funds back to the owner after proving ownership. The researchers repeated their attack against Ethereum after collecting more than 1.7 billion ECDSA signatures. They managed to break 2 unique wallets processing 22% of their signature collection. They decided to stop the attack, considering that the cost-benefit ratio was too low.

Is This Still a Real Issue Now?

Considering only the results related to the Bitcoin wallets, I would say this might not be an issue anymore. The exploitable signatures were generated several years ago, perhaps with the same software that had a serious flaw, which may have already been fixed. The signatures were exploitable not because they used a pseudo-random generator during signature generation, but due to repeated nonce values. At the same time, given that digital signatures are used in so many cases, such as during a cryptographic handshake, this may still be a significant issue. Exploiting the vulnerability indeed requires getting consecutive signatures from a potentially affected server, which is not a trivial problem in the case of a busy server, as many other clients connect to a server between our consecutive connections. Even so, if an attacker manages to exploit a server, the server certificate is compromised without any sign on the server. For as much as the certificate revocation is one of the weakest points of X.509, it is troubling that such a simple flow in an implementation can cause certificate compromise. The case would be even more worrying if there were a similar flow in certificate issuance, as this would compromise a CA, which could result in unpredictable consequences.

Originally published at https://www.balasys.eu.
Licensed under a Creative Commons Attribution-ShareAlike 4.0 International (CC-BY-SA 4.0) License.

Photo by Jievani Weerasinghe on Unsplash

No Zero Trust Network without Strong Authentication

Szilárd Pfeiffer — Sat, 12 Nov 2022 17:06:27 +0000

While there is no one definition for Zero Trust architecture, the major tenets of the approach, as described by CISA, make clear that strong authentication is at its foundation. And strong authentication needs to be a priority for all security architecture, as password issues are responsible for more than 80 percent of data breaches. And strong passwords alone are not enough. Organizations today require more robust protections to defend themselves against the latest threats. Though more and more companies are investing in various strong/multifactor authentication methods, it has proven easier to purchase a solution than to effectively implement and introduce it across an organization. So how should organizations start to actually follow through in adopting strong authentication practices?

What is Zero Trust and why is it important?

Traditional cybersecurity approaches focus on establishing a sound perimeter to keep malicious actors outside a network. This assumes that all users and resources inside the perimeter are trustworthy. In today’s world, however, resources are increasingly hybrid in their structure, and data are spread across an innumerable combination of devices, services, applications, and people. This makes security more complicated than simply keeping bad actors outside a network; good security means knowing more about who’s inside as well. According to the Zero Trust Architecture’s principle guideline – never trust, always verify – no resource should be accessed until successful authentication and authorization have been achieved.

While the Zero Trust security Model was first introduced in the 1990’s, it’s only become widely known and used in the past few years because, like many things, it is far easier said than done. But one thing is for sure: the improved security environment is worth the effort, and it begins with strong authentication. The Zero Trust security model is an approach to the design and implementation of information technology systems. The main concept behind the architecture is de-perimeterization, which refers to the removal of a boundary between an organization and the outside world.

The Seven Tenets of Zero Trust

The United States’ Cybersecurity and Infrastructure Security Agency (CISA) has outlined the following principles in its Zero Trust Maturity Model:

1. All data sources and computing services are considered resources.

Even if the network is composed of multiple classes of devices, this tenant does not allow exceptions. In practice, there should be at least one policy enforcement point in the network where all the traffic goes through and where the policy can be enforced.

2. All communication is secured regardless of network location.

Secure communication and providing trust are no longer based on the location of an asset.

3. Access to individual enterprise resources is granted on a per-session basis.

An evaluation should be made before giving access to a resource independently of whether the resource was previously permitted or not.

4. Access to resources is determined by dynamic policy.

The policies should be generated as real time as possible and should always be appropriate to the attributes.

5. The enterprise monitors and measures the integrity and security posture of all owned and associated assets.

No asset should inherently be trusted, and data integrity should be maintained at all times.

6. All resource authentications and authorizations are dynamic and strictly enforced before access is allowed.

Authentication and authorization should always be rigorously checked at each access request before access is granted to a resource.

7. The enterprise collects as much information as possible about the current state of assets, network infrastructure, and communications, and uses this information to improve its security posture.

Maintaining and improving security posture is a never-ending circle: collecting and analyzing data are essential to the process.

Always Encrypt and Always Encrypt Well

The ‘zeroth’ step in implementing strong authentication is reliable encryption. Zero Trust requires that enterprises never consider their private network as an implicit zone of trust. Assets on a network should always be handled as if an attacker was persistent on the system. As a result, access to resources should be granted in the most secure manner available. This entails not just authenticating all connections, but also encrypting all the traffic. Attackers are smart, and capable enough to eavesdrop on a network, analyze any unencrypted traffic to capture credentials to perform a well-structured attack later.

In the case of a private network or service, it is good practice to allow only the most secure cryptographic algorithms that provides forward secrecy, such as ECDHE and authenticated encryption, such as Poly1305. Attacker toolchains – such as other software systems – often contain legacy parts that do not support the most modern mechanisms. This means that an exploit used against a service that can be accessed solely via encrypted channels, using the most modern algorithms, will fail before it can really begin, even if the attacker can access the system and has valid credentials, as the initiation of the encrypted connection will fail. This approach also has other indirect benefits. Using state-of-the-art encryption increases security as connection failure events can be reported as suspicious behaviors as part of “real-time monitoring,” another requirement of Zero Trust. Beyond this, advanced encryption also helps us identify our own legacy or “shadow” systems that cannot support the most modern forms of encryption, meaning they should either be accessed through a middleware device or be sunset altogether.

Authentication is about Identity, not Methods

Just like there is no reliable authentication without encryption, there is no authorization without reliable authentication. Zero Trust requires strictly enforced authentication and authorization before resource access. Unfortunately, security experts sometimes focus on methods instead of identity. Passwords, certificates, tokens, or biometrics are just methods, they are not the identity itself.

For instance, spyware can collect login credentials by harvesting from the databases of pawned sites, especially when users use their company email addresses for private purposes. Meanwhile, simpler passwords can be brute-forced to be discovered. If security is compromised, the password can no longer prove identity. You might think that a more modern and sophisticated method – such as a digital certificate – could solve the issue of simple passwords, but at the end of the day the strength of the certificates depends on the strength of the password that protects the certificate’s private key. If an attacker can compromise a protecting password, the authentication based on the certificate is also compromised.

Multi-factor authentication (MFA, or 2FA) has not become so popular for nothing, and security experts today encourage both individuals and companies to use at least a second factor in authentication. However, MFA also has weak points in addition to its significant benefits. A typical time-based, one-time password (TOTP) either expires after a minute or is used as part of successful authentication. This means that even if an attacker acquires the actual value, it cannot be used for subsequent authentications. Unfortunately, this does not protect against website spoofing, which can prompt a user to provide both their password and the TOTP on the spoofed authentication form. Using the intercepted credentials, the attacker can then authenticate in the name of the victim.

A more convenient, but less secure, second factor in MFA is “push-based” authentication where, following a successful password-based authentication, the user confirms the login by clicking “Yes, it was me” in a push notification received on their mobile device. However, if an attacker can compromise the password, several login attempts can be made that cause several push notifications on the mobile device of the victim. Just one confirmation in error that occurs either accidentally or through getting tired of the spoofing (MFA fatigue attack) is enough for the attacker to get into the system. It is also important to remember that a mobile-based second factor cannot be any more secure than the protection of the mobile device itself. If the mobile device isn’t locked, we risk the sense of the second factor, and this is the case if confirmation can be done without unlocking, or the locking pattern is simple, predictable, or vulnerable to smudge attack.

To make a long story short: MFA is an essential best practice, but no matter how perfect a solution may seem, security awareness is still essential.

Authentication is Necessary, but not Sufficient

Zero Trust Network Architecture requires both encryption and authentication, but these are means, not an end in and of themselves. According to the National Institute of Standards and Technology (NIST) in a special publication on Zero Trust architecture, both authentication and authorization should be strictly enforced every time before access is granted to any resources inside or outside of the private network, in line with the principle of least privilege. This means that the classification of a resource should also vary the conditions of the resource access. Depending on the observable state of client identity, the requesting assets, or other behavioral and environmental attributes, various levels of access can be granted for a resource.

The principle of least privilege (PoLP), also known as the principle of minimal privilege (PoMP) or the principle of least authority (PoLA), requires that in a particular abstraction layer of a computing environment, every module (such as a process, a user, or a program, depending on the subject) must be able to access only the information and resources that are necessary for its legitimate purpose.

For instance, under certain circumstances, strictly “read-only” access may be granted to a particular resource, but upon further authentication “read-write” access can be provided as well. The situation reflects approaches to physical security, where entering physical environments with higher classification requires additional authentication. In terms of network and data security, access to data should mirror physical security considerations and authentication requirements for access to a location. In this dynamic, authentication is not just a single step – pass through the doors and you’re in – but rather a repeatedly executed task of the policy enforcement process based on what is being accessed and how.

The first tenet of Zero Trust says that all computing services are considered resources. Authentication services should be considered as resources, meaning that access should be granted taking the least privilege principle into account. It means that the number of authentication requests, including successful and unsuccessful, should not exceed a certain number. Rate limiting unsuccessful authentication requests helps to prevent brute-force attacks against the first factor of the authentication, such as a password. However, this does not diminish the importance of choosing properly secure passwords. Rate limiting successful authentication requests helps to avoid the compromise of the second factor after the first factor has already been compromised. For instance, an MFA fatigue attack is hardly feasible if the number of successful authentication requests is limited. Zero Trust requires monitoring and measuring the security posture of all owned and associated assets, meaning that exceeding the rate limit should be monitored and could trigger the deactivation of a user account that is suspected to be compromised. This is how identity handling becomes dynamic, something which is required at the optimal level of maturity in the Zero Trust Maturity Model.

Human Factor is Unavoidable in Authentication

While trying to achieve the higher and higher levels of Zero Trust maturity, we should not forget about the weakest link in all security system chains: humans. Security is often contrary to comfort, yet discomfort is usually also contrary to security. Even the trendiest, most cutting-edge , methods can have weak points that attackers can successfully exploit while the most traditional methods might work effectively under such circumstances. Not surprisingly, users tend to bypass security systems if the discomfort they experience exceeds a certain level. Bothering users with frequent password changes is a particularly good example of this. Users who are forced to frequently change their passwords often fall into two kinds of bad habits for security when trying to remember their passwords. The first is writing it down where others can see it, and therefore steal it. The second is that making memorable but predictable alterations to their existing password to fulfill the password policy requirements.

A conventional authentication method such as a password can be secure, but its security level does not depend on the expiry period. – rather, it depends on the entropy. The easier it is for a user to remember a strong password, the likelier it is they will use it. Security is about minimizing risks that cannot be taken without cooperation. High security requirements are good, but followable security rules that are in line with the risk are the best.

Originally published at https://www.balasys.eu.
Licensed under a Creative Commons Attribution-ShareAlike 4.0 International (CC-BY-SA 4.0) License.
Photo by regularguy.eth on Unsplash

The internet is a global village, not a metropolis

Szilárd Pfeiffer — Wed, 11 May 2022 12:59:54 +0000

The internet: where everybody is your neighbor

The internet is not as big as you might think. Until IPv6 arrives, there are fewer than 4 billion IPv4 addresses. In theory, it would be almost 4.3 billion addresses, but 600 million IPv4 addresses are reserved. In practice, there are approximately 3.7 billion public and routable IPv4 addresses. Finding vulnerable machines among this crowd of IP addresses might seem like looking for a needle in a haystack, but this is not true. With automated tools, anybody can systematically scan the internet for various vulnerabilities in public services.

This kind of mass scan requires fewer resources than you might think. Assuming that checking for a vulnerability takes a maximum of one second, in one month hackers need to check around 1,400 machines a second on average to find every single device on the internet that is exploitable for a particular vulnerability. If the available period is just a week or a day, you need to scan around 6,000 or 43,000 services a second, respectively. Sounds like relatively high numbers, but if you consider that even a huge country like China uses less than 8% and Russia uses less than 1% of the available IP addresses, you can see that focusing on smaller targets decreases the required resources to a tenth or a hundredth.

The numbers above demonstrate that it is theoretically possible to mass scan the entire internet. Effective free-to-use tools can be used to perform mass scans, converting this academic opportunity into practice. ZMap, Masscan, and others promise to scan the entire public IPv4 address space in some ten minutes or so on a typical desktop computer with a gigabit Ethernet connection. Researchers proved that the time needed to perform application-layer scans in some cases could be further reduced, meaning that mass scans will be able to discover any accidentally or willingly published application layer services in a short space of time.

Search engines for potential weaknesses

Hackers do not even have to perform mass scans themselves, as commercial services sell bulk data from their mass scans. It would still be challenging to inspect, cleanse, transform, and model the bulk data to discover necessary information, though companies such as Shodan or ZoomEye have already done the data analysis task for you. They also provide search engines to access their well-structured databases, which contain near real-time information about exploitable services worldwide. These databases can only be accessed for free with substantial limitations, but of course, you can pay a subscription fee to reduce these limitations. The entry-level subscription cost is just a few dollars per month, meaning that the information price for the exploitable services is not high. Together with the numerous free-of-charge tools, this creates dramatically low barriers to entry for self-appointed hackers.

Some can put the mentioned tools and services together and create a well-automated system, especially since they provide APIs to access their database. As you can see, you do not have to be a nation-state actor to perform effective scans for vulnerable services on the internet. Criminal and hacktivist groups can also do it by using the mentioned services or creating and managing their systems – this is common practice. However, the situation is actually worse than that, as even a script kiddie could also find vulnerable services in their interest. The aforementioned search engines make it possible to filter the services by protocol, vendor, vulnerability, geolocation, etc. Someone with a low level of preparedness could cause harm to an organization by identifying targets with the mentioned tools using proof-of-concept implementations of the vulnerabilities. This can be true even if the author of the proof-of-concept implementation discloses it responsibly.

Operation technology is also under attack

Cyberattacks may use devices that are not strictly part of the IT infrastructure to achieve their goals. For instance, with Shodan, you can easily find webcams without any authentication or default username and password near or inside a targeted organization. A hacker can use unauthorized access to a webcam to observe the targeted site and create plans to circumvent the guards. Some hacking techniques depend on getting devices into the targeted organization or near it. For instance, installing an open Wi-Fi hotspot near the targeted network may cause devices of the targeted network to connect automatically to the malicious Wi-Fi hotspot. If this happens, a hacker can eavesdrop on any unencrypted traffic sent or received by the connected device to get usernames and passwords. Even if the data is encrypted, metadata can still be collected, such as the domain name of the visited sites. It also opens the possibility of an intrusion attempt that exploits vulnerabilities on the connected device. An open webcam increases the risk that someone might install a malicious device unnoticed, even if it is a guarded factory site far from overcrowded districts.

Some people might think that no part of the critical infrastructures or Industrial Control Systems (ICS) are ever connected directly to the internet without robust authentication. The reality is different. For instance, a necessary Shodan query result contains thousands of Schneider Electric devices, mainly from Spain, France, and the United States. The manufacturer provides digital solutions for the energy and automation sector. You can also find thousands of devices by searching for network protocols (e.g., Modbus, DNP, Fieldbus, PROFINET) used in SCADA or Industrial Control Systems. It is possible that published programmable logic controllers (PLC) do not lead to the most severe risk, as you can also find human-machine interfaces (HMI) published on the internet. These devices are usually accessible by the Remote Desktop Protocol (RDP) servers, which can have both configuration and implementation issues. For instance, they may use NTLM authentication that has weaknesses and vulnerabilities.

Zero Trust to the rescue

Under such circumstances, neither industry nor other areas should assume that cyberattacks still don't target them. Today, an organization does not have to be targeted by attackers directly. Automated tools systematically search the internet for vulnerable public services and attempt to exploit them immediately. If they are successful, the tools begin a lateral movement to spread themselves through the organization as extensively as possible and wait for the attackers' commands. At that point, we have already lost. The best advice to prevent such a situation is what the Zero Trust Security Model has been advising for decades and is now also followed by US governmental offices in line with President Biden's executive order:

Handle everything equally as a resource, independently of whether it is part of information technology (IT) or operational technology (OT), as both have the same importance.
Allow access to resources only after strict authentication independently from the resource accessible from the internet, as an intranet service can still be a target of an insider attack or malicious software brought to the intranet by a personal device (BYOD).
Prohibit plain text communication and use only encrypted connections with robust encryption algorithms to avoid the first step of each attack, eavesdropping.
Apply the least privilege principle during the authorization to minimize the risk that an infected device might cause on your network, as it is able to access everything that the device user is permitted to access.
Apply these controls in a session-based manner to minimize the period between the revocation and the enforcement of an authorization level revocation.
Continuously monitor your devices and network to prevent or notify of any suspicious behavior.

Licensed under a Creative Commons Attribution-ShareAlike 4.0 International (CC-BY-SA 4.0) License.
Photo by Robynne Hu on Unsplash

Cryptography Weakening: A Tale of the Law-abiding Criminal

Szilárd Pfeiffer — Fri, 04 Mar 2022 17:57:13 +0000

The EU Council says backdoor and security are compatible. The idea is fundamentally flawed.

It is not the first and probably not the last time that the right to privacy and the fight against crime and terrorism face each other in the world political arena. Various means of encryption play an essential role in both. A non-public draft of the Council of the European Union was published by a law enforcement organization called Statewatch. Based on the document's content, the European Union is preparing to adopt regulations that would allow law enforcement agencies access to material encrypted by criminal groups. The United States, during the Trump era, also represented this direction, and the new presidential administration composition does not foresee any change in this regard. The goal is undeniably positive. Despite big technology and advocacy organizations are protesting. But why? The answer is simple: you can not have your cake and eat it.

The point of secrecy and confidentiality is the same in virtual and real life. What only one person knows is a secret. What more than one person knows is not a secret. Encryption algorithms grant confidentiality and privacy in the virtual world. If someone can access data despite being encrypted due to encryption weakening or backdoor installation, secrets are not secrets anymore. Access to secrets can have compelling public interest. For instance, the fight against terrorism, cybercrime, or the spread of child pornography. However, we must ask the following questions: to what extent can weakening encryption serve the purposes of the fight against terrorism and other criminals? Should we pay, and if so, what price to achieve that goal? Are these two goals proportional to each other? Is privacy or terrorism threatened more by weakening the encryption? Do we not already have the methods to deal effectively with the problem on the technological or legal front? Before answering these questions, it must be stated that the European Union's draft, which has now been revealed, emphasizes the importance of robust encryption methods. Although, it is difficult to put this draft into practice without weakening the encryption. The reason is the fact that end-to-end encryption software such as Signal, recommended by Elon Musk, or Telegram, provides messages unencrypted at only the two endpoints while data is transmitted encrypted over the internet.

The methodological part is unclear in the EU draft. It is not an accident, even if the Council is currently only drafting the directive that can explain it. However, there could be two directions that are certainly conceivable. Let's see these methods of cryptography weakening and the practical consequences they would be.

One possible solution to weakening the cryptography is to make the algorithm fragile for only notable organizations, such as the police or secret services. Any actors who know the weak point of the encryption algorithm can break it. Of course, the extent of weakening should be as small as that exploitation requires significant resources available only to a state body. However, several state bodies and huge companies may have enough resources to break the encryption, so the secrets of a citizen are vulnerable to many foreign countries and large firms.

Another possible solution is to use a kind of master key that carries a much higher risk than the previous one. In this case, anybody could decrypt the encrypted data even with modest resources. The situation is very similar to that in the story of Achilles, one of the best-known figures in ancient Greek mythology. He was invulnerable in all of his body, except for one heel. Anyone who knows the weak point of the warrior can overcome it. In this case, anyone who knows the key can break the encryption. If these keys were compromised, there is no need to possess nation-state-level technical or financial conditions for decryption. The consequences are unthinkable. Using a backdoor technique would cause the same problem. If an unauthorized party knows how to use the backdoor, it can easily decrypt the encrypted content. An essential difference from Achilles' story is that we know the weak point exists, meaning we can prepare for it.

Whichever method we choose, we will certainly put a severe target in the hands of criminal groups, terrorists, or even foreign secret services. It would be their primary interest to obtain information on breaking encryption. It is just a question of political conviction on which countries' services we envision fighting with each other to gain access to the data of European Union citizens.

The quest to find encryption algorithms' weaknesses exists independently from the current EU legislative idea. The history of cryptography is also the history of searching for the Achilles heel of the algorithms. Whoever was ever interested in the secrets of others searched for the weak points of their encryption. After finding the weak point tried to keep them secret, for he could only abuse them. Disclosure of the secrets and the intention to keep secrets can guide the search for weaknesses of the encryption methods. Drawing attention to potential problems makes the quality of encryption constant over time.

The Law-abiding Criminal Problem

The weakening of encryption algorithms, even if it ever materializes, does not mean that previous methods, which public bodies cannot decipher, will disappear from the world. Both the math toolkit and the software used to stay with us. Legislators can outlaw the encryption algorithms, which no one can decipher. They can punish the usage of these algorithms, but it is doubtful that this will affect a law-abiding citizen, who can be held accountable through state bodies. It would be absolute naivety to think that members of a group preparing to commit a terrorist act can be deterred by legislation that exposes them using encryption that cannot be deciphered by bodies seeking them.

Encryption tools that are free for anyone and their infrastructure can cause extremely low entry thresholds for a wide variety of criminal groups into the world of unbreakable encryption. It would not be wise to underestimate the preparedness of either individual criminal groups or intelligence services in this area. Raising the entry threshold by prohibiting the robust encryption algorithms would undoubtedly exclude citizens who want to protect their privacy at an earlier stage from using practical cryptographic tools. The source code of a significant part of the software that implements encryption is freely available, so they do not need to be re-developed in the case of a possible ban. Only the existing ones should be improved, which undoubtedly requires some expertise, but it is far from impossible for slightly more prepared organizations and criminal groups.

Of course, it is also conceivable that the use of these applications will be restricted or prohibited by law and that ISPs are required to filter out this type of traffic. However, the feasibility of this is highly doubtful. We would have already stopped it if it were technically easy to prevent the spread of viruses, spam, and any illegal content on computer networks. As a counterexample, many cite the Chinese Golden Shield Project (or Great Firewall), which can prevent citizens from accessing content that the state deems undesirable. And here comes the catch-22: the firewall can be bypassed, but there are only a few who dare to risk the severe consequences. Due to the few attempts, the bypass methods are relatively easy to recognize, so the only way to reserve the right to use robust encryption algorithms is to use them in mass, regardless of whether an individual needs them.

In Europe, encryption is unlikely to be banned in general. Specific algorithms will not be allowed at most, particularly those that the Union's institutions cannot decrypt, so they will be filtered out of internet traffic. It should be noted that this is a much more complex and resource-intensive task than what China is doing. Of course, the proper regulatory framework can marginalize encrypted communications, especially if the confidence in their effectiveness is shaken. And then, we return to China because if there is little encrypted traffic, all its participants can be observed.

Certainty of Uncertainty

Suppose there is a process by which someone could decrypt your encrypted materials (documents, photos, videos, conversations, health, or other personal information). In that case, you will face grave uncertainties or, at least, serious risks. The most obvious question would be who will have the right, and under what conditions, to access data that you have believed and intend to be private. The fight against crime needs fast reaction times, meaning that a decision on the declassification and decommissioning of a secret must be made quickly. It naturally increases the probability of mistakes, so it can be guaranteed that there will be cases where people's private data is disclosed, even to a limited extent, which should not have been. What about this information? How long will it be kept, where, and under what circumstances?

The possibility of intentionality must also be taken into account. How would more straightforward for an officer to be compromised with much broader access to private data than before? Would a citizen be informed about access to the data, or would the access be hidden from that citizen? For what reason and for how long would it be possible to obtain such data? Many difficult-to-answer questions, even if we think in the context of a well-functioning rule of law. Even though the questions are hard to answer, there is no reason not to look for answers, these are just the doubts, but there are also serious risks. In the case of a poorly functioning the rule of law or a state organization or society burdened with corruption, we must reckon with the damage caused by mistakes and possible intentional abuses. However, one case or another is not necessarily serious, but the losses in the entire society can be significant. It must be kept in mind that the method can fall into the wrong hands, outside the Union and within it, and politicians can utilize it. The interception of Angela Merkel may have been in the interest of the NSA and particular groups within the Union.

It is also an open question of how such legislation can and should be applied to prioritize people, companies, and other organizations. Suppose these measures do not cover private bodies and companies equally. In that case, that may mean that we leave a loophole for criminal groups, and citizens could justifiably argue the point of why companies, organizations, or groups of people have an advantage over others in this regard. Of course, there may be ideologies for this and even practical reasons. We can say that trade secrets have an advantage over private secrets because the former can have a much more significant
impact. A company can be seen as a concentration of economic power. On the one hand, revealing its secrets substantially impacts the community more than a particular citizen. On the other hand, there can also be a significant concentration of power in the case of a politician, a business leader, or a media owner. Giving up equality before the law to favor or disadvantage individual groups can quickly become a political issue, generate social contradictions and mistrust of the measure, and discredit the goal.

Risk on Back of Risk

It is assumed that the cryptographic algorithms should be intentionally weakened from those mentioned above. However, the proposal in question states the importance of robust cryptographic algorithms. The Union, its bodies, and any other parties with the appropriate methodology information could exploit the weakening. Access to these methodologies must be verified in the strictest possible way for understandable reasons. It must also be mentioned that the intention to acquire them would be undoubtedly firm by criminals and agencies, as they could obtain the private information of EU citizens.

Suppose the European Union takes such an action without consulting its economic and political competitors, and they do not take similar steps at almost the same time. In that case, the EU is going to be at a disadvantage. The Union immediately becomes a target. It makes its citizens a target as the Union would be the only place where encryption algorithms can be broken in possession of the appropriate information. This possibility will undoubtedly encourage the enemies of the Union, including terrorist organizations, to obtain this information. So while conceived in the spirit of fighting terrorism, it would give potential latitude to terrorism.

It is challenging to store the method descriptions and keys needed to break the encryption (master keys, backdoors, or other technologies). It is also a difficult task to control their access. Accessing the decryption mechanism from the internet is necessary. Without it, the system would be very cumbersome to use, and long turnaround times jeopardize the original purpose. It is almost irrelevant whether or not a central repository is available to EU members or whether the members would operate the repositories per state. The repository or dozens of repositories would be exposed to cyber-attacks. EU members would certainly have the right and the possibility to share the information obtained with parties who may be considered a friend of the EU in the international political arena. It is uncertain that a good relationship between the EU and a foreign country means a friendly relationship between an EU citizen and the same foreign country.

Limitations of Feasibility

The naive legislative idea that it is enough to ban something by the force of law and result that it will disappear immediately faces serious challenges at this point. Neither the weakening of encryption nor the installation of backdoors is feasible without the support of the largest technology companies, but the European Union has no power to force them.

The number of the software used for different encryption reaches the order of one hundred. There are both free and commercial, open and closed source products. It seems to be very difficult for organizations outside the jurisdiction of the European Union to put pressure on developers to open backdoors to EU offices or support weakened cryptographic algorithms. Open source software developers do not seem so influential as their original goal is the opposite of weakening cryptography. In addition to all this, most developers are individuals - even EU citizens - so the projects are international. Therefore, the EU has only a negligible influence on the operation of those projects. The software projects that provide robust, reliable, and irreversible encryption will not disappear just because the EU may ban their use in certain circumstances.

It is not the first attempt to deliberately weaken encryption algorithms. Almost 30 years ago, the U.S. placed export restrictions on robust cryptographic algorithms. As a result, supporting weakened algorithms has become mandatory for U.S. agencies whose credibility was questioned in the middle of the last decade. In 2015, researchers identified two vulnerabilities (FREAK, Logjam, in which, among others, export restricted encryption algorithms were affected. The logjam attack has primarily impacted U.S. government offices. This act struck back twenty years after the U.S. government placed export restrictions on robust encryption algorithms. In the end, the vulnerabilities affected just those who introduced the restrictions, as everyone has long ceased using these algorithms because of their known weaknesses.

What the EU can achieve at best is banning non-cooperative solutions. But as we know, every law is worth as much as it can be obeyed and enforced. Even if the EU could agree with the major operating system vendors and distributors to permit EU citizens to download software containing weakened encryption implementations only from the app stores, it would not be a comprehensive solution. There are alternative app stores, such as F-Droid, completely vendor-independent Android distributions, such as Lineage OS, and several desktop Linux distributions that would still provide free and open source applications with robust encryption algorithms. The EU has no power to restrict these vendors legally, so there would be little choice but to put the provisions into practice. Recognizing and blocking the traffic generated by software using robust encryption led us towards the Great European Firewall, which, with some irony, could be named after the Chinese one to Aegis, the shield of Zeus.

Those Who Have no Secrets ...

It used to be said, for example, in totalitarian regimes, that those who have nothing to hide from the state do not have to worry. It must be stated that privacy is not about secrets but the right to keep your private life confidential. If nothing is illegal or objectionable in what and how you go about your life, it does not necessarily mean that you want to share details with anybody. This is privacy.

It is a question of who and how actual constraints impact us as per planned action by the EU. Does terrorism, in connection with similar plans, come up from time to time? Hardly. Or perhaps the criminal groups who have disregarded the law so far – do they often enter public discourse? Doubtful. As discussed above, there are many solutions to circumvent such legislation. For instance, despite authorities making every effort to target Tor and the Dark Web, they are still part of the internet today. Although GDPR has forced the law-abiding part of the IT world to make a serious effort, it has not achieved the expected positive impact.

Weakening encryption or incorporating a backdoor is nothing more than a nonsense attempt to solve serious social problems. It's like selling blunt knives only in household stores because people can harm themselves or others with sharp knives. Caution is essential, but in this case, it may dissimulate incredible naivety or, even worse, fanciful naivety. Reducing the effectiveness of encryption tools causes obvious disadvantages to law-abiding citizens, while it has no significant effect on criminal groups and their intentions. It is essential to emphasize that the widespread use of robust encryption is the only way to save ourselves as it raises the cost of mass monitoring unaffordable high.

Licensed under a CC-BY-SA 4.0 License.
Photo by Jason Dent on Unsplash

How would Zero Trust prevent a Log4Shell attack?

Szilárd Pfeiffer — Fri, 11 Feb 2022 12:02:19 +0000

There is a seemingly trivial solution to any remote code execution attacks, namely: not to let the inbound traffic match the pattern that triggers the vulnerability of the server. Easy to say, but hard to do. There are almost endless variations of traffic patterns that could trigger the critical severity Log4j vulnerability. As a result, malicious patterns of inbound traffic are extremely hard to detect. At the same time, outbound traffic caused by a Log4Shell attack site is easily detectable. Because this kind of traffic is always considered suspicious, as it targets random internet sites with a special protocol, according to the Zero Trust model it should not be allowed.

Outbound traffic to a random internet site is a peculiarity of Log4Shell attacks that other arbitrary code execution attacks do not necessarily have. This peculiarity should help the prevention of exploitation not only when network defense is based on the perimeter-less Zero Trust Network Architecture security model but also if it is a perimeter-based castle-and-moat defense. The consequence of a Log4Shell attack is that the exploited server tries to download code from a random internet site owned by the attacker. If the download is successful, the server runs the code, usually to grant a backdoor to the attacker. Some questions arise. How could the access of random internet sites be successful? And even if it succeeded, how was the attacker able to access the opened backdoor?

Zero Trust renders Log4Shell harmless

Unlike Zero Trust, which requires micro segmentation, traditionally demilitarized zones (DMZ) are the best practice to isolate external-facing servers of the organization from both the local area network (LAN) and larger, usually untrusted networks such as the internet. Access to services running on the servers of DMZ, except public services such as web services, should be forbidden or strictly authenticated from the direction of the internet. As there are only a few legitimate reasons why a server in a DMZ would want to access the internet, this kind of traffic should also be strictly controlled. Even if a Log4Shell vulnerability is exploited in the server, it cannot download and later run any malicious code, as the outgoing traffic from the DMZ to the internet would have been prohibited. A strictly defended DMZ could prevent a Log4Shell attack, as exploited servers mostly use protocols (LDAP, JNDI) to download the malicious codes, but it is hard to imagine legitimate reasons to permit the use of these protocols to access anything, or at least a random site on the internet, especially from a DMZ.

Zero Trust specifies and generalizes the methods and approaches already applied in any DMZ. Zero Trust requires the least privilege principle, for instance. Servers could access only the strictly necessary resources – e.g., an update server – on the internet with a specific address, port, and protocol. This way of working does not lead to serious administrative costs for the security team, as there are only a few reasons why a server needs to access the internet, and the firewall rules created as a result of these reasons are rarely changed. If for whatever reason, access to trusted resources cannot be tightened, protocol enforcement and content filtering become priorities. The least privilege also means that a resource can only be accessed with the necessary protocol. Even if an attacker runs a server on a port that is assumed to be allowed from a DMZ, such as HTTP(S), downloading the malicious code will fail when Log4j uses another protocol (LDAP, JNDI) than the expected one (HTTPS). Even if the necessary protocol is used, content filtering can detect malicious codes.

Zero Trust defends against RCE attacks

A remote code execution (RCE) attack does not necessarily require an external server on the internet from where the malicious code is acquired. In these cases, the code is injected into a legitimate but malformed request that exploits the vulnerability. It is almost impossible to detect a pattern that exploits a zero-day vulnerability, so we should focus on the consequences of running an arbitrary code on the exploited server. If the goal is simply to bring the service down immediately after exploitation, the attacker will succeed, as there are several ways to do so only on the privilege level of the exploited service. However, privilege escalation very frequently follows an RCE, which means that a suitable patch management process is essential to minimize this threat.

It is likely that one of the next steps will be lateral movement, i.e., trying to infect other servers of the organization. This requires the conversion of the exploited server to a command-and-control (C&C) machine that requires permanent access. If we follow the principles of Zero Trust, only the public service of a server in the DMZ can be accessed from the internet. Accessing any other services on the server is usually either not permitted or only after strong authentication. If this is indeed the case, the C&C server cannot be accessed by the attacker. Even if the public service itself can be used to control the server, eavesdropping and lateral movement cannot be performed if the server is isolated by micro segmentation in line with Zero Trust principles. Data leakage is still a serious threat if the attack is targeted and the exploited service can be used to leak data, but by applying Zero Trust principles this issue can be localized.

Think global, not local: One-by-one solution is ineffective

Log4j developers have implemented a built-in solution to control the outgoing LDAP and JNDI connections. This could be a solution for those affected by these attacks in the short term, but it cannot save time in the long term, as a large amount of software have or will have similar issues even if we do not know it today. These kinds of security issues are the result of the way the software industry now works. The rapid development cycles, the release pressure, the missing security mindset, the low test coverage, the lack of security testing, among others, can all be the cause of security issues. Security experts, such as penetration testers, hackers, and crackers, think differently than developers. Developers mostly focus on the happy paths, whereas security experts focus on edge and corner cases and how to exploit them. However, the goals of a penetration tester and a cracker are entirely different.

Though controlling outgoing connections from a server generally is a good idea, implementing it separately in all applications is presumably neither the most effective nor the fastest way. Different implementations of functionality like this may have other issues that increase complexity, especially if the goal is a comprehensive solution. Even if the developers of various software projects implement similar controls, it would require significant work. Maintainers then need to update a huge amount of software, which is a risk, and simply not possible in many cases, especially in the case of large organizations and IT infrastructures. According to Zero Trust Network Architecture, there should be a Policy Enforcement Point (PEP), which is responsible for enabling or terminating connections between a subject and an enterprise resource. Using a PEP, any outgoing connection from DMZ to the internet can be controlled carefully if they are not disabled.

Always strictly control your resources

The solution, at least until an era of security by design arrives, is to strictly control resources, where everything is considered as a resource, even if it is on the internet. To avoid the consequences of vulnerabilities such as Log4Shell, we must control how resources can be accessed, how they can access other resources, and how these access rules can be enforced on the network using the least privilege principle and other principles of Zero Trust.

Originally published at https://www.balasys.eu.
Licensed under a Creative Commons Attribution-ShareAlike 4.0 International (CC-BY-SA 4.0) License.
Photo by Jasper Garratt on Unsplash

CryptoLyzer: A comprehensive cryptographic settings analyzer

Szilárd Pfeiffer — Wed, 19 Jan 2022 07:18:57 +0000

CryptoLyzer is a multiprotocol cryptographic settings analyzer with SSL/TLS, SSH, and HTTP header analysis ability. The main purpose of the tool is to tell you what kind of cryptographic related settings are enabled on a client or server.

If you are not interested in the principles behind the project, but the practice, feel free to skip the next section and jump to the Practice section.

Principles

Rationale

There are many notable open-source projects (SSLyze, CipherScan, testssl.sh, tls-scan, …) and several SaaS solutions (CryptCheck, CypherCraft, Hardenize, ImmuniWeb, Mozilla Observatory, SSL Labs, …) to do a security setting analysis, especially when we are talking about TLS, which is the most common and popular cryptographic protocol. However, most of these tools heavily depend on one or more versions of one or more cryptographic protocol libraries, like GnuTLS, OpenSSL, or wolfSSL. But why is this such a problem?

The minor problem is the dependency easily stucks them in SSL/TLS/DTLS as other cryptographic protocols (eg: IPSec VPN, Kerberos, OpenVPN, SSH, …) can not be implemented directly by these libraries. Supporting them by the analyzer application needs extra effort. Anyway, most of the analysis of the cryptographic setting does not require any cryptography because before the parties could agree on the cryptographic algorithms they use plain text.

The major problem is the fact that analysis should test special and corner cases of the protocol that are intentionally triggered. It is hard to do that with a cryptographic protocol library, which was designed for production not for penetration testing or settings analysis. During an analysis, connections are tried to be established with hardly supported, experimental, obsoleted, or even deprecated mechanisms or algorithms to identify which ones are supported by the given client or server implementation. These mechanisms and algorithms may or may not be supported by the latest or any version of any cryptographic protocol implementations.

That is why most of the existing tools require special build(s) of the dependent library where all the protocol versions and algorithms of the good old days are reenabled to make the chance to set up these libraries to offer them as clients or as servers. But what if we want to test an algorithm or a mechanism that has never been implemented by the dependent cryptographic library?

It is not just a theory. A special fork of OpenSSL, maintained by Pluralsight author Peter Mosmans, aims to have as many ciphers as possible. This fork is used and recommended by Mozilla Cipherscan, however, it can offer less than two hundred cipher suites, but there are more than three hundred in the different RFCs according to Cipher Suite Info. The majority of them are weak or insecure, which makes it particularly important to be part of the analysis. In addition, it is also true that there are cipher suites that are not on the Cipher Suite Info list, for instance, Russian standard (GOST) cipher suites. These are rarely used cipher suites, but there is an OpenSSL engine that implements them, so they should be checked.

The situation is similar in the case of other parts of the TLS protocol, like extensions. It is unlikely that a cryptographic library will support each extension, but some of them are possibly implemented by some of the libraries. For instance, multiple certificate status request extension and Diffie-Hellman key exchange in TLS 1.3 are not implemented by OpenSSL 1.x but implemented by GnuTLS and WolfSSL. Another example can be the X.509 certificate-based authentication in SSH protocol, which is supported by Tectia SSH and has an open-source implementation but is not supported by the default OpenSSH. Even though they are not so common a comprehensive settings analyzer tool should run checks against them.

Summing up the main reason for establishing a greenfield project instead of contributing to existing one is the opinion, that an analysis is mostly testing when we trigger special and corner cases, hardly supported, experimental, obsoleted, or even deprecated mechanisms and algorithms, so a comprehensive analyzer should be implemented independently from current cryptographic protocol implementations as much as possible.

Goals

CrypytoLyzer focuses on comprehensiveness, multi-protocol ability, and library independence. To do so cryptographic protocols clients are implemented focusing only on the necessary parts (usually the handshake). This kind of client can check a server against rarely or privately used, deprecated, or completely insecure algorithms and mechanisms.

Zero Trust
- should not require any privilege during the analysis
- Should not require any credentials (except it is strictly necessary)
- should not provide any information for 3rd party
Completeness
- should be able to support as many cryptographic protocols (DTLS, IPSec, OpenVPN, TLS, SSL, SSH, …) as possible
- should be able to handle special/corner cases of the cryptographic protocols
Usability
- should work both in public and private networks
- should be able to give a human-readable summary (Markdown)
- should be able to give machine-readable detailed output (JSON)
- should be customizable to meet special needs (Python library)

The existing solutions are focusing on only one cryptographic protocol (TLS or SSH) despite the fact, that all the cryptographic protocols have the same building blocks (authentication, key exchange, symmetric ciphers, integrity), so they can be analyzed in the same (or almost the same) way, only the information which the analysis is based on can be acquired differently.

Features

The goals above are just goals, not the current state of the development, particularly since the actual version number is 0.8.0 at the time of writing. The supported cryptographic protocol families, for now, are the SSL/TLS and SSH. You can read comparative analysis with either open source or proprietary (SaaS) solutions below. I wanted to stay as objective as an author can with the competitors, but forgive me if I have not been managed to.

Strengths

Transport Layer Security (TLS)

Specialties

Cryptolyzer supports both the early and deprecated Secure Socket Layer (SSL) 2.0 and each Transport Layer Security version up to the TLS 1.3 version including draft versions. Some features although be checked hardly or difficultly using the most popular SSL/TLS implementations (eg: GnuTLS, LibreSSL, OpenSSL, wolfSSL, …) or just not in the scope of them. The checks are the specialties of Cryptolyzer such as supporting each cipher suite discussed on ciphersuite.info (and more) including the Russian governmental standard (GOST) cipher suites, Diffie-Hellman parameter checks, JA3 tag generation, …

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
TLS 1.3 Draft Versions	✅	❌	❌	✅	❌	❌	✅
TLS 1.3 DH Parameters	✅	❌	❌	❌	❌	❌	✅
TLS GOST Ciphers	✅	❌	❌	❌	❌	❌	❌
TLS DH Param Prime Check	✅	✅	❌	❌	❌	❌	❌
TLS DHE Param Reuse Check	✅	✅	❌	❌	❌	❌	❌
TLS ECDH Param Reuse Check	0.9.x	✅	❌	❌	❌	❌	❌
SSH DH Param Check	✅	❌	❌	❌	❌	❌	❌
JA3 tag Generation	✅	❌	❌	❌	❌	❌	❌
JA3 tag Decode	✅	❌	❌	❌	❌	❌	❌
	89%	33%	0%	11%	0%	0%	22%

Opportunistic TLS

Several application-layer protocols have an extension to upgrade a plain text connection to an encrypted one (TLS) without using a separate port referred to as opportunistic TLS (STARTTLS). This mechanism is supported by the analyzer in the case of most protocols.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
FTP	✅	❌	❌	✅	❌	✅	✅
IMAP	✅	❌	❌	❌	❌	✅	✅
LDAP	✅	❌	❌	❌	❌	❌	✅
LMTP	✅	❌	❌	❌	❌	✅	❌
MySQL	0.9.x	❌	❌	❌	❌	✅	❌
NNTP	✅	❌	❌	❌	❌	✅	❌
POP3	✅	❌	❌	❌	❌	✅	✅
PostgreSQL	✅	❌	❌	❌	❌	✅	✅
RDP	✅	❌	❌	❌	❌	❌	✅
Sieve	✅	❌	❌	❌	❌	✅	❌
SMTP	✅	❌	✅	✅	❌	✅	✅
XMPP (Jabber)	✅	❌	❌	❌	❌	✅	✅
	92%	0%	8%	17%	0%	83%	67%

Extensions

There are several extensions for the original TLS specifications which have both security and compatibility considerations. A high number of extension types can be analyzed by CryptoLyzer compared to most of the competitors.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
Fallback SCSV	✅	✅	✅	❌	❌	✅	✅
TLS Clock Skew	✅	❌	❌	❌	❌	✅	✅
Compression	✅	✅	✅	❌	❌	❌	✅
Secure Renegotiation	✅	✅	✅	❌	❌	✅	✅
Insecure Renegotiation	0.8.x	✅	✅	❌	❌	✅	✅
Session Caching	✅	✅	❌	❌	❌	✅	✅
Session Ticketing	✅	✅	❌	❌	❌	✅	✅
Elliptic Curves	✅	✅	✅	❌	❌	✅	✅
EC Point Format	✅	❌	✅	❌	❌	❌	❌
App. Layer Proto. Negotiation	✅	✅	❌	❌	❌	✅	❌
Next Protocol Negotiation	✅	✅	❌	❌	❌	✅	❌
Extended Master Secret	✅	❌	❌	❌	❌	✅	❌
Encrypt-then-mac	✅	❌	❌	❌	❌	✅	❌
GREASE check	0.9.x	✅	❌	❌	❌	❌	❌
Handshake Length Intolerance	0.9.x	✅	❌	❌	❌	❌	❌
	80%	73%	40%	0%	0%	73%	53%

Secure Shell (SSH)

In addition to TLS, there are some other cryptographic protocols, such as SSH. The cryptographic protocols have the same building blocks (authentication, key exchange, symmetric ciphers, integrity), so they can be analyzed in the same manner and it can be done with the same tool using CryptoLyzer without compromise.

	Crypto Lyzer	Moz. Obs.	Operous SSH Check	Rebex SSH Check	SSH Conf. Auditor	ssh-audit	ssh-scan
Algorithms	✅	✅	✅	✅	✅	✅	✅
Host Keys	✅	❌	✅	❌	✅	✅	✅
Host Certificates	0.9.x	❌	❌	❌	❌	❌	❌
X.509 Certificates	0.10.x	❌	❌	❌	❌	❌	❌
DH Param Check	✅	❌	❌	❌	❌	❌	❌
Banner Analysis	✅	❌	✅	❌	❌	❌	✅
	67%	17%	50%	17%	33%	33%	50%

HTTP Headers

Security

There are security mechanisms in the application layer too, not only in the transport layer. The HTTP protocol is a good example of that as there are encryption (HSTS), authentication (Expect-CT, Expect-Staple), content integrity (subresource integrity, content security policy) related headers which are parsed by CryptoLyzer in detail.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
Content Security Policy	0.9.x	❌	❌	❌	✅	❌	❌
CSP Report-To	0.9.x	❌	❌	❌	❌	❌	❌
Expect Certificate Transparency	✅	❌	❌	✅	✅	❌	✅
Expect OCSP Staple	✅	❌	❌	❌	❌	❌	❌
HTTP Strict Transport Security	✅	✅	✅	✅	✅	✅	✅
HTTP Public Key Pinning	0.10.x	✅	❌	❌	✅	✅	❌
Referrer Policy	✅	❌	❌	❌	✅	❌	❌
Subresource Integrity	0.8.x	❌	❌	✅	✅	❌	❌
X Content Type Options	✅	❌	❌	✅	✅	❌	❌
X Frame Options	✅	❌	❌	✅	✅	❌	❌
X XSS Protection	0.8.x	❌	❌	✅	✅	❌	❌
	55%	18%	9%	55%	82%	18%	18%

Generic

Several HTTP headers are not closely related to security. Although, some among them, especially caching headers (Age, Cache-Control, Date, ETag, Expires, Last-Modified), can have significance when obsolescence of security-related information (eg: CRL) is indicated by caching headers.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
Content-Type	✅	❌	❌	❌	✅	❌	❌
Server	✅	❌	❌	❌	✅	✅	❌
Application banner	0.9.x	❌	❌	❌	❌	✅	❌
Age	✅	❌	❌	❌	❌	❌	❌
Cache-Control	✅	❌	❌	❌	✅	❌	❌
Date	✅	❌	❌	❌	✅	✅	❌
ETag	✅	❌	❌	❌	✅	❌	❌
Expires	✅	❌	❌	❌	✅	❌	❌
Last-Modified	✅	❌	❌	❌	✅	❌	❌
Pragma	✅	❌	❌	❌	✅	❌	❌
Proxy banner	0.9.x	❌	❌	❌	❌	✅	❌
Set-Cookie	0.10.x	❌	❌	✅	✅	✅	❌
	75%	0%	0	8%	75%	42%	0%

Weaknesses

Transport Layer Security (TLS)

X.509 Extensions

There are some X.509 related mechanisms in TLS that can have a serious impact on security in particular the revocation checking which is a weak point of the public key infrastructure (PKI). Cryptolyzer lagging behind competitors in this field.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
OCSP Stapling	✅	✅	✅	✅	✅	✅	✅
OCSP Must Staple	0.8.x	✅	✅	✅	❌	✅	✅
Certificate Transparency	0.8.x	✅	✅	✅	❌	✅	✅
Multiple Trusted Root CA Stores	0.9.x	✅	✅	✅	✅	❌	✅
Revocation Status	0.10.x	✅	❌	✅	❌	✅	❌
	20%	100%	80%	100%	40%	80%	80%

Generic

Usability is another field where Cryptolyzer should evolve. Comprehensiveness is not enough, weaknesses, vulnerabilities should be highlighted to make the analysis results more easily understandable.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
Client Emulation	❌	✅	❌	❌	✅	✅	❌
Weakness Check	❌	✅	✅	✅	✅	✅	❌
Vulnerability Check	❌	✅	✅	❌	❌	✅	✅
Highlights in Output	❌	✅	✅	✅	✅	✅	❌
	0%	100%	75%	50%	75%	100%	25%

Opportunities

DNS Records

Nowadays information related to security settings is stored in DNS records especially, but not exclusively in the case of the SMTP protocol. These records can be analyzed by CryptoLyzer and the necessary implementations are planned to be done soon.

	Crypto Lyzer	Qualys SSL Labs	Immuni Web	Hard enize	Moz. Obs.	testssl .sh	SSL-yze
DNS CAA	❌	✅	✅	✅	✅	✅	❌
MTA-STS	❌	❌	❌	✅	❌	❌	❌
DANE	❌	❌	❌	✅	❌	❌	❌
SPF	❌	❌	❌	✅	❌	❌	❌
DMARC	❌	❌	❌	✅	❌	❌	❌
TLS-RPT	❌	❌	❌	✅	❌	❌	❌
	0%	17%	17%	100%	17%	17%	0%

Other protocols

Some cryptographic protocols use the variant of the TLS protocol. Some protocols wrap the original TLS protocol such as OpenVPN, QUIC, and there is a modified version of TLS, according to the need of UDP, called DTLS. Their support would be implemented relatively easily in CryptoLyzer in the future.

Practice

Installation

CryptoLyzer can be easily installed from The Python Package Index (PyPi)

$ pip install cryptolyzer

or via Docker from Docker Hub Container Image Library (docker hub)

$ docker pull coroner/cryptolyzer

and there are prebuilt packages for DEB (Debian, Ubuntu, …) and RPM (Fedora, RHEL, SUSE) based systems.

Usage

The command-line interface is slightly similar to the OpenSSL command-line tool. On each level of subcommands, comprehensive help can be given using the --help option. The analysis subject can be given multiple times as an argument in URL format. Most of the time scheme and port part of the URL has default values according to the analyzer (eg: tls, ssh).

$ cryptolyze tls all www.example.com

$ cryptolyze tls1_2 ciphers https://www.example.com:443

$ cryptolyze ssh all www.example.com

$ cryptolyze ssh2 ciphers ssh://www.example.com:22

$ cryptolyze http headers http://www.example.com/path?query#fragment

or can be used via Docker

$ docker run --rm coroner/cryptolyzer tls all www.example.com

$ docker run --rm coroner/cryptolyzer tls1_2 ciphers https://www.example.com:443

$ docker run --rm coroner/cryptolyzer ssh all www.example.com

$ docker run --rm coroner/cryptolyzer ssh2 ciphers ssh://www.example.com:22

$ docker run --rm coroner/cryptolyzer http headers http://www.example.com/path?query#fragment

Licensed under a Creative Commons Attribution-ShareAlike 4.0 International License

Photo by Jon Tyson on Unsplash

Modern Techniques to Prevent Malware instead of Detecting It

Szilárd Pfeiffer — Tue, 18 Jan 2022 13:10:16 +0000

Though we would not argue against the importance of detecting malware, there should also be a cheap and effective step before detection, namely prevention. A malicious email that is never delivered to the recipients will never cause security issues. According to CSO, 94% of malware in 2020 was spread through the email system. Email, which remains an essential tool and still forms the basis of many business processes, uses one of the oldest and least secure communication protocols on the internet. Even though there are several ways to apply security policies on the mail transfer protocol to prevent the delivery of suspicious mail, these methods are not nearly as widely declared and enforced as they should be.

The security of the other commonplace, older protocol, HTTP, the basis for the World Wide Web, has been in focus for more than ten years, leading to significant progress in this area. It is enough to mention the fact that HTTP has been almost completely replaced by HTTPS, an encrypted version of the same protocol that provides confidentiality, integrity and authenticity during data transfer. It should not be forgotten that just a few years ago, several sites sent credentials, customer data, or payment information on the internet without any kind of encryption. Post CCPA, GDPR, and other similar regulations, this is now almost unimaginable. A simple news site without any kind of authentication now uses HTTPS, quite rightly to avoid modification of content during transfer, just for instance. Try to imagine the potential consequences of a man-in-the-middle attack on one of the most popular news sites. As many business processes use the web and the protocol itself could not be improved, the ecosystem as a whole had to be improved to prevent downgrade attacks, clickjacking, cross-site scripting, and other malicious activities. But how secure is the transfer of electronic mail? Email is unquestionably still the basis of many formal and informal business processes, and so as long as this is the case, it is going to be the target of malicious parties. The answer to the issue of email security is a simple one, but not so encouraging. The state of mail transfer security is not nearly as good as in the domain of web page transfers.

There are significant differences between mail transfer and web page transfer. Web page transfer is usually performed between a web client (browser) and a web server. Mail, on the other hand, is not usually transferred to the recipients by a mail client, but instead to the server of the sender’s mail provider. It then passes the mail to the mail server of the recipient’s email provider, and then finally the recipient will download it asynchronously from that server. As you can see, there is a server-server communication that cannot be influenced by either of the clients. The details of that communication are determined by the participant servers and the simple mail transfer protocol (SMTP), which was designed at the beginning of the 1980s to be simple, not secure.

Originally, the protocol did not contain any encryption capability. However, the opportunistic use of TLS was introduced in 1999, though only as an optional extension. Twenty years on, a publicly referenced server still must not be required to use the mechanism to deliver mail locally according to the related RFC. It means that the owner of a recipient domain, notwithstanding a legitimate interest, is not allowed to require encryption during the mail delivery process. It depends on the sender’s goodwill to initiate encryption. Without it, both client and server are exposed to a malicious third party with man-in-the-middle capability that can enforce a lack of encryption, independently from the fact that both parties intend to encrypt. Without initiating encryption, the recipient cannot authenticate the sender and vice versa and beyond this, the mail transfer is vulnerable to eavesdropping and also content modification. Under such circumstances, the recipient may believe a sender is trusted and therefore believe that the attachment is trusted. However, the attachment could potentially be malware forged into the mail by a malicious third party.

Possible Solutions

There are several mechanisms like BIMI, DANE, DKIM, DMARC, DNSSEC, MTA-STS, SPF, TLS-RPT, which have aimed to remedy the situation, with a greater or lesser degree of success. We will examine the theories behind these mechanisms, their practical limitations, configuration difficulties, and the achieved effect.

In general, it can be said that the most important limitations are prevalence and enforceability. As can be seen from the chart, there are significant differences in prevalence if we compare the top 1,000, 100,000, and one million domains of the Majestic Million. The prevalence is much higher among the most popular domains than the others. The reason cannot be only the fact that the Fortune 1,000 companies enjoy much better financial conditions, as these mechanisms can be implemented without significant cost. The lack of knowledge could be one reason, even though there are several guides and easy-to-use tools available to get the necessary configurations to enable these mechanisms. Another reason might be the fact that there is no regulatory body that can enforce the usage of these mechanisms or the execution of the policies declared by these mechanisms, only recommend them, as NIST does. The last reason to mention is that domain owners also have no power to force the execution of the declared policies on the mail systems owners of the recipient side. In any case, big tech companies like Google, which has a significant market share of the mail service market, have begun to put pressure on domain owners who want to provide mail service for their domain to publish policies, although they do not necessarily strictly comply with them.

In summary, the lack of these mechanisms on the sender side means a relatively high risk that emails sent from the domain are going to be put in a spam folder or rejected, at least by the biggest mail providers. The lack of checking of the policies declared through these mechanisms on the recipient side may mean a relatively high risk of incoming mails coming from unauthorized, or even malicious actors who can impersonate trusted parties. Usually, impersonation is the first step of a targeted attack where fraudsters can inject misinformation or bogus content such as scams, viruses, spyware, and malware into the organization.

Sender Policy Framework (SPF)

Concept

The sender side can publish a policy through this mechanism, declaring which servers are allowed to send mail in the name of the given domain. The recipient side can use the published policies to check that the incoming mail is actually sent by an authoritative server. The policy itself is published in a TXT record of the given domain (e.g. example.com) and the servers can be declared by their IP addresses or indirectly, referring to other DNS records. The simplest case, at least in this respect, is that the domain owner maintains its email server and the MX record of the domain contains the IP address(es) of the mail server(s), so only the MX record should be referred to. From other points of view, it may seem easier to use a software-as-a-service solution to implement your mail service. In that case, you can include the policy of the SaaS provider or you can simply redirect to it.

example.com. IN TXT “v=spf1 mx -all”
example.com. IN TXT “v=spf1 include:_spf.google.com -all”
example.com. IN TXT “v=spf1 redirect:spf.protection.outlook.com -all”

Threats

The tricky part of a sender policy framework is not the explicit declaration of the servers, but the declaration of the default action. Default action answers the question of what should be done when the recipient realizes that there is no explicit rule that matches the IP address of the actual sender. For this, a special mechanism (all) can be used optionally with a qualifier. All the examples above use the fail (-) qualifier, so the evaluation result of the Sender Policy Framework will fail for clients not explicitly stated in the policy, meaning that they are not authorized to use the domain.

Although default fail is considered the most secure method, only less than one-third of the SPF records use it. This may be because this is not the most convenient method from the perspective of maintenance, as the lack of an IP address may cause failure in delivery.

There are two other options to avoid this problem. Domain owners can explicitly state that they are not asserting whether the IP address is authorized or not. This can be done by adding the neutral (?) modifier. This might seem convenient in terms of the maintenance side, but has almost no added value on the security side. The result will be the same if the default is missing and the case is similar when the whole SPF record is missing. Counting all the MX records, not only the ones which have SPF TXT records, the ratio of neutral modifiers with a default value (all) is more than one-third. More than half of the SPF records use the soft fail modifier (~) when declaring the default behavior, meaning that the host is probably not authorized. From a security perspective, this is just a little more than if it were neutral.

Why is default action so important? Just imagine a firewall policy that contains a default rule declaring what should happen when no rule matches on the traffic explicitly when it could say that the traffic is not asserting or probably not authorized. It would be a security nightmare. Domain owners should know which servers are authoritative and should declare it strictly. In that case, the recipient can be sure that the sender is authoritative and the mail from a representative of a trusted company really comes from them, and as a result, avoid being a victim of a phishing or social engineering attack. Though strict rules may affect business continuity – something which must be taken into consideration – this issue should be handled by another mechanism, like DMARC, not by making the policy more permissive.

A severe limitation of the Sender Policy Framework is that servers are authorized by their IP address(es), which is vulnerable to IP spoofing, and they could be declared indirectly referring to IP addresses by hostnames or other DNS records which are vulnerable to DNS spoofing. As the policy is stored in a DNS record, it should be transferred in a way that can guarantee the integrity of the data: without DNSSec or at least DNS over TLS/HTTPS, the policy information is vulnerable to tampering and altering. The most serious issue with the Sender Policy Framework is that while its prevalence is relatively high, the really important question is what the ratio of the enforcer servers is. In any case, though the Sender Policy Framework can improve the level of security, publishing a policy has only a modest effect, which means that it should be combined with other mechanisms as it can prove neither the integrity nor the confidentiality of a mail.

DomainKeys Identified Mail (DKIM)

Concept

In comparison, a Sender Policy Framework makes it possible to determine whether the server is authoritative to send an email in the name of the given domain, whereas the DomainKey Identified Mail method can be used to authenticate the mail itself independently from what the IP address of the sender was. The sender inserts a header (DKIM-Signature) into the mail which contains at least the hash of the body (bh), the signature of the content (b), the list of the headers (h) included when the signature is computed, the algorithm (a) used to create the signature and location (d, s) of the public key which can be used to verify the signature.

DKIM-Signature: v=1; a=rsa-sha256; d=example.net; s=brisbane;
     c=relaxed/simple; q=dns/txt; t=1117574938; x=1118006938;
     h=from:to:subject:date:keywords:keywords;
     bh=MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTI=;
     b=dzdVyOfAKCdLXdJOc9G2q8LoXSlEniSbav+yuU4zGeeruD00lszZVoG4ZHRNiYzR

The recipient mail server simply decrypts the signature with the public key found in the given location. The fact of the success of the decryption proves that the mail was signed by an actor who owns the private key related to the public key used to decrypt the signature. The decrypted value is the hash of the original mail that has been computed by the signer module of the sender server. The recipient server recomputes the hash and compares it with the result of the decryption. If the two hashes are identical, the mail body (including the attachments) and the listed headers are guaranteed not to have been altered by a third party. This essentially means that there is no malicious attachment in the mail unless the sender has attached it and nobody has modified any data in the mail, for instance, an account number, committing an email fraud.

Weaknesses

Choosing the appropriate signature algorithm and public key size requires care with this mechanism, as a not-so-wise decision may make the efforts so far meaningless. The traditional RSA public key type, with at least a 2048 bit key size or any elliptic curve-based key with at least a 224 bit key size can meet expectations, according to NIST. As multiple keys can be set, you can configure either an RSA key for compatibility and an ECDSA for modernity and to enjoy the benefits of elliptic curve-based cryptography, like smaller key sizes. Signature algorithm SHA-2 with any message digest size could be suitable. However, SHA-1 is strongly contraindicated as there is a chosen-prefix collision attack against this algorithm. It needs a high amount of computational power, which makes a real-time break almost impossible, even taking into account the average delay time of a mail, though there is an opportunity of a replay attack, where the attacker can send almost the same mail later with a modified bank account number and payment deadline while the signature is still valid. The weak point of the DKIM mechanism is not the cryptography, but the fact that DKIM cannot provide confidentiality.

MTA Strict Transport Security (MTA-STS)

Concept

MTA Strict Transport Security is intended to solve the confidentiality issue of mail sending. Though the SMTP protocol originally did not contain any encryption capability, later an optional encryption capability was added as an extension, which is still not required. The situation is similar in relation to the unencrypted HTTP and the encrypted HTTPS protocols. An attacker with man-in-the-middle capability is able to force a client to use the unencrypted HTTP if the connection to the server was initiated with that protocol independently of whether the server supports the encrypted HTTPS protocol or not. It means that all the traffic communicated between the parties, which may contain sensitive information, can be intercepted by the attacker. There is a condition that must be met for a successful attack to be carried out, namely that the server must support the unencrypted version of the protocol. HTTP is not necessarily present, but SMTP always is, which means the information that encryption can be used must reach the client. This is the Strict Transport Security, which is implemented as a header in case of HTTP and a DNS record publication in case of SMTP.

_mta-sts.example.com.  IN TXT "v=STSv1; id=20210602165800Z;"

During our investigation, we identified that most of the MTA-STS supporting domains use date in the id field. Assuming that the date in the id field is the publication date of the latest policy, and also assuming that the policies are not changing very often, a diagram can be drawn, in a cumulative manner, that shows how many MTA-STS policies were published. Though the curve may suggest a rise, the current prevalence does not even reach one percent.

Unlike the HTTP protocol, the SMTP protocol cannot be redirected to an encrypted channel, as SMTP protocol does not contain something like the “moved permanently” response status code in HTTP, so the information that the server wants to use encrypted communication must reach the client on an independent channel. In this case, DNS is used, just like in Sender Policy Framework or DomainKey Identified Mail. Similar to the earlier mechanisms, MTA Strict Transport Security is published in a TXT record. The difference is that only the fact of presence and current version are indicated by TXT records, but the policy itself is distributed via HTTPS from a ‘well-known’ source. The DNS record contains a short string(id) that uniquely identifies a given instance of a policy so that senders can determine when the policy has been updated.

version: STSv1
mode: enforce
mx: mail1.example.com
mx: mail2.example.com
max_age: 86400

The policy is similar to the Strict Transport Security header in the case of HTTP. It contains a list (mx) of servers by which mails for this domain might be handled, with a lifetime value (max_age) for the policy, meaning that a client can cache a policy for up to this value and a mode (mode) indicating the expected behavior of the sending server in the case of a policy validation failure. The policy may declare that the sending server must not deliver (enforce) the message to a server which fails hostname matching or certificate validation, or has no TLS capability. For testing purposes, there is a mode (testing) that makes it possible to deliver mails despite the fact that there was validation failure, but a sending server can send a report about the failure (see TLS-RPT later). There is also a mode (none) indicating that there is no active policy. Today, just over half of the owners of MTS-STS publishing domains want to enforce the policy, while the other half is presumably in a testing phase. However, it should be noted that the ratio of domains supporting MTA-STS in the top one million is only 0.7%.

Threats

Publishing MTA-STS policy is necessary, but far from sufficient. It should be enforced to minimize the risk of a MITM attack to ensure the unencrypted method of SMTP communication. Nonetheless, the existence and enforcement of the TLS protocol do not necessarily guarantee confidentiality. The quality and performance of the encryption is highly dependent on the details of the TLS settings. They should be determined and reviewed regularly with due care, bearing in mind the requirements of the related compliances (NIST, PCI DSS, HIPAA) if necessary.

Weaknesses

TLS has a significant benefit, namely that it solves not only the problem of confidentiality but also the problem of integrity targeted by DKIM. However, it should be noted that the MTS-STS cannot guarantee confidentiality and integrity throughout the entire process of mail sending, despite Pretty Good Privacy (PGP) or Secure/Multipurpose Internet Mail Extensions (S/MIME) – just in respect of MTA communication, as the name suggests, so in the case of client (MUA) server communication, the same issues exist when a protocol has only opportunist TLS capability, just like IMAP and POP3. Disadvantages of the mechanism include the fact that it requires the maintenance of a web server to publish the MTA-STS policy independently of whether the domain owner intends to publish a web page or not, and also that it requires the maintenance of an X.509 certificate for that web server, as only HTTPS is allowed to download the policy.

SMTP TLS Reporting (TLS-RPT)

Concept

Though the topics discussed so far have been strictly related to security, we should also consider business continuity. Introducing the discussed mechanisms carries a risk in terms of mail delivery. The risk cannot be transferred and should not be accepted, as it can be reduced by the SMTP TLS Report mechanism declaring reporting endpoints where the sending servers can report when experiencing policy validation failures. The policy can be published in one or more TXT records that contain the aggregate report URI (rua), which can either be an email address or an HTTPS endpoint.

_smtp._tls.example.com. IN TXT "v=TLSRPTv1; rua=mailto:reports@example.com"
_smtp._tls.example.com. IN TXT "v=TLSRPTv1; rua=https://reporting.example.com/v1/tlsrpt"

Weaknesses

One advantage of the mechanism is that it makes it possible to receive information about MTA-STS verification failures of sending mail servers, something which can be either a simple technical issue or an active attack. The disadvantage of the mechanism is that it may be necessary for the mail servers to use a protocol (HTTPS) that is unrelated to the mail sending and therefore requires extra functionality in the mail server implementation. Furthermore, the protocol has its own security mechanisms (HSTS, Report-To, Network Error Logging, Expect-CT, …) which should be supported as an HTTP client in a mail server to provide the highest available security level. Policy violation reporting based on mails requires properly configured DomainKey Identified Mail (DKIM) on the report sending mail server side. If DKIM verification fails on the side of the report recipient mail server, the report can be ignored, according to the RFC. However, the RFC does not discuss the fact that the Sender Policy Framework can also be checked, as the report is actually sent by a mail server in the name of a given domain. With these checks, the reporter can be authenticated in a slightly weak way though, as an attacker can buy a domain and can apply proper DKIM and SPF settings and to send misinforming reports. However, with that activity there is a risk that the mail server is going to be added to a blacklist. The other option, sending the report via HTTPS, provides a much better opportunity to the attacker to send misleading reports, as it is much harder for the recipient to authenticate the sender of the report. Maybe this is the reason why the vast majority of the domain owners have chosen reporting MTA-STS violation in mail, not via HTTPS. This means there is a risk that spoof reports will be sent by an attacker to generate false positive alerts, thus undermining the trust in these kinds of reports, which could indicate man-in-the-middle attacks against our mailing system.

Threats

Another problem of the reporting is the fact that it is completely voluntary. Publishing a TLSRPT policy with an appropriate aggregate report URI (rua) value does not necessarily mean that you will actually receive any reports, as the recipient may or may not support the TLSRPT mechanism. Even if the recipient supports the mechanism, it is far from sure that the recipient supports report sending and also intends to send reports, as configuration and a certain amount of resources are required to do so.

Domain-based Message Authentication, Reporting and Conformance (DMARC)

Concept

The relation between the previously discussed TLS-RPT and the MTA-STS is the same as the relations between DMARC and the Sender Policy Framework (SPF) or DomainKey Identified Mail (DKIM). When the verification of SPF or DKIM fails, the recipient mail server can send a failure report to the domain owner defined in the DMARC TXT record. Compared to TLS-RPT it can be noted that there are more sophisticated settings here. Not only a single type of reporting endpoint can be set, but one for aggregate feedback (rua) and another for message-specific failure information (ruf). Report interval (ri) can also be set to indicate a request to mail receivers to generate aggregate reports separated by no more than the requested amount of time, and last but not least a policy is published as part of DMARC applies to the domain and can also apply to subdomains. The domain owner may request mail receivers to take no specific action (none) to treat mail that fails the DMARC check suspicious (quarantine) and place them into a spam folder, for instance, or to reject (reject) email that fails the DMARC check.

_dmarc.example.com.  IN TXT "v=DMARC1; p=reject; rua=mailto:rua@example.com; ruf=mailto:ruf@example.com"

The ratio of the required actions dramatically shows how heavily the business depends on the email systems and how much more important the business continuity is than the security. Using a security first mindset any mail that fails the DMARC check would be treated as suspicious and should be quarantined or rejected. Except for the top 1,000 domain owners, only the minority are brave enough to ask the mail receivers to quarantine or reject. In other cases, the majority asks the mail receivers to take no action which questions the usage of DMARC has any sense.

Weaknesses

Authenticating the issuer of the report is as hard as it is in the case of SMTP TLS Reporting (TLS-RPT), so there is a risk that the reports will be spoofed. Despite the TLS-RPT, DMARC supports only delivering reports by mail, so authentication of the sender is possible, but only in a limited manner.

Threats

Incidentally, a situation is conceivable when there are two servers which support DMARC and intend to send reports when experiencing an issue, but the DMARC fails mutually during the check of the reporting mail due to a DMARC configuration or DMARC check implementation issue. In that case, the DMARC report send is triggered mutually by the DMARC report of the other server, which could potentially cause an endless loop. There are also limits to DMARC specification, like percentage (pct) of messages to which the DMARC policy is to be applied, or the earlier mentioned report interval (ri), but these options are designed to help the introduction of DMARC, not to avoid attacks.

Brand Indicators for Message Identification (BIMI)

The mechanisms discussed above are about machine-to-machine (M2M) communication. There are no – or no well-declared – user-facing consequences of success and failure. BIMI seeks to fill this gap, and makes the result of the earlier mechanisms visible to the user. BIMI enables the display of brand-controlled logos within supporting email clients (MUA) only if the email is well authenticated. This essentially means that the sender domain must publish DMARC for the domain and the subdomains, where the policy must be set to either quarantine or reject and where the percentage subdomain policy cannot be set to anything less than 100. For the brand’s logo to be displayed in the email client, the email must pass DMARC authentication and BIMI validation checks, ensuring that the organization’s domain has not been impersonated and the brand indicator is valid.

default._bimi.example.com.  IN TXT "v=BIMI1; l=https://example/bimi/logo.svg"

Weaknesses

The configuration is relatively simple compared to the previously described mechanisms, but it must be noted that there could also be a relatively simple attack against it. Once the logo is published, a malicious actor could register a lookalike domain, configure DKIM, DMARC and copy the indicator to mimic the attackable domain and publish it as if it were its own. In that case, the same logo would appear in the email client of the recipient as the email coming from either the attacked or the attacker domain, so for the recipient it is just as hard to distinguish the two domains as it was before. The situation is much worse when the attacked domain does not publish BIMI, just the attacker one, so the latter may seem more trustworthy than the former. This weakness is targeted by the Verified Mark Certificates (VMC), which provide digitally signed evidence that the organization is allowed to use the brand indicator, like the extended validation where not only the domain but the organization name and other descriptive data are certified. The mechanism is based on X.509, so it carries all the complexity and difficulties of it, including validation, revocations checking and certificate transparency. Verified Mark Certificates can only be purchased by an organization if their logo is trademarked, so it is feared that the VMC part of BIMI takes us back to pre-Let’s Encrypt times, when there was no chance of signing a certificate with a widely accepted certificate authority for free, meaning it cannot reach high prevalence, just like extended validation.

Domain Name System Security Extensions (DNSSEC)

Concept

DNSSEC is a suite of extension specifications for securing data exchanged using the DNS protocol in untrusted networks, just like the internet. DNSSEC ensures the authenticity and integrity of data, but not confidentiality. As the information stored in DNS records related to the previously discussed mechanisms (SPF, DMARC, DKIM…) are public, confidentiality is not required, but authenticity and integrity are, meaning that we need to be sure that the information is not compromised by an attacker. DNS over TLS (DoT) and DNS over HTTPS also provide authenticity and integrity in addition to confidentiality, but DNSSEC has a significant advantage over them, namely it can ensure authenticity and integrity during the potentially recursive name resolution procedure. DoT and DoH can provide confidentiality, integrity and authenticity between the DNS client and the DNS resolver, but it is not possible to ensure that all DNS servers use DoT/DoH when serving the request and the client has no information about whether they used DoT/DoH or not. It must be noted that these two protocols were never intended to achieve that goal, but instead to provide confidentiality and also privacy to the end-user.

Weaknesses

In the case of DNSSEC, integrity and authenticity are ensured by digital signatures. Answers from DNSSEC-protected zones are digitally signed by a key whose public part can be found in the DNSKEY record of the given domain. Using this public key, a DNS client can check the integrity and authenticity of the answer. Just like any other cases where integrity and authenticity are ensured by digital signatures, the key issue is the chain of trust and also key types, sizes, and digest algorithms used in the chain. Public keys found in DNSKEY records are signed by the DNSKEY of the upper level domain up to the top level (TLD) and the root domain. The parameters of these keys determine the security level. As DNSSEC is a relatively old protocol of the internet, and the earlier versions did not support strong encryption algorithms and large key sizes, so backward compatibility leads to the fact that small key sizes are still in use, however, large ones are also supported.

Only an insignificant minority of the top-level domains currently use the 1024-bit RSA key as the strongest key. Mostly, 2048-bit RSA keys are used, though it must be noted that RSA keys with lesser key sizes are also used. The prevalence of elliptic-curve-based public keys (ECDSA), despite all their advantages, is particularly low among the top-level domains. Only 256-bit keys are used, where the RSA equivalent key size is 3072, neither greater key sizes, nor Edwards-curve Digital Signature Algorithm (EdDSA) curves are used, though the latter is now recommended. Digest algorithms do not follow the same pattern as key types and sizes did: backward compatibility has no significant effect. Only an insignificant minority offer the weekend SHA-1 algorithm. The majority of the top-level domains offer strong digest algorithms.

It should be noted that the prevalence of DNSSEC is unfortunately low despite NIST declaring that organizations should deploy DNSSEC for all DNS name servers and validate DNSSEC responses on all systems that receive emails to provide authentication and integrity protection to the DNS resource records discussed above.

DNS-based Authentication of Named Entities (DANE)

Concept

DANE is a protocol that allows binding X.509 certificates to domain names using the previously discussed DNSSEC. The most important result of binding domain names and X.509 certificates is that it makes any certificate authority (CA) needless, as it can declare an X.509 certificate or public key used on a specific service (for instance an email server) of the domain. In this case, an X.509 public key does not need to be certified by a third party (CA) that the X.509 public-key relates to a specific domain. As part of the issuance of a domain validated certificate, the domain owner must prove its authority to the certification, mostly by creating a DNS record with a specified value. However, in the case of DANE, a TLSA record has already been created to store the public key or its hash, so the authority is proven by matching the public key or X.509 certificate in the DNS record and provided by the service. The mechanism also makes certificate common name and subject alternative name (SAN) matching needless, as the certificate and the domain name are bound by a DNS record related definitely to the domain name. This means there is no need to certify the binding by a certificate authority as it is cryptographically proven by DNSSEC. Certificate revocation check, which is the Achilles heel of the complete Public Key Infrastructure (PKI), also becomes needless, as the validity of a certificate has no sense in this environment: there is no certification by a third party that could expire, and the domain owner simply stops publishing a key if it is suspected to be compromised. The situation is the same when the domain owner just wants to regularly change keys to decrease their validity period, as it would be strongly recommendable, and now blocked by the certificate authorities despite the several weaknesses of revocation check mechanisms.

_25._tcp.mail.example.com. IN TLSA 2 0 1 ( E8B54E0B4BAA815B06D3462D65FBC7C0 CF556ECCF9F5303EBFBB77D022F834C0 )

The previously emphasized way of working does not mean that DANE cannot work together with PKI. TLSA records have four data fields: certificate usage (2), selector (0), matching type (1), and certificate association data respectively. Certificate usage specifies that certificate association data contains a leaf certificate (end entity or EE) or a CA (trusted anchor or TA) and also specifies whether the certificate given by the server should be issued by a CA trusted by the application doing the verification (PKIX) or not (DANE). Certificate usage can take the following four values, where the certificate provided by the server during the TLS handshake must:

be issued by an already trusted CA whose certificate published in the certificate association data (CA constraint or PKIX-TA)
be issued by an already trusted CA whose certificate must match exactly the certificate published in the certificate association data (Service certificate constraint or PKIX-EE)
be issued by the CA published in the certificate association data (Trusted anchor assertion or DANE-TA)
match exactly the certificate published in the certificate association data and it can be a self-signed certificate (Domain issued certificate or DANE-EE)

Selector specifies whether the full certificate (0) or the subject public key info are part of the certificate sent by the server should be matched against the association data, while matching type specifies how the certificate association data is presented. The entire selected data can be presented either in the certificate association data (0) or by a hash (1/2).

DANE would mean a giant leap in the direction of a decentralized internet that preserves the support of the traditional PKI by providing some kind of digital self-determination for the domain owner. At the same time, it would also solve the problem that clients now trust in a predefined group of root CAs, where the members of the group depend on the vendor of the client application, independently from the fact that the domain owner trusts in only one CA that actually issued the certificate used by a server. Supporting DANE constraints could be applied to a number of trusted CAs – including the number zero, meaning that a domain owner could decide to skip CAs entirely.

Weaknesses

Despite all the advantages of DANE, there is one unavoidable disadvantage, namely the lack of client application support, something which makes DANE a theoretical solution only to several serious practical issues.

Conclusions

Given the serious shortcoming of the old fashioned mail transfer protocol, it is strongly recommended to arm email systems with as many additional security mechanisms as possible, according to NIST which recommends organizations to deploy SPF, MTA-STS, DANE, and DNSSEC in order to avoid receiving mail from unauthenticated sources on an unreliable channel. There are, however, overlaps between the discussed mechanisms, all of them are needed to ensure the complete confidentiality, integrity, and authenticity of the received mails and also to be aware of whether senders are experiencing any suspicious behavior. The discussed mechanisms have no significant introduction or maintenance costs, especially given their undoubted benefit. Organizations should declare policies to help each other make informed decisions. These policies are about what servers are authoritative to send emails by the name of their domains (SPF) and whether they want to use encryption (MTA-STS, DANE) to ensure confidentiality when receiving mails, what the evidence is that a received mail has factually been sent by sender (DKIM), and how parties can inform each other when they experience errors or suspicious behaviors (DMARC, TLS-RPT). They also have to prove the integrity and authenticity of these policies (DNSSEC). This means that the “never trust, always verify” principle of the Zero Trust Security Model is essential to filter out any suspicious senders and their deceptive content and malicious attachments, ideally before such content can cause a business email compromise.

Licensed under a Creative Commons Attribution-ShareAlike 4.0 International (CC-BY-SA 4.0) License.
Photo by Markus Winkler on Unsplash

Zero Trust: Is it anything new?

Szilárd Pfeiffer — Thu, 16 Sep 2021 16:19:22 +0000

In theory, it isn’t particularly new. The term zero trust has been around for more than 55 years. De-perimeterisation, the main concept behind Zero Trust Architecture, was defined and promoted on the Jericho Forums, which was founded 20 years ago. Even the management of risks associated with de-perimeterisation were discussed almost two decades ago. John Kindervag coined the concept while he was at Forrester in 2009, and Google implemented a Zero Trust Architecture framework, referred to as BeyondCorp, in the same year. Even so, in practice Zero Trust should mean more, than just a marketing hype, especially given that Joe Biden has ordered that “the Federal Government must adopt security best practices; advance toward Zero Trust Architecture”. The publication of NIST can serve as both a theoretical and practical guideline, which should be applied to achieve worthwhile changes. But what are these theories and practices and why they are so important? Let’s take a look.

What is Zero Trust?

It should be pointed out that Zero Trust is not a product, but a model. Though it can be facilitated by one or more products, it primarily necessitates a change in approach. Before, common sense was that a private network has definite perimeters with a small number of entry points, and the goal was to protect them. This way of thinking bears the strategic approach of the late medieval and early modern period. The defense of an area with definite boundaries and the assets concentrated behind the walls of the fortress. Both attacking and defending armies were mostly focused on the entry point, just like red and blue teams are focused on network defense tools in this castle-and-moat (network) security model. However, it is well-known, as it was in the medieval period, that there is a much easier and more profitable way than a siege, namely sabotage. In the castle-and-moat mode, if the authentication is circumvented at the entry point there are no other mechanisms to prevent malicious activity, as you are inside the perimeter. You are trusted if you are inside the perimeter – this could be the motto of any malware developer. Zero Trust Architecture is looking to overtake this old-fashioned perimeter approach. \
There are significant changes in perimeter approach which makes the rise of Zero Trust quite timely. Before, there was a dogma that it was hard to obtain access to private resources from outside the private network, so successfully authenticated users could access any resources on the private network. In the age of virtual private networks (VPN) and cloud services, private resources can be obtained very easily from the internet, as there are no definite perimeters, with just a small number of entry points. This means that the way we defend the assets of the organization should change. Any access to any resources by any user or machine must be authenticated and authorized independently of whether the resource tried to be accessed from inside or outside of the organization’s private network. Zero Trust means that lack of authentication means no trust is given at all. Access can be given after successful authentication, but in a restricted manner, just like in real life. Network security is no different from other types of security: it uses the same tenets and learns from the history of them all, as discussed above.

Everything is a Resource

Zero Trust Architecture requires us to consider all data sources and computing services as resources, with no exceptions, even if the network might be composed of multiple classes of devices. Practically speaking, this means there should be one or more control points (Policy Enforcement Point) in the network where all the network traffic goes through and where the policy can be enforced. As a result, the castle-and-moat security model is completely inadequate. With Zero Trust, there is no resource concentration, no definite perimeters, and the focus is on the traffic paths of the communication instead of entry points.

As traffic paths can be controlled comprehensively in computer networks, there is no need to control the entry point itself. It is necessary to segment the network as much as possible and separate these segments from each other. This technique is known as micro-segmentation, as it creates several micro-perimeters or segments in the network. As the crossing between these micro segments are controlled and transit is permitted in a restricted manner, accurate authentication and authorization can be performed at the borders. The situation is the same as it is with real life borders, except that there are no – or at least there shouldn’t be –green borders in computer networks. Lateral movement cannot be performed in the network as it is no longer hierarchical, and there are no resources of distinct importance, as all resources are treated equally, meaning access to all resources is verified independently from the classification of the resource, just like any passengers are authenticated at the borders independently of whether the passenger is a particularly important person or not.

Secure Communication

Secure communication is an essential part of the Zero Trust Security Model for several reasons. Secure communication provides confidentiality, integrity, and authenticity. Authenticity makes it possible for the communicating parties to identify each other, and also makes it possible for the Policy Engine to identify the source of communication. The Policy Engine can then make a decision about whether access can be granted to a resource for a given subject, which will be enforced at the Policy Enforcement Point. Confidentiality inhibits the passive attacker to get credentials or other valuable information by eavesdropping on the network, which can be used during an active attack. Integrity ensures that the communication cannot be altered without the knowledge of the communication parties, making it impossible to modify sensitive information, such as a bank account number or invoice amount, in order to add misleading information or fishing for part of the original content.

Session Based Access

According to the Zero Trust tenets, access to the resources are granted in a session-based manner. Both authentication and authorization are session-based, and the users must be granted only the level of access needed to fulfill their role, which means we must follow the least privilege principle. A session-based approach guarantees a time limitation, as the access to a resource is not necessarily granted in a subsequent session or with the same privileges, as privileges should also be limited to those that are strictly necessary, session by session.

Strictly Enforced Authentication and Authorization

As has already been mentioned, the basic concept is that no one is trusted by default from either inside or outside the network. Authentication and authorization are always checked at each access request before access is granted to an organizational resource, though a question arises of how a user can be authenticated. The most-used authentication mechanisms are indirect, meaning they cannot supply direct evidence to the user’s identity, just certain factors such as something the user knows (knowledge), something the user has (possession) or something the user is (inherence), assuming the exclusivity of knowledge, possession, or inherence. Single factor, like a password might be compromised, but the probability of compromising multiple factors with different type is negligibly low, which is why it is so important to use multi-factor authentication.

One fundamental problem of identification by knowledge is that if it is unchanged over a long time, just like a password or a certificate, and becomes compromised, it does not identify the user, yet the abuse is hard to detect. Credentials that change over a short period of time, such as a Time-based One-Time Password (TOTP) are one option, but this solution cannot solve the problem on its own, as an attacker who has stolen the shared secret, which is also a long-term credential, can generate a valid TOTP. However, combined with a possession-based factor, this can help to identify the human itself instead of just their knowledge. This is especially true when accessing the TOTP generator with software or hardware tokens that can be accessed after an inherence-based identification, such as unlocking a mobile device or a security token by fingerprint.

However, for the user, or client in general, identification is just one factor in dynamic polices. The identification process can also encompass any associated attributes assigned by the enterprise to an account. Characteristics of the device used by the client, such as software versions installed, patch level, network or physical location, time and date of request and previously observed behavior, can also be part of the verification of a client and can also determine the applied policy. Behavioral attributes can also be measured, and deviations can be checked against the observed usage patterns before access is granted to a particular resource. The sensitivity and the classification of the resource should also vary according to the conditions of the resource access. For instance, under certain circumstances only read-only access is granted to a particular resource, but after additional authentication, by a second or a third factor, read-write access can be provided. The situation here is the same as it is in physical security, where entering a higher classified place requires additional authentication. In terms of network and data security, higher data acts like a location in physical security.

Monitoring devices in real-time

Establishing a continuous diagnostics and mitigation (CDM) system is also a requirement of Zero Trust Architecture. Knowing the current security-related state of the network and the actors involved is essential, as restrictions should be applied on a client or a server when a security issue can be assumed to be related to them. For instance, if a device runs a service that has a remotely executable vulnerability which is currently unpatched, the access of the affected service should be limited until the service is patched to mitigate the vulnerability. To be able to do that, it is also necessary to have the information that there is a security issue in the organization. This information can come from a CDM system and may imply a change in the earlier mentioned dynamic policies once a security issue is recognized and subsequently fixed.

Appearance of a new device on the network is a typical scenario where monitoring is essential, as rules must be applied to the network traffic of the newly appeared device. Zero Trust requires that we do not trust in a device just because it is inside the private network, so the rule could simply lead to a denial. However, it is also possible that only one path should be opened which makes possible to register the device on the network for the user, especially if it is a mobile or a bring your own device, which can access only a limited part of the network with limited privileges. Independently from the applied policies, the information about the fact that there is an unregistered device that has appeared on the network which tries to communicate is a must, as it could indicate a legitimate usage of the network, but also an illegitimate or at least a suspicious one.

Not just the devices, but the network traffic they generate should also be monitored. As part of the incident management, during investigation we will need all available information. Before any incident occurs, changes in resource accesses may indicate a security issue. For instance, requesting a higher level of privilege during a resource access, like requesting writing permission instead of the ordinary read-only one, or requesting it from an unusual network location, like from a foreign country the organization has no connection with, at an unusual time, like at midnight in case of a colleague that works 9-to-5 , or trying to discover the network, may all indicate the presence of a malicious software that may generate input to the CDM, causing quarantine of the device to prevent the spread of ransomware, for instance.

Conclusion

The NIST does not articulate any requirements of network security in its Zero Trust publications that would not have already been articulated by others before, but it does so in a way that makes it possible to reach not only C-level executives, but also state leaders – as it has influenced the Biden Administration’s plans for strengthening US cybersecurity. Leading technology research firms, such as Gartner and Forrester, also promote the Zero Trust model, which makes the concept almost unavoidable on providers’ side and also generates hype about the topic. Beyond business considerations, we should keep the basic statement of Zero Trust in mind: there could be attackers both inside and outside of the organization, so we should never simply trust, but always verify and enforce the principle of least privilege.

Licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Why Do Certificate Revocation Checking Mechanisms Never Work?

Szilárd Pfeiffer — Thu, 24 Sep 2020 18:45:01 +0000

The certificate revocation system, just like any other cyber security mechanism, is needless, as long as everything is going well, but becomes essential when things go wrong, namely a certificate has been compromised. A compromised certificate makes it possible for a malicious third party to create a service that acts like the legitimate one or insert itself as a man-in-the-middle eavesdropping on any data transferred between the parties without any knowledge of the client or the server. The revocation system is about to give a chance to the client to learn a server certificate is revoked, as it has been compromised. Nothing shows better that we are facing a severe problem than the fact that several thousand certificates are revoked day by day.

Unfortunately, the revocation checking mechanisms have several theoretical and practical difficulties. Although the basis of the mechanisms is revocation, this alone is far from sufficient. There should be a revocation checking mechanism that could work under actual circumstances. Each stakeholder – client application developers, server administrators, middleware solution manufacturers, certificate authorities – have their requirement, and none of the existing revocation check mechanisms can satisfy all the requirements. It must be stated when a certificate gets compromised, we should assume that a malicious third party – who has no authority over the host(s), to which the certificate belongs – could impersonalize any service (web, mail, remote access, etc.) with any domain names the certificate is validated to. The situation should last only as long as the certificate is revoked, but clients have challenges learning the fact of revocation.

Revocation Check Mechanisms

First, take stock of the possibilities in chronological order and summarize how they work and their main issues.

Certificate Revocation List (CRL): The oldest mechanism which is nothing else than the collection of the revoked, but not expired certificates managed and published by certificate authorities. It is useless to keep the expired certificates on the revocation list as the expired certificates are handled as invalid by the clients independently from the fact that they have been revoked or not. As the Let's Encrypt CAA checking issue showed, it can happen that a certificate authority must revoke millions of certificates at the same time. In that case, the CRL size dramatically grows, and all the client applications (e.g., browsers, mail clients, ...) that want to check the revocation state of a certificate must download a large CRL file.
Online Certificate Status Protocol (OCSP): The mechanism aims to solve the size issue that CRL suffers from by making it possible to ask a central server, OCSP responder, about the revocation state. Any client can send a request to an OCSP responder – managed by the certificate authority – whether a certificate issued by that certificate authority is revoked or not. In terms of information theory, revocation state is just a bit of information that is not affected by the number of actually revoked certificates but makes it possible to track the client by its requests.
OCSP Stapling: OCSP Stapling is currently the best method for checking certificate validity. In the case of this mechanism, the server proves that the certificate authority has not revoked the certificate that it will show the client. The server obtains a time-stamped ("stapled") proof from the certificate authority, which has issued the server certificate, about the revocation state and caches it. When a client connects to the server, it offers the cached proof during the TLS handshake. This mechanism transfers the responsibility of the certificate revocation state serving from the certificate authority to the server maintainers. As the mechanism is optional, revocation check state serving depends on the server's capability and configuration.
OCSP Multi-Stapling: The mechanism is almost the same as the OCSP Stapling. The difference is that in part of OCSP Stapling, a server can send only one staple during the TLS handshake, but OCSP Multi-Staple makes it possible to send multiple staples, not only for the server certificate but also for the intermediate ones. Although this mechanism makes it possible to check the whole certificate chain, the most famous cryptographic library (OpenSSL), does not support it.

Problem factors

The evolution of the certificate revocation check mechanisms is not a success story. There are several stakeholders, and they sometimes have slightly different requirements against the certificate revocation system. Although the primary influence factor should be security, the operability challenges sometimes undermine this principle. Revocation check mechanisms cannot fulfill all the requirements of security professionals, software vendors, or user needs at the same time, which causes revocation check mechanisms to fail.

	CRL	OCSP	OCSP Stapling	OCSP Multi-Stapling	OCSP Stapling (TLS 1.3)
Source	✗	✗	✓	✓	✓
Integrity	✗	✗	✓	✓	✓
Access	✗	✗	✓	✓	✓
Size	✗	✓	✓	✓	✓
Privacy	✓	✗	✓	✓	✓
Full Chain	✓	✓	✗	✓	✓
Validity	✓	✗	✗	✗	✗
Penetration	✓	✓	✗	✗	✗

As you can see, there is no silver bullet of certificate revocation check mechanisms. Newer and newer mechanisms have appeared, but they could not solve the issues and only changed pre-existing problems into new ones.

Source

The first issue we have to discuss is how systems can know the source of revocation state information. Without this information, none of the stakeholders, client software (e.g., web, mail, remote access clients), middleware devices (e.g., firewalls) cannot acquire the essential information. The problem itself is independent of the type of the client software. It only depends on the kind of revocation check mechanism.

Certificate Revocation List (CRL): There is an X.509 extension in the issued certificate, called CRL distribution points (CDP), which may contain one or more URIs pointing to the CRL, managed by the issuer certificate authority (CA). The problem is that the CDP extension is non-critical, so not necessarily found in a certificate. However, the vast majority of the certificate authorities fill the extension (filled in 2207 out of 2233 intermediate CA certificates), but not all of them. Middleware devices could use certificate revocation lists very effectively as they have the necessary resources to prefetch the information on which certificates have been revoked and regularly update that. The list of distribution points of all the certificate authorities is a prerequisite, but there is no official list, even though the PKI is a highly centralized system. Mozilla has an intermediate CA certificate collection, which can be used to achieve the CDP locations in the case of the intermediate certificates but cannot be used in case of server certificates. The latter would be more important as the probability of compromise is much higher in a server certificate than an intermediate one. It should also be considered that the number of server certificates is much higher than the number of intermediate certificates. Anyway, CRL fetching could be done on the fly when a connection has been initiated, but it may take a relatively long time. Until the fetch has finished, the client must be awaited, which may cause significant latency for the client.
Online Certificate Status Protocol (OCSP): The working method is almost the same as the CRL. There is an X509 extension in the issuer certificate, called Authority Information Access, which may contain one or more URIs pointing to the OCSP responder of the CA. The problem is that the extension is non-critical, so it is not necessarily found in a certificate. Although the vast majority of the certificate authorities fill the extension (filled in 1994 out of 2233 intermediate CA certificates), there is still a shortage. Middleware devices could use OCSP responders to prefetch, cache, and update the revocation information of the most popular sites. Still, there is a prerequisite – the OCSP responder locations of all the certificate authorities – which is not satisfying lack of an official list of responder URLs, even if the fact that Mozilla's intermediate CA certificate collection can also be used to achieve the OCSP responder locations of the intermediate certificates, but cannot be used in case of server certificates.
OCSP Stapling: The source question does not occur according to the working method as the proof about the revocation status of the server certificate is sent by the server itself, so no extra information (URI) is needed to achieve it.

Integrity

Data integrity is an important question when discussing revocation mechanisms, as the client must be sure that a malicious third party cannot modify the revocation information. The revocation information goes without encryption on the network, but this in itself is not a problem.

Certificate Revocation List (CRL): The most commonly used protocol to achieve the CRL is HTTP, not HTTPS, regardless that HTTP cannot guarantee either the integrity of the CRL or the whole HTTP traffic. The CRL is digitally signed by the certificate authority that manages and publishes the CRL and has earlier issued the certificate that has been revoked later, so a malicious third party cannot forge it without the user's knowledge, assuming that it verifies the signature. Modifying the HTTP traffic during a man-in-the-middle attack could not compromise the CRL directly. Still, it can indirectly disrupt download by changing HTTP-related parameters, forcing clients to use a soft fail mechanism. It must be noted that the hash algorithm of the CRL's digital signature is as important an issue as in the case of certificate signing.
Online Certificate Status Protocol (OCSP): OCSP responders also use the HTTP protocol to serve the responses, just like CRL distribution points. The certificate authority issued the certificate whose revocation state is in question, manages the responder, and digitally signs any response. The integrity of OCSP responses is guaranteed identical to CRL distribution points, and disruption can be done in the same way.
OCSP Stapling: The server provides the OCSP staple during the cleartext part of the TLS handshake. As the certificate authority digitally signs the staple with the certificate used to sign the server certificate, the integrity is implemented in the same way as for CRL distribution points and the OCSP responders. Anyway, the TLS handshake closes with a verification, where the parties calculate the checksum of the handshake received by them from the other party and compare it with the checksum sent by the other party in the encrypted form integrity is double-checked in that way.

Access

Client applications may have difficulties accessing revocation state information, primarily when a firewall or other security system controls the internet and the revocation information comes from another server than the original server that the client wanted to be accessed.

Certificate Revocation List (CRL): Revocation information comes from a different IP address (CDP IP address instead of the server's one), port (80 instead of 443). It uses a different protocol (HTTP instead of HTTPS) than the original server. According to these differences, the firewall policy may not permit the client to access the CDP, even if it permits accessing the server.
Online Certificate Status Protocol (OCSP): The problem is precisely the same as in the case of the CRL distribution point. OCSP responders are also on different IP addresses (OCSP responder IP address instead of the server's one), port (80 instead of 443), and use a different protocol, so the OCSP could not solve the problem that occurred on CRL distribution points.
OCSP Stapling: As the server sends OCSP staple during the TLS handshake, the problem is not applicable, as the client application has already accessed the server. Only the OCSP stapling could fully solve the problem of access.

Size

As the Let's Encrypt CAA checking issue showed, it can happen that millions of certificates must be revoked at the same time, so a revocation check mechanism must be able to handle that case.

Certificate Revocation List (CRL): The CRL size can grow dramatically. All the client applications (e.g., browsers, mail clients) that want to check a certificate's revocation state must download the CRL file. The revocation list is an unsuitable mechanism for client software as the size may extend suddenly. Even small revocation lists would cause significant memory usage on mobile devices because clients should store all the revocation lists of the considerable amount of intermediate certificates in memory. Regular updates of the revocation lists would cause significant traffic charges, which may have financial implications on mobile devices. Major browsers do not use this mechanism at all. Chrome and Firefox have their mechanism respectively called CRLSets and OneCRL to solve the problem. CRL collections maintained by Google and Mozilla are updated regularly in the background without restarting or updating the browser. Several other browsers and other clients (e.g., mailing clients) do not use CRLs at all. For middleware devices, revocation lists could be the optimal solution. They could prefetch, regularly update, and cache the revocation lists, as the vast majority of certificate authorities support the revocation list both in intermediate and root certificates, so they can use revocation lists to check a complete certificate chain. However, there are massive deficiencies. The first one is the fact that the most popular certificate authority does not support CRL for server certificates, so almost 200 million actively used certificates cannot be checked that way. The second is that middleware devices can not prefetch CRLs without an official list of all CRL distribution points. CRL is a comfortable mechanism for certificate authorities as client applications (e.g., browsers) mostly do not use it. Meaning there is no considerable load on CRL distribution points from distributed sources, making it difficult to defend against a DDoS attack.
Online Certificate Status Protocol (OCSP): The mechanism solves the size issue that CRL suffers from by making it possible to ask a central server, OCSP responder, about the revocation state. Any client can send a request to an OCSP responder, managed by the certificate authority, whether a certificate issued by that certificate authority is revoked or not. It is just a bit in terms of information theory, and the amount of information is not affected by the number of actually revoked certificates. OCSP completely solved the size problem for client software as it is small enough to download it even for all new connections. For middleware devices, the size has never been a significant problem. The necessary bandwidth is given in a typical environment where they are used. Nevertheless, OCSP is imperfect as OCSP responses cannot be prefetched and used as a database like a certificate revocation list. However, it can be fetched on-demand and cached for later use for a moderate amount of time. For certificate authorities, it is a comfortable mechanism. In practice, no client uses that mechanism to check the certificate revocation, so it causes minimal load on the OCSP responder.
OCSP Stapling: As OCSP staple has almost the same size as an OCSP response, the size problem has been solved for both client software and middleware devices. It is especially true since the server sen the OCSP staple, so it is just some extra bytes during the TLS handshake. For the certificate authorities, OCSP stapling is a demanding mechanism, if we consider that a certificate authority can have 200 million active certificates, which is the actual case of Let's Encrypt. Only if a third of servers supported OCSP stapling and a validity period of the staple was 5 minutes, it would result in 200.000 requests per second on the OCSP responder, which is apparently causing difficulties.

Privacy

We can think of certificate revocation states as a database, where the fact of a query is sensitive data as if you perform a query for a certificate of a specific website, likely, you are just visiting that site so that you can be tracked in that way. As the overwhelming majority of the web traffic nowadays is encrypted, the overwhelming majority of the traffic can be tracked in that way.

Certificate Revocation List (CRL): A certificate revocation list can be considered a database, so the problem is not applicable, as the whole database can be downloaded and queried locally, not on the certificate authority side.
Online Certificate Status Protocol (OCSP): An OCSP response contains verified information about the revocation state of a certificate that has been issued by a CA managing the OCSP responder, so the OCSP request that the client has sent must contain information that identifies the certificate in question. A certificate is issued for a domain that the client wants to visit, so by an OCSP request, the CA learns what domain the client visited. For a client software, it is a considerable risk as this mechanism makes the user of the client software trackable by a third party whose responsibility would be to increase the privacy level, not to decrease it. For a middleware device, it can be a considerable risk, as the organization using the device can be tracked, but a specific member of the organization is not.
OCSP Stapling: OCSP staples are served by servers, so the problem is not applicable, as the client application has already accessed the server so that it can be tracked by the server anyway.

Full Chain

Certificate revocation checking should not be limited to the server certificate itself. Revocation check of the issuer certificates is also essential. If an intermediate certificate is compromised, any server certificate can be issued to any domain by the malicious third party that compromised the certificate. Revocation check of a root CA certificate is like a serpent eating its tail. They are self-signed, so they verify themselves.

Certificate Revocation List (CRL): As the vast majority of the certificates, both intermediate and leaf ones, contain CRL distribution points, their revocation state can be checked by CRLs so that the complete certificate chain can be checked in practice.
Online Certificate Status Protocol (OCSP): As more than ten percent of the intermediate CA certificates do not contain OCSP responders, their revocation state cannot be checked by OCSP. If we want to be sure, the revocation check must be combined with CRL checking.
OCSP Stapling: As discussed earlier, this mechanism depends on the certificate authority OCSP support. Hence, customers of almost ten percent of the certificate authorities cannot provide OCSP stapling as a mechanism to the clients. Another major problem is that a server can send only one revocation state during the TLS handshake before TLS 1.3. In contrast to CRL and OCSP, a client cannot validate the full certificate chain but just the server certificate itself. For client software, OCSP stapling might even be suitable, as mechanisms like CRLSet or OneCRL can check intermediate certificates. For middleware devices, OCSP stapling could be a perfect mechanism as server certificates can be checked by OCSP stapling, and other certificates can be checked by CRLs. Still, we must not forget about the relatively low level of server support, 34-36 percent among the top 150 thousand domains, in that case.

The situation is exacerbated by the fact that more than one percent of the intermediate CA certificates contain neither CRL distribution points nor OCSP responder, making the whole chain revocation check impossible in the case of these certificate authorities.

Validity

Certificate status information – independently from the revocation check mechanism – must be verified by the certificate authority which issued the certificate in question. Without this, the status information would not be authoritative, and a malicious third party can modify it without the knowledge of the peer who tries to verify the status information.

Certificate Revocation List (CRL): Revocation lists can be signed by the certificate authority, which issues the server certificate directly or indirectly. It can be verified easily – when the revocation list is signed by the issuer of the server certificate – as the peer must have all the issuer certificates to verify the server certificate itself. There is an additional step if the revocation list is signed indirectly by the issuer of the server certificate, as the revocation state of that intermediate certificate – which has signed the revocation list – has to be also checked, which means that we have to acquire revocation list for this intermediate certificate. This revocation check mechanism highly depends on digital signatures, as the integrity of the revocation list is guaranteed by a signature. Weak signature algorithms (e.g.: MD5, SHA1) have completely disappeared as a certificate signature algorithm – according to Qualys SSL Pulse – but already used as a signature algorithm of the revocation lists, which potentially makes possible compromising the revocation information. Two fields control the lifecycle of a revocation list. The update field shows when the CA issued the revocation list. The revocation list will be issued no later than the value of the next update field, independently of whether the content of the revocation has been changed. A revocation list is considered valid between these two dates, apart from delta CRLs – however, updating the revocation list before a validity check is necessary.
Online Certificate Status Protocol (OCSP): OCSP responses, just like revocation lists, can be signed by the certificate authority which issues a server certificate directly or indirectly, so the problem is the same as it was in the case of the revocation list. There could be an intermediate certificate in which revocation state should also be checked, so an extra OCSP response or revocation list should be acquired. OCSP responses also depend on digital signatures. They still use weak signature algorithms even though OCSP is a more modern revocation check mechanism than the revocation list. OCSP requests and responses contain the hash of the issuer's distinguished name and the hash of the issuer's public key to identify the issuer, so there is another place where weak hash algorithms are possibly used. OCSP responses contain this update and next update fields defining a recommended validity interval, just like revocation lists. This interval corresponds to the this update and the next update interval in revocation lists. OCSP responses where next update value is earlier than the current time or the this update value is later than the current time should be considered unreliable.
OCSP Stapling: The situation is almost the same as in the case of OCSP, but there is an additional issue we have to consider. Along with the fact that the server serves OCSP staple, the validation cannot be entirely performed without checking the staple issuer's revocation state, which needs acquiring an OCSP or a revocation list from another port of another server with notable latency.
OCSP Multi Stapling: OCSP multi stapling cannot solve all the issues above. Although a server can send staples about the revocation state of the server certificate and all the intermediate certificates, the number of OCSP staples must not be higher than there were the number of certificates in the chain, so delegated OCSP issuer certificates are uncovered. TLS 1.3 deprecates OCSP multi stapling, but OCSP staple can be attached to each certificate in the chain, that would seem correct at first sight, but it suffers from the same problem as OCSP multi-stapling. Suppose delegated certificates are used to issue the OCSP responses. In that case, a server cannot send their revocation status to the client part of the TLS handshake, so the comprehensive revocation check needs either extra OCSP requests causing additional latency or prefetched revocation lists that traces back OCSP stapling to the plain old revocation list.

Penetration

Certificate Revocation List (CRL): As CRL is the oldest revocation check mechanism, the original RFC (2459) was published more than twenty-one years and the latest major update was released seven years ago, cryptographic libraries support it very well.
Online Certificate Status Protocol (OCSP): As OCSP is not a new story, the related RFC (6960) was released eight years ago, cryptographic library support is vital.
OCSP Stapling: The mechanism cannot be considered new, as there is a proposed standard from eight years ago, and the technical background is the same as OCSP, so both cryptographic libraries and significant web servers have been supporting the mechanism for several years, but it must be noted that early TLS versions do not support OCSP Staple extension. For server software, OCSP stapling is a demanding mechanism because they have to take over a part of the burden from the OCSP responder as they have to be the reliable sources of the staple. For server maintainers, OCSP stapling is a risky mechanism. For security reasons, the OCSP stapling should go together with the OCSP Must-Staple extension in the certificate, which tells the client if a certificate with that extension is served. An OCSP staple must also be served, but if there is an outage on the OCSP responder side or any fetching problem on the server-side, it causes validation failure on the client-side. However, browsers do soft fail revocation check, meaning if the requested staple is not served or not served fast enough, browsers will ignore it. Because of that or not, only the third of the top 150.000 sites provides OCSP stapling, and the dynamic growth in recent years (3-5 percent point per year) seems to have come to a halt.
OCSP Multi Stapling: Although OCSP multi stapling RFC (6961) published right after the OCSP RFC, the most popular cryptographic library, OpenSSL, does not support it neither in its latest stable release (1.1.1g), nor in its development release (3.0 alpha3), even though one of its competitor, wolfSSL, has OCSP multi stapling support since 2017.

Nowadays the focus is on the major browsers, however the problem is the same when we are using a minor text-based web browser or a popular command line tool in an automated environment. Not to mention that although the majority of the internet traffic is HTTPS, we should not forget about the several other protocols (FTP, IMAP, LDAP, LMTP, MySQL, NNTP OpenVPN, POP3, PostgreSQL, RDP, Sieve, SIP, SMTP, XMPP, …) which use TLS, and some others (IPSec/IKE, WireGuard, QUIC, ...) which are not, but may also be affected by the problems of certificate revocation.

Conclusions

The revocation check system has some theoretical and practical problems, making the situation difficult for all stakeholders (client, server, middle application, cryptographic library developers). No revocation check mechanism can fulfill the requirements alone, and their combination also had severe difficulties. The TLS version 1.3 solves the theoretical problems – making it possible to serve OCSP staple for each certificate in the chains – in vain if it increases the load on the shoulders of client and server software developers and decentralizes the problem by involving the new player in the game, namely the system administrator, who is responsible for making this best mechanism available for the customers. In vain, the more and more recent versions of revocation check mechanisms (OCSP, OCSP stapling, and OCSP multi stapling) when they may still trace back to the plain old CRL. In vain is the high penetration of CRL if some certificate authorities do not support this mechanism. In vain, PKI is highly centralized if there is no official collection of revocation check resources. There are too many tripping hazards to do it flawlessly.

Licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Photo by Bernard Hermant on Unsplash.

Minority Report on Let’s Encrypt CAA Rechecking

Szilárd Pfeiffer — Thu, 04 Jun 2020 15:26:33 +0000

After the Let’s Encrypt CAA rechecking incident we have to say that the certificate revocation system is not only theoretically broken, but also in practice. Four days after the incident was published 3 million certificates should have been revoked, but Let’s Encrypt changed course on revocation, because of the minimal security risk. Now, that all the affected certificates have already expired, see behind the problems of the certificate revocation system.

First of all, let us be clear that Let’s Encrypt had a very important mission making the encryption free and easy to use on the internet. Their effort was essential to make it possible that even the smallest internet sites can protect the customer’s data and privacy by using transport layer security (TLS), but now they fall into the deep hole of the broken revocation system.

It is known that there is no certificate revocation mechanism which can satisfy the requirements of each stakeholders, namely client application developers, server administrators, middleware solution manufacturers, certificate authorities, but it must be stated when a certificate gets compromised – until it is revoked and the clients learned the fact of revocation – it should be assumed that a malicious third party – who has no authority to the host(s), which the certificate belongs to – could impersonalize any kind of service (web, mail, remote access, etc.) with any domain names the certificate is validated to. The fact that a certificate was issued during the period for which CAA rechecking issue was persited, does not mean that certificate has been compromised, just there was a possibility that Let’s Encrypt issued a certificate to a domain, which is disallowed the issuance for Let’s Encrypt by CAA record, but malicious third party who wanted to exploit the issue had to be verified the ownership over the domain.

Without going into the technical details, which are discussed in detail by Scott Helme, 3 ways of revocation checking used today must be summarized (skip it if you want)

Certificate Revocation Check Mechanisms

Certificate Revocation List (CRL)

The oldest mechanism which is nothing else than the collection of the revoked, but not expired, certificates managed by certificate authorities. The most important disadvantage of CRL is the size. As the Let’s Encrypt CAA checking issue showed it can happen that millions of certificates must be revoked at the same time. In that case, the CRL size dramatically grows and all the clients (eg: browsers, mail clients, ...) that want to check the revocation state of a certificate must download the CRL file. It might cause significant delays until the website content can be shown to the user. If that was not enough the CRL should be updated regularly. You can imagine what it means on mobile or embedded devices with a limited amount of memory or bandwidth.

That is why browsers do not use this mechanism at all. Chrome and Firefox have their own mechanism respectively CRLSets and OneCRL to solve the problem. Both of them are CRL collections maintained by Google and Mozilla, which are updated regularly in the background, without the user needing to restart or update their browser. It is fair enough if we are talking about Chrome/Chromium and Firefox, what about the several other browsers and other types of clients (eg: mailing clients) and several other protocols that use TLS or the organisations which want to extend the trusted store with their private CAs?

Online Certificate Status Protocol (OCSP)

This mechanism could solve the size issue that CRL suffers from by making it possible to ask a central server (OCSP responder) whether a certificate is revoked or not. The problem is the fact that the OCSP responders are managed by the CAs and if you are asking for the revocation state of a certificate of a website it is quite likely that you are just visiting that site, so you can be tracked in that way.

OCSP Stapling

OCSP Stapling is currently the best method for checking certificate validity.. It solves both the size issue of CRL and the privacy issue of OCSP by transferring the responsibility to the server. In case of this mechanism, the server proves to the client that the certificate which will be shown is not revoked. The server obtains the proof from certificate authority which issued the certificate and the server caching it. When the client connects to the server, it offers the cached proof to the client during the TLS handshake. The bitter pill we have to swallow is that only one revocation state can be sent by the server, prior to TLS 1.3, so in contrast to CRL and OCSP, the full certificate chain cannot be validated, but only the server certificate.

What is a certificate authority good for?

What should we expect from a certificate or a certificate authority? A certificate, at least the domain validated ones, prove that you are communicating with a server which is operated by the authoritative owner of the domain (eg: google.com) and nothing else. Simply put you visit the page you are expected to visit. The certificate authority is the trusted third party who is responsible for checking the legitimacy of the certificate request for a given domain. If the request was valid the certificate authority signs the request to prove that. When a certificate authority learns that the certificate has been compromised it must revoke the certificate, regardless of who is the responsible, the certificate authority or the certificate requester. In this way, the visitor can recognize – using one or more revocation checking mechanisms mentioned above – that there is an issue with the certificate.

After the Let’s Encrypt CAA rechecking issue we have to ask that the revocation is a must or a certificate authority can consider arguments and counter-arguments! In that case this is a theoretical question, as the issue has minimal security risk, but what if a serious issue will happen in the future.

On the one hand, one could say that in that case 3 million internet sites would stop after the revocation. Maybe it could be a reason for a certificate authority to give an unspecified grace period for millions of certificates which were not renewed by the domain owners after the issue had been announced. On the other hand, it needs to be emphasized that all the revocation check mechanisms depend on the fact of revocation. Until the grace period would run out millions of certificates would remain valid and the customers of the services, who use these certificates, cannot recognize that there is any security issue with these sites. In that case there would be a chance for a malicious party to collect sensitive information (eg: passwords, card numbers) when we are visiting an exploited site while we think we are still safe. The customer’s trust is violated because they cannot decide they really want to visit the site which has just a potentially compromised certificate.

The problem is if we do not know the reason for the revocation, we lose this information. There is no kind of moderately compromised certificate. If there is a little chance or it is 100% sure that the certificate is compromised, there is no difference in what the certificate authority should do, the protocol is the same. They have to revoke the certification, there is no room to distinguish those cases. On the other side of this story, we have a simple customer, who does not know what happens. A compromise of a certificate is not their responsibility,as they have no skill to make that kind of decision. That is what a certificate authority is good for.

Business or security first?

It needs to be clarified for whom the certificate authority is responsible to. For the domain owner who paid for the service? For the customer who trusts is in the certificate authority that it will revoke a certificate if there is any chance of compromise? We might have to determine if we are asking the right question. What the certificate authority is responsible for? I believe the answer should be security first and the unbroken confidence in the whole PKI system. There is an obvious similarity between the responsibility of the public health systems in the event of an epidemic and the certificate authorities in the event of a security incident. Assess the severity of the problem, mitigate the adverse effects, keep up the trust in the system, avoid panic. There is also an obvious dissimilarity. While there is no cure for certificate compromise, there is a cure for the running service that uses the compromised certificate, namely certificate renewal.

What could be an unfavorable effect of a certificate compromise? Clients cannot distinguish that the services run by an authoritative owner of the domain or a malicious third party, so a man-in-the-middle attack can be performed and all the data transferred between the service which runs with a compromised certificate and the client traffic can potentially be intercepted by a malicious third party. The user's credentials (eg: passwords) should be considered as compromised and should be changed as soon as possible. The data transferred while the certificate was not renewed may be considered as leaked and altered according to the likelihood of the compromise.

The trolley problem occurs here, which also occurs in case of self-driving cars. How the car should decide if an accident cannot be avoided and either the car owner or the somebody outside of the car will die. If the car prefers the case where the owner dies, the question is who will pay for that type of car, when the end of the situation is definitely no-win. In our case the problem has no such serious consequences, but the question is still the same.

If a certificate authority tends to revoke a certificate in the slightest suspicion of compromise, means that the service will stop, which causes financial loss to the service provider and inconvenience for the client. Why should the domain owner pay for it? A domain owner should pay because the real value of a certificate is the customer trust of the PKI system, where the trust is represented by a certificate authority.

If the protocol depends on the amount of the affected certificates, what would be the boundary, where a certificate authority can postpone the revocation? Security issues like the one in question, or more serious ones, may and always will happen. It is not great, not terrible. The problem is the information delay and the risk it carries. Should anybody trust the PKI system if certificate authorities can hide security issues from internet users by not revoking possible compromised certificates? It is a question even if the hiding is a temporary act, caused by only a postponement of the revocation. Anyway the situation solves itself in a relatively short time, if we were talking about Let’s Encrypt, validity is only 90 days, so certificates that might pose a threat to clients will expire in 3 months. In case of other certificate authorities the validity can be any 27 months.

The only option to mitigate long turn around times, is to create an environment where quick and efficient reactions are possible via automation. And that is what Let’s Encrypt excels at, as all the processes around the certificate revocation and renewal are fully automated. Let’s imagine a situation when a poorly automated certificate authority, without Automatic Certificate Management Environment (ACME) protocol support, announces that 3 million, or just 300 thousand certificates are compromised! How much time would be needed to renew all of them, 1 week, 1 month or more? Should we tolerate that services with possibly compromised certificates run during that time if every opportunity was given to the domain owner to renew the certificates? Actually, that was the case. It is just icing on the cake, that Let’s Encrypt issues certificates with the shortest validation period (90 days).

What would be the solution?

Without questioning the significance of business continuity, the position that possibly compromised certificates should always be revoked is hardly modifiable. Namely without this step neither standard certificate checking mechanisms (CRL, OCSP, OCSP stapling) nor the implementations depending on them can work. Both client applications (eg: browsers, mail clients, ...) and middleware solutions become powerless as they are not able to recognize the problem if the compromised certificates are not revoked. The only workaround would be that vendors implement custom, likely not automated, mechanisms to collect and maintain the certificates which are potentially compromised, but officially nor revoked, which would be a waste of significant resources and would be highly vendor specific, just like CRLSets and OneCRL. We should not think of extending the grace period, while the certificates are not revoked. We should think of how the costs and risks of troubleshooting can be minimized on the business side.

Let’s Encrypt showed the clear way of cost and risk minimization. Nothing can be cheaper and reliable than a fully automated system. Security issues are part and always will be part of the IT industry, but reaction times can be reduced. The case in question will solve itself in 90 days as all the affected certificates will expire at most 90 days. If a serious issue would happen to most of the certificate authorities the problem would not solve itself within 90 days, but within 27 months (!) as 1-year max validity ballot failed at the CA/B Forum. Let’s Encrypt voted yes anyway. Under such circumstances, there is no pressure on automation so there is a huge amount of certificates that are not checked and renewed automatically, which may cause that revocation to postpone. What could be the practical solutions?

Revocation times in near future

Both in CRL and OCSP entries there is a field containing the time of revocation. If this time would be in near future in such cases it would give enough, but fixed-term grace period to businesses and it also made it possible to check certificates against the future revocation having the right to decide how that situation is handled. Client applications may warn their users that the certificate of the visited site will be revoked in the near future, so there is a risk to visit. Middleware solutions may apply strict rules on this future revocation according to their configuration.

Automated renewal of revoked certificates

Software solutions that implement the ACME protocol may check whether a certificate will be revoked in the near future just like they check whether a certificate will expire in the near future. In both cases, the certificate would be renewed which would solve the problem within a day as this kind of software solution runs in a daily period. There are already some ACME compatible tools, libraries and certificate authorities in the wild, that we can use to implement our automated certificate renewal process.

Monitoring and alerting of non-automated systems

Many systems cannot be automated, for instance where the certificate authority does not support the ACME protocol or an extended validation (EV) certificate is used, where automation is not an option. In those cases monitoring and alerting systems could give appropriate solutions. Just like in other cases certificates should be checked against the future revocation, but instead of renewal – which is not possible in an automated way – alerts would be sent with the right severity to the right person.

How is the revocation checking broken?

Unfortunately the revocation checking mechanisms have several theoretical and practical difficulties. although the fundament of the mechanisms is the fact of revocation, this alone is far from sufficient. Client applications should have a revocation checking mechanism which could work under real circumstances, but they do not have one.

Certificate Revocation List (CRL)

for client software revocation lists are unsuitable mechanism as
- the size may extend suddenly as the case in question shows
- even small revocation lists would cause significant memory usage on mobile devices because all the revocation lists of the significant amount of intermediate certificates should be stored in memory
- regular update of the revocation lists would cause considerable traffic charge which may have financial implications on mobile devices
- CRL distribution points might inaccessible for a client as it comes from another server (maintained by CA) and different protocol (HTTP instead of HTTPS)
- neither major browsers nor other clients software (for example email clients) uses the CRL mechanism
for middleware devices, revocation lists are almost the optimal solution as
- Middleware devices can prefetch, regularly update and cache the revocation lists easily
- the vast majority of certificate authorities support the revocation list in their intermediate certificates which issue the server certificates directly,
- revocation list can be used to check a full certificate chain, but
- there is a huge deficiency namely, the most popular certificate authority does not support CRL, so almost 200 million certificates cannot be checked that way
for certificate authorities, it is a comfortable mechanism as
- It is not used by large amount of client applications, means there is no huge load on CRL distribution points which came from distributed sources, which makes defense difficult against a DDoS attack

Online Certificate Status Protocol (OCSP)

for client software, OCSP is an unsuitable mechanism as
- although the size of the OCSP server response is small, so the size problem of the CRL has been solved, but
- OCSP responders may or may not accessible for a client as it comes from another server (maintained by CA) and different protocol (HTTP instead of HTTPS)
- all the other problems of revocation lists still exist in case of OCSP, but
- causes another problem, namely, the client can be tracked by an OCSP responder as a client software can check a certain certificate, so it can be concluded that the client communicates with the holder of the certificate
for middleware devices, OCSP is an imperfect mechanism as
- OCSP responses cannot be prefetched and used as a database like a CRL, but
- it can be fetched on-demand and can be cached for later use for a moderate amount of time
for certificate authorities, it is a comfortable mechanism as
- in practice there is no client which use that mechanism to check the certificate revocation, so it cause minimal load on the OCSP responder

OCSP Stapling

for client software, OCSP stapling is a suitable mechanism as
- it solves all the problems of CRL and OCSP, but
- it is supported by only the third of the top 150.000 sites, and the dynamic growth in recent years (3-5 percent point per year) seems to have come to halt
- Furthermore, a new problem arises that has previously been solved by CRL and OCSP, namely only the leaf certificate could be checked by OCSP Staple
  - early TLS versions do not support OCSP Staple extensions
  - in TLS version 1.2 only OCSP Multi Staple extension solves the problem, but it is not supported by the major crypto library OpenSSL
  - in TLS version 1.3 multiple OCSP staples can be used by design
for middleware devices, OCSP stapling could be a perfect mechanism as
- leaf certificates could be check by OCSP stapling
- intermediate certificate could by check by CA
for certificate authorities, OCSP stapling is a demanding mechanism as
- If a certificate authority had 200 million active certificates, which is the actual case of Let’s Encrypt, and only third of servers would support OCSP stapling and a validity period of the staple would be 5 minutes it would result 200.000 requests per second on the OCSP responder, which is apparently causing difficulties
for server software, OCSP stapling is also a demanding mechanism as
- they have to take over a part of the burden from the OCSP responder as they have to be the reliable sources of the staple
for server maintainers, OCSP stapling is a risky mechanism as
- for security reasons the OCSP stapling should go together with OCSP Must-Staple extension in the certificate which tells the client that if a certificate with that extension is served an OCSP staple must also be served, but if there is an outage on OCSP responder side or any fetching problem on server side it cause validation failure on client side, however browsers do soft fail revocation check, meaning if the requested staple is not served or not served fast enough they ignore it

Nowadays the focus is on the major browsers, however the problem is the same when we are using a minor text-based web browser or a popular command line tool in an automated environment. Not to mention that although the majority of the internet traffic is HTTPS, we should not forget about the several other protocols (FTP, IMAP, LDAP, OpenVPN, POP3, RDP, SIP, SMTP, XMPP, …) which use TLS, and some others (IPSec, WireGuard, QUIC, ...) which are not, but may also be affected in the problems of certificate revocation.

Conclusions

To make the long story short the revocation check system has some theoretical and several practical problems. Now a brand new question has arisen in the fundamental operation of the certificate authorities. May or may not a certificate authority postpone the revocation of the possibly compromised certificate because it would made a huge number of internet sites inaccessible?

What would be the threshold, 1 thousand or 1 million sites? In my humble opinion it is a qualitative question instead of a quantitative one. Certificate authorities have to be strict and consistent when they decide on revocation to preserve the trust in the PKI system, especially that in the last few years there were some issues (PROCERT, Symantec, WoSign, …) which tore down the reputation of the certificate authorities, however the case in question cannot be compared to them. In parallel all the certificate authorities must make the certificate handling as easy and as automated as Let's Encrypt does to minimize the risk and the cost of the certificate renewal after a security issue on the certificate authority. It helps to avoid the case when there are millions of possibly compromised certificates still running on servers and helps to declare that if a certificate owner had an easy way to avoid the consequences of a sudden revocation the business need comes after the security.

Licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.