DEV Community: Terrence Chou

Have You Ever Care About Identity Integrity?

Terrence Chou — Fri, 12 Apr 2024 04:28:03 +0000

Because of fascinating features and thorough services, organisations are more willing to embrace public cloud platforms (AWS, Azure, GCP, etc) to enlarge their business footprint. A robust defence and recovery framework against cybersecurity incidents is always a key to ensure business continuity. Intrinsically, every technical stuff could be fully controlled in that people define each of them; in other words, every employee could be a potential exposure point, accidentally revealing their business environment to the external world. According to numerous public researches, it is really. We all must keep in mind that modernisation not only means service frameworks but also cyberattack patterns! Every catastrophic disruption could result from the organisation's pillars and one of them is identity!

Identity Attacks Have Raised Than Ever

In general, the types of attacks we have learned could be summarised below:

To pause your service(s) functioning - The most common case is the DDoS attack and it happens quite often, especially when a bunch of competitors share the same commercial market(s). Intrinsically, this kind of disruption would be a short-term period in that the intention behind the scenes does not aim to completely destroy your service(s), but temporarily stop you gaining revenue from the specific event(s) instead. The common DDoS attack is volume-based, meaning that your business continuity primarily relies on how many atypical requests you could mitigate before they get in your core infrastructure; for instance, leverage your ISP's Anti-DDoS offering or deploy the RTBH (Remote-triggered Black Hole) architecture to achieve.
To infiltrate your business environment(s) and exfiltrate sensitive data - Compared to the DDoS attack, the infiltration attack is much more difficult to prevent in that it could get in your environment(s) via a variety of manners; for instance, visiting a suspicious website or opening a phishing e-mail without too much awareness. An unexpected daemon/process could reside in your house and steal your treasures silently. When you notice that there is something wrong, it does not mean that the event just gets started, but comes to the end instead. In order to gain more granular visibility and formulate a robust runbook whenever an event comes up, most organisations typically implement the NDR (Network Detection and Response) and EDR (Endpoint Detection and Response) solutions to strengthen.

When we look at an essential of those solutions, they enrich both the observability and security on the infrastructure layer without any doubt; however, they do not take too many focus on the application layer. We have lived in an era where every modernised cyberattack aims to manipulate your service(s) and even tamper with your business data instead of taking over your core infrastructure. The most effective and easiest manner is to penetrate any of the identities's permission (as a trojan horse) within your organisation. Is it FEASIBLE?! Our resources are well-protected via a number of security frameworks across different tiers!

However, the truth is that this kind of tragedy has happened over and over again, it is just because you have not been aware of.

ITDR, A New Norm

The most well-known and widely deployed identity store is Microsoft Active Directory (AD) when excluding any of the Identity Provider (IdP) platforms (well, I have not noticed that Active Directory has been launched on Windows 2000 Server Edition since 1999 😗). Although we understand what benefits we could gain from both the managed application services and cloud IAM services, the thing is that the Microsoft service frameworks are still valuable for a large amount of enterprises around the world; Microsoft keeps increasing its market share in the Cloud Infrastructure Services arena that could demonstrate this point.

What does that mean on the other hand? That is to say that you should invest in how to ensure your Active Directory's integrity as much as possible. According to Microsoft's research, over 80% of breaches were caused by identity-based attacks. A general pattern is someone who is neither an IT employee nor an employee within the organisation promotes their permission from a user/guest role to an administrator role successfully without any approval, then takes away/tampers with business-sensitive data.

Let us look at the security boundary of Active Directory. Since it covers not only the IAM principle but also the EDR scenario (all the activities are written in Windows Event Logs), a new security framework (or more precisely, a marketing term 😜) joins the game accordingly - Identity Threat Detection and Response (ITDR). What ITDR does could be spotlighted on the discovery, tracking, and notification pillars.

Discovery - According to your Active Directory's profiles, correlate all of them with the IOA (Indicators of Attack), IOC (Indicators of Compromise), and IOE (Indicators of Exposure) indices to dig into any potential security leak/vulnerability.

Tracking - Every Active Directory's object, including User Accounts, Group Policies, and DNS Records could be modified by any authorised user/role, which means each workflow should contain four W-ingredients to ensure every single change is well-monitored: Who made the change? When did this change take place? Which object was influenced? And, what action applied to this object?

Notification - Since every single change is well-monitored, a warning message will be delivered immediately to inform all the correlatives that something goes wrong and must react right away whenever an atypical/unauthorised modification is detected.

Because of these weapons, you have a clear blueprint of what you should defend and precise guidance on how to consolidate your Active Directory's defence boundary; no fears anymore 😎

You Should Be Greedy For Visibility

How will you take any action once you are aware of something atypical? Why GuardDuty, Inspector, CloudTrail, and other equivalent services are extremely prominent for detecting and responding to any unsecure exposure? In effect, those concerns emphasise one spotlight in common: every access must be well-traced in that each of them could be a potential clue of an incident, which visualises every single behaviour granularly so that you will be able to react promptly if something goes wrong.

As I mentioned from the outset, most modernised cyberattacks aim to open up your infrastructure (door) and take over applications (control) on your behalf; nothing will be deemed illegitimate as usual! Of course, ransomware is a thing you need to beware of; however, Active Directory is another stuff you must pay attention to. Please bear in mind that Active Directory is not a single and individual component, but a complex and multi-relational ecosystem instead. None of the enhancements/optimisations are based on what you feel, but what you observe instead. Data are always out there and waiting for mining, the challenge is always how to utilise them more straightforwardly and efficiently. That is why XDR (Extended Detection and Response) comes into play; X could even mean Anything in that it is a methodology/framework rather than a single product, and ITDR is part of the XDR subsets.

Never Too Late To Commence

Hunting always follows the footprints! In the IR (Incident Response) world, every effective reaction relies on how many clues could be investigated; otherwise, you will be overwhelmed or even exhausted by countless false alarms. Active Directory is an invaluable but invisible contributor, your service framework could not function properly without its reliability. To be hunted or to be hunting? Think about the position you want to be.

Is Host-based Cloud Platform Useless?

Terrence Chou — Fri, 08 Dec 2023 07:48:44 +0000

For most organisations, one of the intentions to kick off their cloud journey is to get rid of maintaining any underlying infrastructure component entirely due to a variety of considerations, for instance, tedious hardware lifecycle and different technology focus. However, does that mean the host-based cloud platform could not present any value to organisations? Not at all! Because everything depends on the use case nonetheless.

Before We Jump Into The Topic...

Because of agility and elasticity, most organisations get started on their cloud journey and get involved in the whole cloud ecosystem widely. Other than those factors, the most fascinating point is the pay-as-you-go model which gives organisations another way to stretch their service capacity for supporting any short-period/temporary situation without investing in traditional infrastructure as they used to. Everything looks quite rational, doesn't it? However, one key sometimes is missed from the outset - Which framework will you adopt for either cloud extension or cloud migration? Rehost, Replatform, or Refactor?

We have heard that embracing the cloud is an inevitable trend quite often, however, the influence it makes is more than just a trend. The whole cloud ecosystem not only breaks the traditional boundaries (roles and responsibilities) but also forms a brand-new working model. This new norm changes each technical team's ownership significantly because each of them is able to provision resources, grant accesses, expose services, and even more without involving other teams as they used to. But, this transition also forms inconsistency and confusion which could potentially result in several side effects, for instance, increasing operational difficulty and unwanted spending, especially when the organisation is a large-scale enterprise.

From the above-mentioned situations, we are able to correlate them with an extremely prominent concern: Do we just want to launch/move every single workload to the cloud without too many changes? Or, are we keen to refactor the service framework completely?

Is Anything Discrepant in Cost Across A Decade?

From my personal perspective, moving to the cloud could not be just for reducing whatever cost because this is a tremendous misunderstanding if you put "cloud is cheap" into your mindset. If it really is, why does the FinOps principle come into play?

However, human nature is an interesting evolution, especially in the cloud era. When we go back to the era of operating everything by ourselves, we pay for the hardware and software lifecycle annually; in other words, we only need to debate the payment once. Because of this reason, we are not regularly chased up by the expenditure until the next cycle comes. The story could be totally different on the cloud albeit we keep the same cost unchanged. Why? What happens? The bill! You are able to monitor every cost generated by each deployment on a daily basis and feel shocked about either invisible spending, unexpected consumption, or even both. Because of this transparency, most organisations are keen to reduce the whole spending before they decide to move on to the next stage.

On the other hand, each CSP encourages every customer to embrace more cloud-native services/features instead of building self-managed frameworks due to a variety of considerations, for instance, solution integrity, product familiarity, modernised architecture, or cost optimisation. If you still do not have a clear cloud blueprint then you will get stuck in the concern I raised previously eventually.

Cloud-native Is Not Really A Must-be!

Is the cloud-native service architecture required? Although the feedback could be positive (Yes, that is a milestone we aim to achieve!) or negative (Well...that is a goal certainly, but it depends on if we are eager to revamp our service framework...), we should deem the whole process as an evolution instead of enforcement. We do not have the Infinity Gauntlet, giving us the power to rewrite anything immediately by snapping our fingers... We shall classify which service architecture will stay unchanged and which one will be revamped in practice. After classification, you will figure out that the most cost-effective manner for hosting those unchanged frameworks is the host-based platform, for instance, Dedicated Hosts or VMware Cloud, and here are the reasons.

Optimise your license fee - Let me use the Microsoft SQL Server as an example for elaboration. There are two purchasing models: per-vCPU and per-core, and the adoption depends on how many SQL Servers you have. Here is a high-level guidance: If you are able to fully allocate your SQL Servers in a single host then you shall adopt the per-core model that makes your license fee lower. In contrast, if a host could not be fully allocated then you shall compare which purchasing model is more suitable because it does not mean that the per-vCPU model is more expensive.

Remain operational consistency (VMware Cloud) - Most organisations get started on their virtualisation journey with VMware vSphere suites, including vCenter (management console), vSAN (storage), and NSX (networking and network security). These components are also VMware Cloud fundamentals across AWS, Azure, Google, and other CSP platforms. Besides the optimisation of the license fee, if the customer not only aims to keep their service frameworks unchanged but also retains their operation excellences as much as possible, then the VMware Cloud will be the most ideal choice.
Gain more capacity (VMware Cloud) - Why VMware is so powerful? Because of the over-provisioning principle! What is it? That is to say, you are able to allocate compute resources (vCPU, memory, and storage) more than a single host has. How come!? In fact, every resource you allocate to every single workload does not mean that it will be fully utilised; you could see the real usage is lower than 50% or even more in most cases. What VMware does is dynamically reallocate these idle resources to anyone who really needs them, ensuring that every resource a single host has could be completely used. Let me use the r5 Dedicated Host as an example for elaboration, it is able to load 24x r5.xlarge instances; in other words, it could afford much more than just 24x r5.xlarge instances based on the VMware technology.

You could also refer to my post Migrate On-premises Workloads To AWS that introduces the VMware Cloud on AWS architecture in depth.

Slow Down Your Pace

Essentially, the cloud itself is a journey of transformation; it could be an evolution (It is the right time to get rid of legacy architectures) or even a revolution (Why do we have to change?). As I mentioned earlier, nothing could be revamped just by snapping our fingers simply. Every intention has a background, in order to carry out our intentions, we must have a blueprint (How will we get there?), define checkpoints (Are we on track?), review the whole progress (Does anything we missed?), etc. If you look at this journey carefully, you will be aware that this framework is not a single cycle, instead, it takes place over and over again because how frequently each feature, service, or even partnership will be published in the cloud world is faster than your imagination; that is a reason why you shall always keep your mindset in the Day-1 state.

To slow down does not mean pausing everything, instead, it gives you a space to verify what is the goal you aim to fulfil and do you align with it.

Interconnect Mesh Does Matter

Terrence Chou — Wed, 06 Sep 2023 08:48:46 +0000

Leased line or IPSec VPN? These two terminologies always arise when you want to bridge two locations. In this cloud-massive era, IPSec VPN could still be easily deployed in any architecture because what it needs in essence is the Internet; however, when we turn to the leased line, what we could do since we have nothing in the on-premises?

How Did We Adopt In The Past?

Leased line or IPSec VPN? The adoption primarily depends on one of the following factors, some of them, or even all of them if technologies are met, e.g. protocol/feature:

Permanent/temporary use - If the scenario is a PoC, you typically would not prefer ordering a dedicated circuit to verify your requirement from either the schedule aspect, the cost aspect, or both. However, if the scenario is opposite, which means it is a production environment, the Internet-based VPN typically is not the first priority for consideration.
With/without SLA - Not every case you would need a commercial agreement to safeguard your business even if the scenario is a production environment because the requirement highly depends on what is the magnitude of the service.
Cost - Every solution, no matter it is open-source-based or commercial-based, could be divided into two pieces: CapEx and OpEx. The CapEx primarily focuses on how much budget I need to scope and how much expenditure I need to pay. The OpEx primarily emphasises what needs to beware if leveraging any existing resource, e.g. capacity or reliability.

When we turn to technical requirements, e.g., do we have sufficient infrastructure resources to deliver (router/switch/firewall), do those resources have sufficient licenses to support (BGP/GRE/IPSec), we typically do not concern them too much because they are easily qualified by the existing environment.

But, what could we do for the leased line in the cloud world?

Is Anything Changed Nowadays?

Intrinsically, you do not need to panic because the leased line architecture is still out there, it just functions differently. If so, you may be interested in what are the discrepancies between the eras. Software-defined Cloud Interconnect (SDCI) is the answer. The SDCI architecture provides a more agile, flexible, and modernised model to link up with any CSP environment. Compared with the traditional leased line model, the SDCI framework has the following spotlights:

Self-provisioning - In the past, you were able to follow up all the tasks on your side until the circuit was delivered; typically, the lead time took around 2-4 weeks. Based on the SDCI framework, nothing needs to be waited because everything could be manipulated by yourself. What you need to do basically is choose the PoP where is close to your business and the required capacity, feed the credentials from the CSP you specified, and deploy. You could even integrate the existing deployment pipeline with the SDCI platform to achieve the infrastructure-as-code (IaC) principle.
Commitment-free - In the past, one of the key factors in the order process was bandwidth commitment, which meant you had to pay for the bandwidth you did not completely consume. The charge model of each SDCI component is subscription-based as well as those CSPs and SaaS vendors; you would only need to pay for how many resources you actually consume.
Carrier-neutral - Before Google announced its Cross-Cloud Interconnect offering, there was no way to bridge various clouds via a single cloud solution because each of them is proprietary. However, the SDCI framework crosses this boundary; it acts as an octopus that is capable of integrating every cloud with it simultaneously.

Therefore, let us look at Megaport who is one of the well-known SDCI solution providers in the market.

Solutions Outline

Leveraging either the CSP-managed VPN service or the 3rd party firewall instance-based VPN to bridge the clouds is not really a concern if you do not have too many sites; however, it would be tedious and overwhelming once you have tons of environments that need to be managed. In the era of everything seeking out efficiency and as-code gives Megaport an opportunity to demonstrate its capability which is composed of three key offerings: Port, MCR, and MVE.

Hybrid Cloud

Port is delivered as a physical circuit that is capable of linking multiple clouds. As a matter of fact, this offering is quite common, especially from the NSP's aspect, e.g. PCCW, Equinix, hence I do not want to dig it too much detailed.

Cloud-to-Cloud

MCR (Megaport Cloud Router) functions as a concentrated exchange point that is capable of managing the routing across multiple clouds. Unlike Port adopts the layer-2 design to bridge the on-premises with the clouds (the routing relies on the network infrastructure in on-premises), MCR adopts the layer-3 design for every cloud-to-cloud scenario (the routing is totally handled by itself); for this reason, you could construct a hub-spoke transport architecture on either per-region, per-cloud, or even both basis.

Let us slow down our pace for a while. You are probably aware of one single term - VXC (Virtual Cross Connect) because it is associated with both Port and MCR. Actually, VXC is what we are looking for the leased line in the cloud world. Each VXC represents a connector of the destination, e.g. Amazon, Microsoft, or Google. In addition, each VXC could not function without associating with Megaport's services; otherwise, it is just an object.

You are probably curious about how could each VXC be deemed a leased line? The answer is quite straightforward because every single VXC behind the scenes is a CSP-managed interconnect resource, e.g. AWS Direct Connect, Azure ExpressRoute, or GCP Cloud Interconnect. The only discrepancy is that you are not responsible for operating any network infrastructure in on-premises, Megaport takes over this ownership.

Technical Deep-dive

I observed one thing that is worth keeping in mind during the PoC is how BGP Prefix Filter works on MCR because it functions differently when compared with how BGP behaves in general.

Import Prefix Filter does not mean which prefix is received from your BGP peer is allowed to feed into your route table, instead, it means which CIDR of the VPC/VNet is associated with this BGP connection. As you see from my PoC architecture, there are two CIDRs (10.150.16.0/20, 10.150.224.0/20) from my AWS account. From MCR's aspect, those two prefixes must be allowed to import, otherwise, you will not see any prefix in the Received section.

Export Prefix Filter does not mean which origin prefix you want to advertise to your BGP peer, instead, it means which prefix you receive from your BGP peer is allowed to propagate; in other words, which prefix is able to transit over this BGP connection. Looking at my PoC architecture again, there is a single CIDR (10.160.224.0/20) from my Azure account. From MCR's aspect, this prefix must be allowed to export, otherwise, you will not see any prefix in the Advertised section; in other words, you will not any prefix in your VPC/VNet Route Table, either.

Enhanced SD-WAN

MVE (Megaport Virtual Edge) is based on the SD-WAN foundation to provide an optimised path between the on-premises and the CSP/SaaS platforms; in other words, MVE has a prerequisite of supported vendor platforms.

You may wonder why Megaport is able to optimise the network path for SaaS platforms via MVE? The answer is composed of two pieces:

First of all, the Megaport-managed network backbone, which is close to the colocations/datacentres where those CSPs locate. This factor is a prerequisite for most SD-WAN solution providers in the market under the hood.
Secondly, a comprehensive ecosystem from Megaport Marketplace, the most decisive factor from my perspective. Every certified service provider could be represented by a VXC and associated with MVE.

Apparently, since everything is under Megaport's umbrella, the user experience could be eminently optimised.

Wrap-up

One of the reasons why Megaport's offerings are fascinating is its highly elastic design from my perspective. Each service on the bottom could be represented by a VXC and each VXC could be associated with either Port, MCR, or MVE. Imagine that we are in the Lego world, every tier (VXC, Port, MCR, and MVE) could be deemed the Lego blocks, and what those blocks will present that depending on your imagination (requirement).

Although one of the transitions in the cloud era is the Internet is a new network, when we take either the business drivers (e.g. SLA) or the operational concerns (e.g. security) into account, the SDCI solutions are worth evaluating and embracing for your modernised service framework nonetheless.

Access Leak Is Extremely Easy To Penetrate In Your Environment

Terrence Chou — Mon, 17 Jul 2023 01:44:59 +0000

Before We Get Started...

Is access leak easy to penetrate in my environment? The answer is Yes without any doubt. Therefore, another question may arise in your mind: if it really is a spotlight that is worth keeping an eye on, why did I not adopt any reaction in the past accordingly? The answer is cloud.

Before cloud adoption gets popular, every entry point of access is well-defined by either Security or the well-trained operation team.

All kinds of management accesses, e.g. HTTP or SSH, could only be granted from specific sources, no matter the request is initialised from either the internal environment or the Internet.
In addition, all bidirectional communications must be passed through the peripheral appliance, e.g. firewall.

Obviously, there are not too many ways to accidentally expose your services with unwanted profiles. However, what happened after cloud adoption had explosive growth? We all understand one of the cloud essentials is convenience, because of this strength, developers no longer require collaborating with the infrastructure team to publish their ideas over the world, they could do everything they want by themselves completely. On the other hand, this convenience results in unnecessary exposure. When we turn to the previous scenarios, they would look like below:

Developers accidentally expose their resources, including databases, to the Internet directly.
Developers accidentally expose not only the front-end data plane but also the back-end management plane to the Internet.
Developers accidentally grant all the sources, including unknown ones, to communicate with their resources.

Last but not least, developers have not been aware of those anomalies (that is not their fault in essence due to their primary focus is development instead of operation), therefore, some unexpected security events took place afterwards, e.g. ransom attack.

What Do You Need To Take Care Of

A well-protected framework does not mean blocking everything, instead, every access is enforced with the least privilege principle. The whole framework could be narrowed down to the following categories from my perspective:

Configuration review aims to verify if any setup does not follow either standard practices or best practices.
Workload segmentation aims to categorise all the resources into more granular groups and formulate different sets of communication boundary.
Privilege management aims to ensure that every resource could only be manipulated by a limit of users or service roles with the least required permission.

Configuration Review

In terms of reviewing your configuration on AWS, the most efficient way is leveraging Security Hub, a cloud security posture management (CSPM) service that automates best practice checks, aggregates alerts, and supports automated remediation. The following demo gives you a high-level overview of what Security Hub does.

Choose the Security standards that meet your requirement.
Security Hub will summarise all the findings and present those details once the collection process completes.

Essentially, Security Hub is to provide a set of security definitions instead of detecting and remediating any anomaly; these functions are handled by Config instead, a service that enables you to assess, audit, and evaluate the configurations of your AWS resources. As a result, enabling any of the Security Hub's security standards will automatically deploy the corresponding rules to Config.

However, one thing needs to keep in mind is that Config is disabled (off) by default, it could not detect, collect, and remediate constantly until its status is enabled (on).

Those predefined rules may not 100% meet your requirement, hence customisation is required. The following demo presents that a manual remediation rule is launched to inspect if any unauthorised source is detected. Every change made by Config, no matter automatically or manually, would be recorded by CloudTrail as well.

Workload Segmentation

I emphasised how the importance of segmentation in my previous post A Modernised Landing Zone Design Rather Than Just About Reachability, however, a robust security design not only consider the site segments but also the workload segments. Every workload segment could be easily accomplished via Security Group.

Everyone knows that each Security Group is responsible for access control by its nature, but what it does is more than that! When we look at the best practice principle, each component shall have its own Security Group. If a Security Group is either associated or shared with multiple resources then these resources belong to the same segment. In addition, you shall avoid using the IP address to manage your access, the self-referencing rule instead (Security Group over Security Group, or nested Security Group), which gives you a more granular and flexible way to authorise every communication.

The benefit you could get from the self-referencing rule architecture especially when the resource scales by the auto-scaling group due to the IP address allocation is floating in this manner. The IP address shall only be considered in the following scenarios:

The health-check request initialises from the Network Load Balancer (NLB) subnet(s).
Any out-of-VPC communications, e.g. Internet or inter-VPC.

Other than manipulating Security Groups to manage segments, you could also consider leveraging the 3rd party software to streamline your operation.

Aviatrix - Distributed Cloud Firewall, an agentless solution that provides an orchestration layer to simplify every access rule's change. My previous post Enhanced Management of Multi-cloud Networking and Security introduced why Aviatrix is worth considering and what are the benefits that an organisation could get from it as well.

Illumio - Illumio Core, an agent-based solution that could decouple each service group into a more granular slice (micro-segment). Illumio also highly advocates why Zero Trust Segmentation matters, especially from the aspect of preventing ransom attacks.

Privilege Management

We already talked about remediation and segmentation, let us move on to delegation. Obviously, IAM acts as a key factor in this section; however, it could only apply to AWS correlatives, e.g. IAM users or service profiles. If the OS layer is handled by an organisation and the organisation has leveraged Active Directory to manage user and service identities then implementing Group Policies would be the most efficient approach.

When we look at the whole privilege management framework, it primarily focuses on access and identity management from my perspective.

In terms of privileged access management, its principle intrinsically is zero-trust, none of the users is able to manipulate any resource directly; instead, all the manipulations are by way of a managed orchestration layer, which minimises unnecessary exposure, e.g., several unknown communications caused by a malicious process inside an installed package.
In terms of privileged identity management, it covers permission and credentials. Which user could manipulate which resources without needing any firewall between them; in addition, none of the resource credentials could be used directly, instead, every authorised user would be given a temporary key to manipulate. This bundle of the role-based access control (RBAC) foundation and the vault protection enhancement could minimise the risk effectively if credentials were revealed accidentally.

This resource outlines a full picture of what privilege management looks like.

Let Us Wrap Up!

In the past 5 years or even a decade, we could trust that our business was well-safeguarded by the anti-virus and anti-malware software; however, this methodology is no longer enough at all nowadays due to attacks keep innovating as well as technologies. Additionally, because of the maturity and popularity of cloud adoption, everyone has the chance to fulfil their ideas without too many engagements across teams; that is to say, they even could complete everything by themselves. But, when everything becomes too easy and convenient, it sometimes gives us a heads-up that something may need your attention, which is security in most cases.

Everyone needs to care and is responsible for security in this fast-paced generation; more accurately, everyone has to always bear security in mind due to anything that cannot be seen does not mean that it does not exist, it is just because this trick-or-treat is patiently waiting for your carelessness. No one wants to get shocked, does not it?

A Modernised Landing Zone Design Rather Than Just About Reachability

Terrence Chou — Mon, 17 Apr 2023 01:03:25 +0000

Preface

Let us forget anything about the cloud before we get started on the subject today. In the past, when we needed to launch any service on top of the IT infrastructure, what we did could be summarised below typically;

Evaluate how much capacity is required to afford the loading.
Consider how to safeguard applications that expose to the Internet with minimum compromise or without compromising the degraded performance.
Consider how to govern communications between applications across environments with elasticity and granularity.

When we look at the cloud era nowadays, the first task is completely offloaded to the CSPs, hence it is no longer a concern (well, the only thing that you definitely need to care about is how to ensure that you will not get surprised when you receive the bill 😑). However, the rest of the tasks are still our responsibility. Before considering anything about protection, management, or both, you need to build a place to accommodate those business-critical applications; but, what does the landing zone differ from the on-premises infrastructure design? Because a landing zone is able to deem an SDDC (software-defined data centre) in essence.

That is a great question, is not it 😎? Concisely, the traditional infrastructure focuses on reachability; however, the landing zone much focuses on application-driven design. What does it mean? The next section will discover more.

A Modernised Landing Zone

In the SDI (software-defined infrastructure) world, not all the requests would be handled by the same workflow; as a result, three different types of workflow come up: the control, data, and management planes. In the following scenarios, I will take the data and management planes to elaborate on what a modernised landing zone looks like.

Internet Access: Origin Exposure

We all know that there are two types of origin from the CDN aspect: static and dynamic content.

Static Content - Intrinsically, the output that could be fetched by way of the HTTP GET request belongs to this category, e.g. images (.jpg). In addition, these types of content are cacheable due to they could be gained directly without interacting with applications.
Dynamic Content - If the output is fetched by way of several interactions between applications, e.g. HTTP POST, that is to say, it depends on the request then this output belongs to this category, e.g. Web pages (.aspx). Because of this reason, these types of content are not cacheable.

In other words, the dynamic content relies on the compute resources and the static content does not; as a result, that is why each CSP encourages every customer to serve their static content via the object storage, e.g. AWS S3 and GCP Cloud Storage, instead of the block storage. This adoption is not only about the requirement but also about cost optimisation.

CDN is an Internet-face service, meaning every origin must be exposed to the Internet; however, from the security standpoint, every exposure is risky. For this reason, we normally add additional logic on the CDN and origin side for enhancement.

The 1st gate - When the CDN receives the request without any authorised HTTP header, this request would be dropped.
The 2nd gate - When the request passes the inspection by CDN, the CDN adds one or more additional HTTP headers; when the origin receives the request comes from CDN without those HTTP headers, this request would be dropped as well.

But, what if exposure is not required? It would be great from the security standpoint. Luckily, it is feasible when S3 collaborates with CloudFront via OAI (origin access identity) or OAC (origin access control).

The primary benefit we could get from this manner has two if compared with the 3rd party vendors;

The S3 bucket keeps private only, meaning the origin is invisible from the Internet (the bucket must expose to the Internet for any 3rd party vendor communicating with).
You could still benefit from all the enhancements by CloudFront due to the communication between S3 and CloudFront is well-defined and well-protected on the resource level via the bucket policy (it is not straightforward to manage where the 3rd party vendors come from).

Internal Access: VPC Endpoint

When we turn to the internal environment, every Internet access must pass through the NAT process. However, it does not mean that every AWS public service adopts the same way; there are two exceptions, DynamoDB and S3. Both DynamoDB and S3 could be accessed by way of the VPC Gateway Endpoint without involving address translation; all the resources within the VPC are able to access those services via their internal/private address.

But, not every service could be accessed publicly due to they have not been exposed outside of the AWS world and they will not, either. For instance, KMS, Session Manager, and System Manager. In order to reach those services which cannot communicate directly, the VPC Interface Endpoint is required.

The primary difference between the endpoints could be summarised below.

The Gateway Endpoint needs to be associated with the VPC Route Table, but the Interface Endpoint does not.
The Interface Endpoint is able to customise the access control, e.g. service policy and Security Group, but the Gateway Endpoint does not.

Most importantly, what is the message that the VPC Endpoint would like to share with us? That is to say, every solution architecture must be application-aware due to one of the significant differentiators between on-premises and cloud is not everything is reachable on the cloud by default, require additional settings instead, e.g. adaptor (VPC Endpoint) or permission (IAM Policy); as a result, reachability is not a foundation anymore.

This point not only reflects the changes across technologies but also notices that our responsiblity enlarges due to the infrastructure design closely works in conjuction with the application functionality in a modern development framework.

Multi-site Access: Segmentation

Unlike the on-premises environment where everything is put together, which is caused by several factors, e.g. facility location and resource capacity, a system architecture on the cloud is typically composed of more than single one landing zone, e.g., the infrastructure components are deployed in zone 1, the application components are deployed in zone 2, and the security components are deployed in zone 3. This partition does not complicate the whole architectural design, instead, it gives more granularity and elasticity, especially from the ownership perspective.

Essentially, you shall not anticipate that any workload that resides in either the development or the staging environment is able to communicate with any service in the production environment. However, rely on the Network ACL and Security Group to grant access that could be optimised. Another gate that is worth considering is segmentation, to categorise the type of traffic. In order to accomplish traffic categorisation, you could leverage either the Transit Gateway Route Table or the Cloud WAN Segment; the adoption depends on which service (Transit Gateway or Cloud WAN) you use for bridging all the environments together. You could see more in-depth comparisons between Transit Gateway and Cloud WAN in my post - Globally Operate Your Network Transport via Cloud WAN.

Segmentation is just the outset due to it is a prerequisite of traffic inspection, which is a key piece of secure cloud networking. Every communication, no matter the east-west or the south-north access, would be inspected by the security appliance which could be either the Network Firewall or the 3rd party software. You could even leverage the Security Group to form more granular and tiered access management.

Out-of-band: Session Manager

In order to ensure that every underlying infrastructure component is manageable, typically, we would design a completely isolated network instead of operating them via service network, which is also shared with applications. This design gives us a back door to investigate what is happening whenever the service network malfunctions. When we turn to the cloud world, every remote access relies on either a dedicated connection (Direct Connect) or the Internet, and these ingredients belong to the service network in essence; that is to say, you will not be able to operate resources if they are being either maintained or malfunctioned. Is any way to touch the environment without integrating any abovementioned feature? Yes, that is Session Manager.

You neither need to maintain any extra key pair nor operate any additional resource that Session Manager governs. The only thing you need to do is just connect your resource, which is the EC2 instance in particular, via Session Manager. In addition, another security optimisation is that none of the service ports, e.g. RDP and SSH, is required to open due to the communication between the EC2 instance and Session Manager is behind the scenes. Pretty straightforward, is not it 😎?

Wrap-up

As I mentioned at the outset, every landing zone could not inherit the traditional infrastructure criterion to design and implement due to their emphasis is totally diverse. Every landing zone acts not only just a foundation but also as a crucial role to support business functioning with maximum flexibility and minimum security compromise.

If we look at origin exposure, VPC Endpoint, segmentation, or Session Manager as an individual, then you perhaps ask yourself "Should I need them?"; however, whilst we take those enhancements into the same territory, then the outcome is absolutely different.

Supercloud 101

Terrence Chou — Thu, 09 Feb 2023 16:02:38 +0000

What Is Supercloud?

What is Supercloud? You probably have several questions, e.g.,

Is it a new terminology?
What does it differ from the Private Cloud, Public Cloud, and Multi-cloud?

Before we dive into each of them, let us retrospect the transition between the cloud models at the outset.

Typically, most organisations get started from on-premises data centres/colocations where aka the Private Cloud; their operation team fully handles the underlay infrastructure routines, e.g. procurement, installation, or optimisation.
Because the benefits introduced by cloud computing have been accepted widely, more and more organisations kick off their digital transformation journey and move their business on either Amazon, Microsoft, or Google these Public Cloud vendors.
Essentially, when an organisation gets enough familiarity with a single CSP, its offering might not meet the organisation's business requirements across the functionality (Google is strong in data analysis) and risk (should we not put everything on Amazon?) considerations. As a result, the Multi-cloud strategy would be formulated.

When an organisation moves to the multi-cloud phase, either multiple CSPs or with on-premises environments (aka Hybrid Cloud), one thing that would be emphasised is consistency, either architectural or operational. In terms of compute resource deployments, you definitely could extend the existing deployment framework to another CSP and even more; however, it takes time due to each CSP has proprietary service architectures and they are not compatible with others. For this reason, would it be more simply and efficiently if there is an abstracted layer between the operation team and clouds? In other words, the operation team does not require to directly communicate with respective CSPs, instead, they just need to talk to that orchestrator and it would be on their behalf to face those clouds. That is what Supercloud aims to be!

Here is the definition from Supercloud Working Group Definition.

Supercloud is an emerging distributed computing architecture that comprises a set of services abstracted from the underlying primitives of proprietary clouds (e.g., compute, storage, networking, security, and other native resources) to create a global system spanning to all clouds it is interfaced with. In principle, Supercloud has the capability to allow the integration of any current and future hyperclouds or other proprietary cloud architectures.

The following sections will discover more.

Is Supercloud An Evolution of Multi-cloud?

I know that the Supercloud is a bit controversial due to not everyone agrees with it as one of the cloud models. In addition, someone deems that it is a marketing term. As the matter of fact, I do not want to debate what it really is; however, one thing is worth us keeping in mind is that the multi-cloud strategy has transitioned to a by-default pattern instead of by a design requirement.

Because of this transition, you probably wonder that could we deem the Supercloud a next-generation multi-cloud framework? Or, should we treat it as an enhanced Distributed Cloud? Well, in my humble opinion, it is neither the multi-cloud nor Distributed Cloud, it is both instead. That is to say, it shall be recognised as a multi-cloud based Distributed Cloud architecture.

Representatives

Although the Supercloud architecture does not belong to any specific cloud model, it is presented through the IaaS or SaaS manner.

Aviatrix: Air Space and CoPilot, IaaS

For those who have not known Aviatrix, here is a high-level introduction.

The pioneer of Intelligent Cloud Networking™, optimizes business-critical application availability, performance, security, and cost with multicloud networking software that delivers a simplified and consistent enterprise-grade operational model in and across cloud service providers.

What it actually does is be an orchestrator/abstracted layer to concentrate your network transport management across clouds. As the following diagram, every data-plane provision (e.g. Aviatrix Gateways or AWS Transit Gateway) and manipulation (e.g. adjustments of the VPC/VNet Route Table) is fully handled by Aviatrix Controller.

For this reason, the operation team does not need to integrate their deployment pipelines with individual CSPs; additionally, they do not need to login each CSP's console for either seeking where does the compute resource locate across accounts, regions, and landing zones, or observing if anything is abnormal on the transit network.

My post Enhanced Management of Multi-cloud Networking and Security has a very deep-dive introduction in why an organisation might need Aviatrix and what are the benefits that an organisation could get from it. It is worth taking a look!

VMware: Cross-Cloud Services, SaaS

Typically, most organisations adopted VMware vSphere for their compute resources before the concept of the cloud gets popular; therefore, the entire VMware Cloud offering would be a fascinating choice if they do not really need those comprehensive features each CSP caters.

As of today, VMware has launched its vSphere stack on Amazon, Microsoft, Google, IBM, Oracle, and Alibaba for catering any preference across CSPs; in other words, you could leverage existing deployment pipelines to manage your virtual machines via vCenter and containers through Tanzu, regardless of which the CSP is. Other than that, both VMware Aria (formerly vRealize Cloud Management) and VMware NSX Advanced Load Balancer (formerly Avi Networks) adopt the same way as well. The VMware Cross-Cloud Services is not a single platform, instead, it is a total solution or even an ecosystem.

My old post Migrate On-premises Workloads To AWS outlines the benefits you could get from VMware Cloud on AWS. Although it is a bit outdated (I released it in 2019), the core concept is still valid; additionally, you could get more information from the YouTube links below;

Intro: How VMware Cross-Cloud Services Works
Demo: Application Transformation, Accelerating Cloud Transformation, and Hybrid Workspace

Conclusion

Although the term "Supercloud" is not acceptive across the board and I rarely hear it from the market, I still agree with its principle - distributed and abstracted across clouds. Eventually, every organisation will move to this stage when its business is launched on the multi-cloud architecture.

In this extremely fast-paced era, how could an organisation augment its business territory more rapidly and operate its environments more efficiently? Consistency is the only answer, it does not matter whether you are a resource consumer or a platform provider.

Globally Operate Your Network Transport via Cloud WAN

Terrence Chou — Fri, 13 Jan 2023 06:42:46 +0000

Retrospect

In my old post, The Evolution of Cloud Networking on AWS I elaborated on what and why Transit Gateway could revamp your network transport. Although Transit Gateway has been generally available since November 2018 (ready to 4th anniversary), it is still the most powerful feature in the Cloud Networking space across the board.

If you think that Transit Gateway will be the last fascinating networking offering then you are definitely wrong!

New Launch

In July 2022, AWS formally announced another cool feature called Cloud WAN. As the matter of fact, I was a bit confused about its name due to WAN typically means external/public networks; however, what Cloud WAN is responsible for is not really about WAN, instead, it is more about globally consolidating all of your network ingredients, e.g. VPC, Transit Gateway, Site-to-Site VPN, and SD-WAN across regions into a single and unified management console.

Other than the name, I was also a bit confused about the key differences when compared with Transit Gateway in terms of their functionalities, especially after I read the Preview post. As a result, the following questions came up in my brain;

Is Cloud WAN able to completely take over all the functionalities that Transit Gateway supports?
Does Cloud WAN aim to replace Transit Gateway entirely?

In terms of the standpoint of the key offering, I do not think that AWS is willing to see there are two products without any differentiators. The best way to demystify any uncertainty is always to deploy, verify, and observe instead of reading documents without any implementations.

Comparison

From my viewpoint, it would be more straightforward to dig a new feature by comparisons; hence I use the Transit Gateway architecture to compare with the Cloud WAN one. Its architecture involves the following scenarios;

Each Transit Gateway has four Transit Gateway Route Tables which aka VRFs 😁.
Cross-region communications are via Transit Gateway Peerings.

When transforming to the Cloud WAN architecture, it looks quite similar to the Transit Gateway one; however, there are a few of differences between them.

First of all, the role and capability of Core Network Edge or CNE are typically identical to Transit Gateway; the only difference is that all the CNEs within the Cloud WAN core network automatically link with each other by nature. When turning to Transit Gateway, you need to manually create the Transit Gateway Peering for cross-region communications instead.

As following screenshots, all kinds of attachments, e.g. VPC and Site-to-Site VPN are associated with corresponding Core Network Edges as well as the Transit Gateway model; however, you might be interested in what is the Transit Gateway Route Table attachment?

Typically, the Transit Gateway Route Table attachment is equivalent to the Transit Gateway Peering attachment; create the peering connection and then associate it with the segment. In addition, one thing needs to keep in mind is that Direct Connect does not support to be directly associated with Core Network Edge yet; therefore, you need to leverage Transit VIF at this stage.

Secondly, each Cloud WAN segment exactly functions in the same way as Transit Gateway Route Table does; a slight difference in between is that you could leverage specified tag set(s) to move any Cloud WAN attachment to another segment. In my case, I use the key cloud-wan-segment as a condition and the value Development; as a result, that means once this criterion matches, this attachment will be moved to the Development segment. When turning to Transit Gateway Route Table, you need to re-associate the specified attachment instead. From my viewpoint, this feature is extremely handy, especially when you do not need to involve any change of the configuration.

Thirdly, a whole Cloud WAN core network could be completely managed via a JSON file by nature and every change is well-recorded (versioning), too. When turning to Transit Gateway, it does not have this feature in place, you need to manage it by your automated framework instead. In terms of change management, this feature is a significant spotlight of Cloud WAN from my perspective. Because of the versioning, you are able to roll back any change more efficiently; even before you commit any changes, you are able to see the comparison between the current profile and the new profile. This convenience lets me recall Cisco IOS-XR and Junos due to that is what they behave 😁.

{
  "version": "2021.12",
  "core-network-configuration": {
    "vpn-ecmp-support": true,
    "asn-ranges": [
      "4200000001-4200000100"
    ],
    "edge-locations": [
      {
        "location": "ap-southeast-1"
      },
      {
        "location": "ap-northeast-1"
      }
    ]
  },
  "segments": [
    {
      "name": "Shared",
      "require-attachment-acceptance": true,
      "allow-filter": [
        "Production",
        "Development",
        "Staging"
      ]
    },
    {
      "name": "Production",
      "require-attachment-acceptance": true,
      "allow-filter": [
        "Shared"
      ]
    },
    {
      "name": "Development",
      "require-attachment-acceptance": true,
      "allow-filter": [
        "Shared"
      ]
    },
    {
      "name": "Staging",
      "require-attachment-acceptance": true,
      "allow-filter": [
        "Shared"
      ]
    }
  ],
  "segment-actions": [
    {
      "action": "create-route",
      "segment": "Shared",
      "destination-cidr-blocks": [
        "0.0.0.0/0"
      ],
      "destinations": [
        "attachment-0123456789abcdefg"
      ]
    },
    {
      "action": "create-route",
      "segment": "Production",
      "destination-cidr-blocks": [
        "0.0.0.0/0"
      ],
      "destinations": [
        "attachment-0123456789abcdefg"
      ]
    }
  ],
  "attachment-policies": [
    {
      "rule-number": 100,
      "condition-logic": "and",
      "conditions": [
        {
          "type": "tag-value",
          "operator": "equals",
          "key": "cloud-wan-segment",
          "value": "Production"
        }
      ],
      "action": {
        "association-method": "tag",
        "tag-value-of-key": "cloud-wan-segment"
      }
    },
    {
      "rule-number": 101,
      "condition-logic": "and",
      "conditions": [
        {
          "type": "tag-value",
          "operator": "equals",
          "key": "cloud-wan-segment",
          "value": "Development"
        }
      ],
      "action": {
        "association-method": "tag",
        "tag-value-of-key": "cloud-wan-segment"
      }
    },
    {
      "rule-number": 102,
      "condition-logic": "and",
      "conditions": [
        {
          "type": "tag-value",
          "operator": "equals",
          "key": "cloud-wan-segment",
          "value": "Staging"
        }
      ],
      "action": {
        "association-method": "tag",
        "tag-value-of-key": "cloud-wan-segment"
      }
    },
    {
      "rule-number": 103,
      "condition-logic": "and",
      "conditions": [
        {
          "type": "tag-value",
          "operator": "equals",
          "key": "cloud-wan-segment",
          "value": "Shared"
        }
      ],
      "action": {
        "association-method": "tag",
        "tag-value-of-key": "cloud-wan-segment"
      }
    }
  ]
}

Lastly, other than leveraging the Transit Gateway Route Tables, you could also separate various business intentions via more than one Transit Gateway. When turning to the Cloud WAN architecture, each region could only have a Core Network Edge within a Cloud WAN core network. As you see in the above architecture diagrams, all the components reside in the Cloud WAN core network; therefore, the separation is taken place on the core network (well...please ignore Blackhole 😅).

Anything Else?

Other than the above-mentioned capabilities, there are additional three differences between Transit Gateway and Cloud WAN.

Simpler operations - for operators who are keen to address the following situations, Cloud WAN is what they seek for;
- They are not familiar with managing multi-hierarchies/segments of the network transport.
- They aim to streamline the whole Transit Gateway manipulation, e.g., segmentation across a variety of Transit Gateway Route Tables or cross-region routing exchange across a number of Transit Gateway Peerings.
- The Network-as-Code is a preferred method for routine operation.
More Agility - when you want to filter any route entry, what you would perform on Transit Gateway is not to propagate the whole CIDR of the VPC to Transit Gateway Route Table, a more accurate static route instead; in other words, the control takes place on the route-entry level, the import/export cannot be on the segment (Transit Gateway Route Table) level. In terms of Cloud WAN, it supports both ways; that is other than the prefix filter, all the segments could be used as the objects for import/export.
Higher CapEx - intrinsically, the charge model and pricing are quite close or even identical between Transit Gateway and Cloud WAN except for Core Network Edge. Transit Gateway itself is not charged, but each Core Network Edge charges $0.5/hour. Imagine that you have 4 Core Network Edges across regions in your environment, which means you need to extra pay $1,440/month without any attachments and data processes when compared with Transit Gateway.

For those reasons, if you are familiar with the whole Transit Gateway manipulation and all the modifications involved in your automated framework already then Cloud WAN may not able to catch your eyes because of its cost model.

Conclusion

For mitigating any unwanted requests toward your service origins as close to the sources (clients) as possible, we typically defend them on the edge (CDN) side. When looking at cloud security on the infrastructure level, both Transit Gateway and Cloud WAN adopt the same strategy as well; the isolation happens on the segments layer instead of the network access layer, e.g. Network Firewall/ACL and Security Group.

Why Cloud WAN is more fascinating than Transit Gateway from my viewpoint because it not only hugely simplifies operations but also completes the absences of Transit Gateway. Further, it makes the cloud networking operation close to the traditional MPLS VPN model 😁.

But, I still have to admit that its overall pricing is not so tasty 😅.