DEV Community: Chintan Thakker

Where Is My API Gateway?

Chintan Thakker — Thu, 19 Mar 2020 19:11:14 +0000

As the monolithic application gets broken up into several smaller parts, the infrastructure to support it has to change.

Applications are changing and so is the infrastructure required to support these applications. Earlier, as applications were developed and deployed as monoliths, so was the network infrastructure around it. A monolith needed services from the perimeter proxy like security and monitoring. But as the monolithic application gets broken up into several smaller parts, the infrastructure to support it has to change.

At the center of this change is how the proxy has adapted to providing infrastructure for services which are smaller pieces of the monolithic application. The culmination of services has also created a service mesh pattern where typical application functions that were baked into an application (like traceability library integrated with application) have moved to the proxy.

So far, the community has broken up network infrastructure into ones that serve either North-South traffic needs or East-West traffic needs. The East-West traffic needs are dictated by functionality that used to be a part of an application.

What’s a Gateway?

Gateway is typically associated with a proxy that manages North-South traffic. Typically your traditional perimeter proxy is the gateway proxy. For a monolith, all your network function is implemented at this point of enforcement.

What’s a Mesh?

Mesh is associated with a fleet of proxies that manage East-West traffic. Services (or applications or APIs) call other services (or applications or APIs) to provide a business function. This east-west communication has been the point of choice to implement traceability, observability and also an encryption/decryption point to enforce security.

A Proxy With a Network Function Injection Point

Both a gateway and a service proxy provide a way to inject network function. Can the functionality implemented today, at the edge, be run next to a service? Can it be run for a set of services and not the perimeter?

The boundary between these proxies is gradually diminishing. Consider rate-limiting as a network function typically associated with API gateways. There are several reasons an API may be rate-limited, it could be some business logic to meter service or protect a service from abuse.

Rate limiting network function can be implemented on the proxy typically at the perimeter or may be embedded as a library in the service.

The Perimeter-Proxy/Cloud-Proxy, the Node Side-Car, the Ingress Controller Proxy, and the Pod Side-Car Proxy

A microservice run from a Kubernetes cluster in a cloud environment can potentially go through three network proxies.

One is the perimeter proxy or the cloud provider proxy also called the edge proxy. This is typically created by a LoadBalancer type of service inside the Kubernetes cluster.

Next is the Kubernetes ingress controller that is responsible for getting traffic inside the cluster. This is typically associated with the Ingress resource inside Kubernetes.

Some service mesh implementations like linkerd can run a proxy per node or the node side-car to implement mesh functionality. Encryption/decryption of traffic is functionality implemented at this level.

And then there is the side-car service proxy that is implemented in the pod along with the actual service. This typically primarily runs functions like encryption/decryption.

Every application is different and the choices provided are generally too opinionated or complex.

Opinionated solutions come with assumptions to run network functions at a specific location. This comes at the cost of flexibility and simplicity. A team cannot make decisions on what level of services it needs to run its infrastructure and if often forced to swallow all the complexity even when its needs are minimal.

Melting Boundaries

Proxies are a way to implement network function. There should be a clear path to running a network function at a place that is most convenient for service. Some functions today that are typically run for north-south traffic maybe eventually pushed closer and next to services using a side-car pattern. Network functions associated with API gateway could be a candidate for such a transition.

The underlying network infrastructure choices should not constrain the flexibility available to the operator when adopting microservices, cloud, service mesh and which proxy to use to provide network function.

Enroute Universal Gateway

We built the Enroute Universal Gateway to simplify network function injection points along the traffic path for a service (or an application or an API). The flexibility is critical when breaking up the monolith into microservices, adopting DevOps processes, moving to a cloud or running Kubernetes in the organization.

Enroute Universal Gateway supports advanced rate-limiting and Lua scripting function at perimeter proxy, Kubernetes ingress or a side-car proxy. It can run at Kubernetes ingress without any support of an external database or with a database to run elsewhere. It is completely containerized and can support non-REST protocols such as gRPC. Since it is built on top of Envoy Proxy, it provides the necessary performance for all the above use-cases.

The boundaries between different proxies, load balancers, and application delivery controllers are shrinking and there needs to be a solution to simplify and unite all these technologies. A unified solution will simplify complexity in an organization and provide flexibility to realize the digital transformation objectives of an organization.

Technology — the common denominator for business agility

Chintan Thakker — Sun, 30 Jun 2019 22:26:37 +0000

Technology — the common denominator for business agility

Introduction

Technology has become the lifeblood of businesses today. A lot of business rely on technology to achieve or improve their business objectives.

Technology today is so integral to a business that there is a cost to not evaluating and adopting the current trends. Good investments in technology translate into business savings and agility — both of them extremely critical in today’s ultra-competitive environment.

Lets attempt to understand it using a couple of small case studies (a 10-min crash course on how technology impacts both the top-line and bottom-line for a business)

Left Inc. vs Up Inc.

Imagine I drive for a ride hailing company. Say I have two major options as a driver — I either drive for Left-Inc or for Up-Inc. And, I think they are pretty much the same. So it doesn’t make much of a difference. I don’t really have any specific criteria in picking one over the other.

Now say Left-Inc understands this. They realize that they become more successful when more drivers offer rides through Left Inc. And they decide to tackle it. Say, they incentivize a driver if they provide more rides through Left-Inc on weekends.

Up-Inc feels that they can do better. They come up with a scheme that incentivizes the driver by miles driven over the week and not for the rides provided.

The success of these initiatives depends on how agile Left-Inc is compared to Up-Inc or vice-versa. The pace of business innovation is also set by how fast technology can deliver these services.

Now, say, for this particular case, one of them was running micro-services and another one was running a monolithic application. Whose technology stack will adapt faster?

Syn-pay vs. Ack-pay

Lets think of another example. Think of payment gateways and their customers. Syn-pay and Ack-pay, both are in the payment gateway services, they are fierce competitors. Syn-pay Inc comes up with a program to provide more rebates to customers over Ack-pay. And Ack-pay decides to not charge for a first few thousand transactions every month. Both innovate to address their target segment.

In both the above businesses (ride-hailing and payment-gateways), innovation is a critical aspect. Businesses today have the need to innovate to stay relevant. Additionally, for businesses, technology forms a core and critical component of their business. They are dependent on technology for their competitive advantage. Without the technological challenges tackled and streamlined, they cannot be agile.

Cloud(s) —efficient utilization with faster time to market

Now say, Syn-pay realizes that it’s easier for it to prevent frauds on its platform if it scores a request against a ML-model. But for that, it needs to build and iterate on an ML-model, and that’ll take time. Is there a way to quickly build the model?

For the sake of simplicity, say building a model needs collecting data, cleaning data, ingesting data, building real-time ingestion pipeline, scaling infrastructure to handle the data pipeline, building expertise to manage all this, then building a model, improving its accuracy and scoring it. Without Syn-pay’s investment into this, it will take a lot of time and effort.

Syn-pay realizes that a bunch of the above mentioned tasks can be outsourced. For a small fee, there are services out there that can be readily consumed. Now it can focus on building the model to target its use-case. The rest is taken care of.

Syn-pay uses that cloud provider to work on its data driven initiatives. Syn-pay has just committed to using a service outside of its current infrastructure footprint.

Since Syn-pay has brought down the number of fraudulent transactions, it is saving a lot of money. Syn-pay passes on the savings to its customers. Ack-pay needs to innovate and improve. They need to find a way to stay competitive.

For businesses, apart from not having to manage their IT infrastructure, they will adopt cloud to innovate faster.

Microservices — Shrinking development lead times

Cloud adoption holds the promise to optimize the IT spend. It also provides an opportunity to adopt and experiment with micro-services. The eventual goal of every organization (in addition to delivering growth by aligning resources for their core offering) is to make the organization more agile for quickly delivering IT/operations/applications.

The way forward

Moving to the cloud has its advantages. Using micro-services based approach to redevelop software while moving to the cloud is probably an ideal choice.

However the real world consists of existing infrastructure, people with existing expertise and applications that support and run the business. To get to a cloud native micro-services based software architecture will need careful planning by the business and stakeholders.

There are multiple ways to move forward. It could be as simple as using migration tools provided by major cloud providers to run a lift-n-shift kind of operation. Or, applications can be moved to the cloud while rewriting them as micro-services.

Conclusion

It is quiet clear that cloud adoption has benefits for an organization. At the same time, adopting micro-services also adds a lot of value to an organization’s agility and ability to innovate quickly.

But the center of this change is the people in the organization. While the benefits of these technologies are well known, the ability of an organization to take on this risk and invest resources to move forward will determine how competitive it will stay going forward.

Cloud and micro-services adoption is a journey. The innovation and planning needed to reach the destination won’t be limited to just the IT organization. It is a change that will need active participation from a larger part of organization and at the same time will have an impact across the organization.

If you have feedback on this story, you can reach us by visiting contact page on Enroute Universal Gateway.

Ubiquitous Service Mesh

Chintan Thakker — Wed, 29 May 2019 00:36:55 +0000

Service mesh has become a popular way to deploy, manage and deliver micro-services. The variety of options out there to realize the service mesh shows its popularity. Initiatives like the Service Mesh Interface are an attempt at standardizing the mesh interface. With Google offering istio to define and provide service mesh, a group of other companies including Microsoft have started this initiative to standardize the service mesh.

The visibility and benefits provided by a service mesh are only limited to the micro-services environment. Extending them outside the mesh can provide substantial benefits, especially when it comes to running operations in cloud-native environments.

Here we touch upon some of the sailent features of service mesh, the providers of service mesh on kubernetes, some initiatives around it and conclude with how Enroute Univerasal API gateway enables service mesh architecture for APIs.

Zero trust

Zero trust assumes there is no trust between entities. Or, phrased in a different way, if a process (or micro-service) runs in an environment with other processes, they do not trust each-other. In absence of trust, there has to be a way to establish trust among each other.

Every entity in a service mesh needs an identity. And, it needs an ability to verify that identity. Once the identities are established, it needs a way to specify policy around who can talk to who else.

Zero trust environments provide the identity and policy based segmentation that can be effective against a breach. It can be also be a way to enforce compliance. A breach, typically lands using a vulnerability and moves laterally to spread itself. It also tries to communicate with external services (like Command and Control center) to report about the environment. A zero trust environment prevents this by ensuring the compromised entity cannot talk to unauthorized services. This limits your blast radius.

Visibility

Service meshes provide telemetry to inspect all the aspects of a system. This is critical. Smooth operation of a service mesh needs the administrator to quickly look at the telemetry and identify problems if any.

Imagine an application that invokes several microservices to serve an external facing API. Imagine if there is a problem with one of the microservices. Deep telemetry that provides detailed information about each microservice can help quickly identify the problem. This is a critical aspect of running a service mesh.

Istio: Realization of service mesh by Google/IBM/Lyft

Istio is a popular service mesh and uses the Envoy proxy as a sidecar to realize the critical aspects of the service mesh.

As described above, service mesh also needs an identity provider. You need a way to establish identity, and then enforce it. Citadel in istio, provides identity in form of a X.509 certificate. The enforcement piece (sidecar envoys) use these certificates to ensure communication across pods only happens according to the specified policy.

The sidecar envoys that run alongside services in every pod also report telemetry to help observe these microservices. In addition to regular statistics like latency, throughput etc., these sidecars also facilitate distributed tracing using Jaeger, Zipkin, Opentracing etc.

Linkerd (and Service Mesh Interface by Microsoft and others)

Another popular realization of the service mesh is linkerd. While istio runs Envoy as a sidecar proxy (in every pod), linkerd has its own proxy (written in rust) that runs alongside every node in your kubernetes cluster (Daemonset). It uses an identity provider to get a certificate and can encrypt traffic across nodes to secure communication.

The inline proxy also reports telemetry that can be used to observe micro-services.

The popularity of service mesh can be seen through the efforts towards standardization of the mesh. One such effort at democratizing the service mesh is an initiative by Microsoft — Service Mesh Interface. As of writing this (5/2019), it is in very early phases. It attempts to define kubernetes objects (or API) that standardize functions like access control, traffic characterization, traffic split and metrics. A standard API to control these aspects of the mesh would allow an administrator to have a uniform API to apply policy across a hetrogenous mesh environment.

Extending tracing outside the mesh

Disaster recovery for an application can be achieved by replicating the application across data-centers. A DNS rule can distribute traffic across these replicas. However if any form of L7 routing is desired across these application locations, then fronting it with a proxy is a good idea. For instance, in a lift-n-shift type of scenario, when say (1) a monolithic application in a private data-center is broken up into a cloud-native micro-services on a public cloud (2) traffic distribution is desired across public and private cloud in hybrid use cases.

Now, say for use-case (1), for a specific L7 path, we may build a micro-service and deploy it on the public cloud. One way to test this micro-service would be to replicate traffic for this specific L7 path. Alternatively, instead of replicating traffic, a small percentage of traffic is sent to the new service.

In any of the above use-cases, if Enroute Universal Gateway (or vanilla envoy proxy) is run outside the mesh for L7 functions, it can provide end-to-end tracing information. The advantage of this is the ability to visualize the traffic latency end-to-end, right from the time the DNS sends it over to an edge proxy for L7 routing and processing, all the way to where it is served by a micro-service.

In the trace below, the span labelled edge-yastack is the one generated by the edge proxy. It shows the time consumed from the edge proxy till it hits the istio-ingress gateway and then the latency incurred in each of the steps.

Enroute Universal Gateway is an API gateway for your services or microservices. It can be run in standalone mode, at kubernetes ingress or in form of a mesh.

YAStack is performant envoy, it provides for a drop-in replacement for vanilla envoy to boost performance.

Conclusion

We propose a way to provide deep end-to-end visibility while ensuring API gateway functions, L7 routing for multi-cloud and hybrid-cloud use-cases for applications. As applications get broken up into micro-services, and the complexity to manage them increases, Enroute Universal Gateway provides the necessary flexibility, visibility and mesh extension. They can be run anywhere — in public cloud or a private data center, and do not need over-provisioning like legacy hardware based appliances. The homogenous environment provided by such an architecture simplifies operations.

If you are looking to evaluate Enroute Universal Gateway, you can reach out to us using the contact form here.