DEV Community: kelseyevans

Eliminating Local Resource Constraints for Building Cloud Native Applications

kelseyevans — Wed, 10 Feb 2021 22:44:13 +0000

Is Minikube melting your laptop? Are your local integration tests suffering because you can’t run dependencies on your development machine?

As organizations adopt Kubernetes and cloud native architectures, development teams will often run into resource constraints as their architectures get more complex. Additionally, Kubernetes presents new challenges for configuring local development environments in comparison with legacy monolithic applications.

With external dependencies and complex testing infrastructures, application developers are forced to either make sacrifices on realistic testing or wait long periods of time to get their local environments configured properly.

Setting up development environments for Kubernetes

The processes for developing traditional monolithic web applications has been improved and optimized and improved over many years. However, microservices and cloud native architectures still present challenges for developers looking to configure realistic development environments efficiently.

Some common challenges for developers working on Kubernetes applications include:

Frontend developers can't run Kubernetes on their typically resource-limited development machines. Oftentimes, they will have to buy a more powerful laptop or SSH to a remote server that can run the tools they need
Frontend developers can't run the entire backend stack on their laptops because it’s overly resource intensive. Instead they have to rely on accessing/consuming backend services that are running in a shared remote (staging) environment. This creates challenges with contention on the remote environment (sharing environments can increase latency on service calls etc), challenges with managing state (e.g. two or more developers mutating the same data store), and the environment being out of sync with production (or out of sync with assumptions/expectations)
Backend developers can't run all of the dependencies locally that are required for integration testing. Instead, they often create mocks/stubs/virtualised services that codify (potentially incorrect) assumptions about the real systems
Backend developers have to selectively spin up and spin down local dependencies that can't be run at the same time (due to resource issues) in order to integration test the system piece by piece. It’s impossible to test the entire system running locally, and the piece by piece tests may hide a bigger integration problem that will only be found once code is pushed to production

Solutions to Address Local Resource Constraints

Buy More Powerful Hardware

When resources are the limitation, there’s always the option to buy more powerful hardware. This can get expensive, however, especially when considering upgrading the resources of an entire team. You’ll also have to account for the time lost as developers get used to new machines. As your organization grows your systems will continue to become more complex, and it’s easy to find yourself in need of more powerful hardware yet again.

Migrate to Developing Code Remotely with a Cloud-Based IDE

There are an increasing number of open source and commercial cloud-based IDE products that can be used to effectively negate any local development machine resource constraints. The underlying hardware powering the cloud IDE can be scaled vertically, and the integrated cluster networking allows easy horizontal scaling.

This challenges with this approach often relate to customizability, with limited access to the underlying OS and hardware, and also to cost, which is recurring and correlated with the size of the development team.Ambassador Cloud + Your Local Tools for Kubernetes

Get Infinite-Scale Development Environments with Ambassador Cloud & Telepresence

Ambassador Cloud is powered by Telepresence, an OSS tool focused on improving the developer workflow for single Kubernetes developers. Telepresence uses a smart proxy to create a copy of your service running in the cloud and intercepts it to your local machine. With this model, you can run and develop any service—no matter its size—on your local machine.

This demo shows a Java microservice running in Kubernetes. Thanks to the power of Telepresence, the developer can make changes to their services without having to run all of the dependencies on their local machines. Even better, they can see the changes immediately!

If this is a problem you’ve faced while adopting cloud native technologies, we’d love to hear about your story and how you’ve addressed it. Please drop us a line at @ambassadorlabs on Twitter or join our Slack channel to share your story.

Learn More

Check out Ambassador Cloud powered by Telepresence
Learn more about fast, efficient development of Kubernetes microservices
Get started today

From Monolith to Service Mesh, via a Front Proxy-- Learnings from stories of building the Envoy Proxy

kelseyevans — Fri, 24 Aug 2018 18:53:42 +0000

The concept of a “service mesh” is getting a lot of traction within the microservice and container ecosystems. This technology promise to homogenise internal network communication between services and provide cross-cutting nonfunctional concerns such as observability and fault-tolerance. However, the underlying proxy technology that powers a service mesh can also provide a lot of value at the edge of your systems — the point of ingress — particularly within an API gateway like the open source Kubernetes-native Ambassador gateway.

The State of SOA Networking

In a talk last year by Matt Klein, one of the creators of the Envoy Proxy, he described the state of service-oriented architecture (SOA) and microservice networking in 2013 as “a really big and confusing mess”. Debugging was difficult or impossible, with each application exposing different statistics and logging, and providing no ability to trace how requests were handled throughout the entire services call stack that took part in generating a response. There was also limited visibility into infrastructure components such as hosted load balancers, caches and network topologies.

It’s a lot of pain. I think most companies and most organizations know that SOA [microservices] is kind of the future and that there’s a lot of agility that comes from actually doing it, but on a rubber meets the road kind of day in and day out basis, people are feeling a lot of hurt. That hurt is mostly around debugging.

Maintaining reliability and high-availability of distributed web-based applications was a core challenge for large-scale organisations. Solutions to the challenges frequently included either multiple or partial implementations of retry logic, timeouts, rate limiting and circuit breaking. Many custom and open source solutions used a language-specific (and potentially even framework-specific) solution that meant engineers inadvertently locked themselves into a technology stack “essentially forever”. Klein and his team at Lyft thought there must be a better way.

Ultimately, robust observability and easy debugging are everything. As SOAs become more complicated, it is critical that we provide a common solution to all of these problems or developer productivity grinds to a halt (and the site goes down… often)

Ultimately the Envoy proxy was created to be this better way, and this project was released at open source by Matt and the Lyft team in September 2016.

The Evolution of Envoy

I’ve talked about the core features of Envoy in a previous post that covers another of Matt’s talks, but here I want to touch on the advanced load balancing. The proxy implements “zone aware least request load balancing”, and provides Envoy metrics per zone. As the Buoyant team have stated in their blog post “Beyond Round Robin: Load Balancing for Latency”, performing load balancing at this point in the application/networking stack allows for more advanced algorithms than have typically been seen within SOA networking. Envoy also provides traffic shadowing, which can be used to fork (and clone) traffic to a test cluster, which is proving to be a popular approach for testing microservice-based applications in production.

A core feature offered by Layer 7 (L7) proxies like Envoy is the ability to provide intelligent deployment control by basing routing decisions on application-specific data, such as HTTP headers. This allows a relatively easy implementation of blue/green deployments and canary testing, which also have the benefit of being controllable at near real time speed (in comparison with, say, an approach that uses the deployment mechanism to initialise and decommission VMs or pods to determine what services serve traffic).

Observability, Observability, Observability

Matt states in the talk that observability is by far the most important thing that Envoy provides. Having all service traffic transit through Envoy provides a single place where you can: produce consistent statistics for every hop; create an propagate a stable request identifier (which also required a lightweight application library to fully implement); and provide consistent logging and distributed tracing.

Being built around Envoy the Ambassador API gateway embraces the same principles. Metrics are exposed via the ubiquitous and well-tested StatsD protocol, and Ambassador automatically sends statistics information to a Kubernetes service called statsd-sink using typical StatsD protocol settings, UDP to port 8125. The popular Prometheus open source monitoring system is also supported, and a StatsD exporter can be deployed as a sidecar on each Ambassador pod. More details are provided in the Datawire blog “Monitoring Envoy and Ambassador on Kubernetes with the Prometheus Operator”.

Creating effective dashboards is an art in itself, and Matt shared several screenshots of dashboards that he and his team have created to show Envoy data at Lyft. If you want to explore a real world example of this type of dashboard, Alex Gervais, staff software developer at AppDirect and author of “Evolution of the AppDirect Kubernetes Network Infrastructure”, recently shared the AppDirect team’s Grafana dashboard for Ambassador via the Grafana website.

The Future of Envoy

The best place to learn about the future direction of Envoy is the Envoy documentation itself. In the talk that I’ve covered in this post Matt hinted at several future directions that has since been realised. This includes more rate limiting options (be sure to check both the v1 and v2 APIs), and an open source Go-based rate limit service. The Datawire team have followed suite with a series on how to implement rate limiting on the Ambassador API gateway (effectively an Envoy front proxy), and also released demonstration open source code for a Java rate limiting service.

Undeniably the community has evolved at a fantastic pace since Matt gave the talk. The communities for Envoy, Istio and Ambassador (and several other Envoy-based services) are extremely active and helpful. So, what are you waiting for? Get involved and help steer the future of what are looking to be core components of modern cloud native application architectures. You can join the conversation here!

This post originally appeared on the Ambassador blog by Daniel Bryant.

Using API Gateways to Facilitate Your Transition from Monolith to Microservices

kelseyevans — Fri, 08 Jun 2018 20:22:35 +0000

In my consulting working I bump into a lot of engineering teams that are migrating from a monolithic application to a microservices-based application. “So what?” you may say, “and the sky is blue”, and yes, while I understand that this migration pattern is almost becoming a cliche, there are often aspects of a migration get forgotten. I’m keen to talk about one of these topics today — the role of an edge gateway, or API gateway.

Migrating to Microservices

Typically at the start of a migration the obvious topics are given plenty of attention: domain modelling via defining Domain-Driven Design inspired "bounded contexts", the creation of continuous delivery pipelines, automated infrastructure provisioning, enhanced monitoring and logging, and sprinkling in some shiny new technology (Docker, Kubernetes, and perhaps currently a service mesh or two?). However, the less obvious aspects can cause a lot of pain if they are ignored. A case in point is how to orchestrate the evolution of the system and the migration of the existing user traffic. Although you want to refactor the existing application architecture and potentially bring in some new technology, you do not want to disrupt your end users.

As I wrote in a previous article "Continuous Delivery: How Can an API Gateway Help (or Hinder)", patterns like the "dancing skeleton" can greatly help in proving the end-to-end viability of new applications and infrastructure. However, the vast majority of underlying customer interaction is funneled via a single point within your system — the ingress or edge gateway — and therefore to enable experimentation and evolution of the existing systems, you will need to focus considerable time and effort here.

Every (User) Journey Begins at the Edge

I'm obviously not the first person to talk about the need for an effective edge solution when moving towards a microservices-based application. In fact, in Phil Calcado's proposed extension of Martin Fowler's original Microservices Prerequisites article — Calcado's Microservices Prerequisites — his fifth prerequisite is "easy access to the edge". Phil talks based on his experience that many organisation's first foray into deploying a new microservice alongside their monolith consists of simply exposing the service directly to the internet. This can work well for a single (simple) service, but the approach tends not to scale, and can also force the calling clients to jump through hoops in regards to authorization or aggregation of data.

It is possible to use the existing monolithic application as a gateway, and if you have complex and highly-coupled authorization and authentication code, then this can be the only viable solution until the security components are refactored out into a new module or service. This approach has obvious downsides, including the requirement that you must "update" the monolith with any new routing information (which can involve a full redeploy), and the fact that all traffic must pass through the monolith. This latter issue can be particularly costly if you are deploying your microservices to a separate new fabric or platform (such as Kubernetes), as now any request that comes into your application has to be routed through the old stack before it even touches the new stack.

You may already be using an edge gateway or reverse proxy — for example, NGINX or HAProxy — as these can provide many advantages when working with any type of backend architecture. Features provided typically include transparent routing to multiple backend components, header rewriting, TLS termination etc, and crosscutting concerns regardless of how the requests are ultimately being served. The question to ask in this scenario is whether you want to keep using this gateway for your microservices implementation, and if you do, should it be used in the same way?

From VMs to Containers (via Orchestration)

As I mentioned in the introduction to this article, many engineering teams also make the decision to migrate to new infrastructure at the same time as changing the application architecture. The benefits and challenges for doing this are heavily context-dependent, but I see many teams migrating away from VMs and pure Infrastructure as a Service (IaaS) to containers and Kubernetes.

Assuming that you are deciding to package your shiny new microservices within containers and deploy these into Kubernetes, what challenges do you face in regards to handling traffic at the edge? In essence there are three choices, one of which you have already read about:

Use the existing monolithic application to act as an edge gateway that routes traffic to either the monolith or new services. Any kind of routing logic can be implemented here (because all requests are travelling via the monolith) and calls to authn/authz can be made in process
Deploy and operate an edge gateway in your existing infrastructure that routes traffic based on URIs and headers to either the monolith or new services. Authn and authz is typically done via calling out to the monolith or a refactored security service.
Deploy and operate an edge gateway in your new Kubernetes infrastructure that routes traffic based on URIs and headers to either the monolith or new services. Authn and authz is typically done via calling out to a refactored security service running in Kubernetes.

The choice of where to deploy and operate your edge gateway involves tradeoffs:

Once you have made your choice on how to implement the edge gateway, the next decision you will have make is how to evolve your system. Broadly speaking, you can either try and "strangle" the monolith as-is, or you put the "monolith-in-a-box" and start chipping away here.

Strangling the Monolith

Martin Fowler has written a great article about the principles of the Strangler Application Pattern, and even though the writing is over ten years old, the same guidelines apply when attempting to migrate functionality out from a monolith into smaller services. The pattern at its core describes that functionality should be extracted from the monolith in the form of services that interact with the monolith via RPC or REST-like "seams" or via messaging and events. Over time, functionality (and associated code) within the monolith is retired, which leads to the new microservices "strangling" the existing codebase. The main downside with this pattern is that you will still have to maintain your existing infrastructure alongside any new platform you are deploying your microservices to, for as long as the monolith is still in service.

One of the first companies to talk in-depth about using this pattern with microservices was Groupon, back in 2013, with "I-Tier: Dismantling the Monolith". There are many lessons to be learnt from their work, but we definitely don't need to write a custom NGINX module in 2018, as Groupon originally did with "Grout". Now modern open source API gateways like Ambassador and Traefik exist, which provide this functionality using simple declarative configuration.

Monolith-in-a-Box: Simplifying Continuous Delivery

An increasingly common pattern I am seeing with teams migrating to microservices and deploying onto Kubernetes is what I refer to as a "monolith-in-a-box". I talked about this alongside Nic Jackson when we shared the story of migrating notonthehighstreet.com's monolithic Ruby on Rails application — affectionately referred to as the MonoNOTH — towards a microservice-based architecture back in 2015 at the ContainerSched conference.

In a nutshell, this migration pattern consists of packaging your existing monolithic application within a container and running it like any other new service. If you are implementing a new deployment platform, such as Kubernetes, then you will run the monolith here too. The primary benefits of this pattern is the homogenisation of your continuous delivery pipeline — each application and service may require customised build steps (or a build container) in order to compile and package the code correctly, but after the runtime container has been created, then all of the other steps in the pipeline can use the container abstraction as the deployment artifact.

The ultimate goal of the monolith-in-a-box pattern is to deploy your monolith to your new infrastructure, and gradually move all of your traffic over to this new platform. This allows you to decommission your old infrastructure before completing the full decomposition of the monolith. If you are following this pattern then I would argue that running your edge gateway within Kubernetes makes even more sense, as this is ultimately where all of the traffic will be routed.

Parting Thoughts

When moving from Virtual Machine (VM)-based infrastructure to a cloud native platform like Kubernetes it is well worth investing time in implementing an effective edge/ingress solution to help with the migration. You have multiple options to implement this: using the existing monolithic application as a gateway; deploying or using an edge gateway in your existing infrastructure to route traffic between the current and new services; or deploying an edge gateway within your new Kubernetes platform.

Deploying an edge gateway within Kubernetes can provide more flexibility when implementing migration patterns like the "monolith-in-a-box", and can make the transition towards a fully microservice-based application much more rapid.

This post was originally published on the Ambassador blog by Daniel Bryant

gRPC and the open source Ambassador API Gateway

kelseyevans — Fri, 23 Feb 2018 22:46:29 +0000

gRPC is a high performance RPC protocol built on HTTP/2. We're seeing more and more companies move their APIs from HTTP/JSON to gRPC. gRPC offers numerous benefits over HTTP/JSON:

Performance. gRPC uses HTTP/2, with support for streaming, multiplexing, and more (see the difference in action). In addition, gRPC has native support for protobuf, which is much faster at serialization / deserialization than JSON.
Streaming. Despite its name, gRPC also supports streaming, which opens up a much wider range of use cases.
Code generation of endpoints. gRPC uses code generation from API definitions (.proto files) to stub out your endpoints.

gRPC does have a few downsides, the biggest of which is probably that it's much newer than HTTP/REST, and as such, the maturity of tools and libraries around gRPC isn't anywhere close to HTTP.

One of these gaps is in API Gateways. Many API Gateways don't support gRPC. Fortunately, Ambassador does support gRPC, thanks to the fact that it uses Envoy as its core proxying engine.

API Gateways and gRPC

An API Gateway implements cross-cutting functionality such as authentication, logging, rate limiting, and load balancing. By using an API Gateway with your gRPC APIs, you are able to deploy this functionality outside of your core gRPC service(s). Moreover, Ambassador is able to provide this functionality for both your HTTP and gRPC services.

Ambassador and gRPC

Deploying a gRPC service with Ambassador is straightforward. After installing Ambassador, create an Ambassador mapping for your gRPC service. An example mapping would look like the following:

--------
apiVersion: ambassador/v0
kind: Mapping
name: grpc_mapping
grpc: true
prefix: /helloworld.Greeter/
rewrite: /helloworld.Greeter/
service: grpc-greet

Note that gRPC services are not routed like HTTP services. Instead, gRPC requests include the package and service name. This information is used to route the request to the appropriate service. We set the prefix and rewritefields accordingly based on the .proto file. Also, note that we set grpc: trueto tell Ambassador that the service speaks gRPC.

Deploying a gRPC service

We can deploy a simple Hello, World gRPC service to illustrate all this functionality. Copy-and-paste the full YAML below into a file called helloworld.yaml:

--------
apiVersion: v1
kind: Service
metadata:
  labels:
    service: grpc-greet
  name: grpc-greet
  annotations:
    getambassador.io/config: |
      ---
      apiVersion: ambassador/v0
      kind: Mapping
      name: grpc_mapping
      grpc: true
      prefix: /helloworld.Greeter/
      rewrite: /helloworld.Greeter/
      service: grpc-greet
spec:
  type: ClusterIP
  ports:
  - port: 80
    name: grpc-greet
    targetPort: grpc-api
  selector:
    service: grpc-greet
--------
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: grpc-greet
spec:
  replicas: 1
  template:
    metadata:
      labels:
        service: grpc-greet
    spec:
      containers:
      - name: grpc-greet
        image: enm10k/grpc-hello-world
        ports:
        - name: grpc-api
          containerPort: 9999
        env:
          - name: PORT
            value: "9999"
        command:
          - greeter_server
      restartPolicy: Always

Then, run kubectl apply -f helloworld.yaml.

The Hello World client

In order to test that Ambassador and the service are working properly, we'll need to run a gRPC client. First, get the external IP address of Ambassador. You can do this with kubectl get svc ambassador.

We'll use a Docker image that already contains the appropriate Hello World gRPC client. Type, where $AMBASSADOR_IP is set to the external IP address above, and $AMBASSADOR_PORT is set to 80 (for HTTP-based Ambassador) or 443 (for a TLS-configured Ambassador).

docker run -e ADDRESS=${AMBASSADOR_IP}:${AMBASSADOR_PORT} enm10k/grpc-hello-world greeter_client

You should see something like the following:

$ docker run -e ADDRESS=35.29.51.15:80 enm10k/grpc-hello-world greeter_client
2018/02/02 20:34:35 Greeting: Hello world

Summary

Ambassador makes it easy to publish gRPC services to your consumers, thanks to Envoy's robust gRPC support. By publishing your services through Ambassador, you're able to add authentication, rate limiting, and other functionality to your gRPC services.

If you're interested in using Ambassador with gRPC, join our Gitter chat or visit https://www.getambassador.io.

Building Ambassador, an Open Source API Gateway on Kubernetes and Envoy

kelseyevans — Thu, 22 Feb 2018 22:12:09 +0000

API Gateways are a popular pattern for exposing your service endpoints to the consumer. At Datawire, we wanted to expose a number of our cloud services to our end users via an API Gateway. All of our cloud services run in Kubernetes, so we wanted to deploy the API gateway on Kubernetes as well. And finally, we wanted something that was open source.

We took a look at Tyk, Kong, and a few other open source API Gateways, and found that they had a common architecture — a persistent data store (e.g., Cassandra, Redis, MongoDB), a proxy server to do the actual traffic management, and REST APIs for configuration. While all of these seemed like they would work, we asked ourselves if there was a simpler, more Kubernetes-native approach.

Our API Gateway wish list

We scribbled on a whiteboard of requirements:

Reliability, availability, scalability. Duh.
Declarative configuration. We had committed to the declarative model of configuration (here's an article on the contrast), and didn't like the idea of mixing imperative configuration (via REST) and declarative configuration for our operational infrastructure.
Easy introspection. When something didn't work, we wanted to be able to introspect the gateway to figure out what didn't work.
Easy to use.
Authentication.
Performance.
All the features you need for a modern distributed application, e.g., rate limiting, circuit breaking, gRPC, observability, and so forth.

We realized that Kubernetes gave us the reliability, availability, and scalability. And we knew that the Envoy Proxy gave us the performance and features we wanted. So we asked ourselves if we could just marry Envoy and Kubernetes, and then fill in the remaining gaps.

Ambassador = Envoy + Kubernetes

We started to writing some prototype code, shared this with the Kubernetes community, and iterated on the feedback. We ended up with the open source Ambassador API Gateway. At its core, Ambassador has one basic function: it watches for configuration changes to your Kubernetes manifests, and then safely passes the necessary configuration changes to Envoy. All the L7 networking is performed directly by Envoy, and Kubernetes takes care of reliability, availability, and scalability.

To that core function, we’ve added a few other core features: introspection via a diagnostics UI (see above), and a single Docker image that integrates Envoy and all the necessary bits to get it running in production (as of 0.23, it’s a 113MB Alpine Linux based image).

The combination of Envoy and Kubernetes has enabled Ambassador to be production ready in a short period of time.

Ambassador uses Kubernetes for persistence, so there is no need to run, scale, or maintain a database. (And as such, we don't need to test or tune database queries.)
Scaling Ambassador is done by Kubernetes, so you can use a horizontal pod autoscaler or just add replicas as needed.
Ambassador uses Kubernetes liveness and readiness probes, so Kubernetes automatically restarts Ambassador if it detects a problem.
All of the actual L7 routing is done by Envoy, so our performance is the same as Envoy. (In fact, you could actually delete the Ambassador code from the pod, and your Envoy instance would keep on routing traffic.)

And thanks to Envoy's extremely robust feature set, we've been able to add features such as rate limiting, gRPC support, web sockets, and more in a short period of time.

What about Ingress?

We considered making Ambassador an ingress controller. In this model, the user would define Ingress resources in Kubernetes, which would then be processed by the Ambassador ingress controller. After investigating this approach a bit further, we decided against this method because Ingress resources have a very limited set of features. In particular, Ingress resources can only define basic HTTP routing rules. Many of the features we wanted to use in Envoy (e.g., gRPC, timeouts,rate limiting, CORS support, routing based on HTTP method, etc.) are not possible to express with Ingress resources.

Using Ambassador

Our goal with Ambassador is to make it idiomatic with Kubernetes. Installing Ambassador is a matter of creating a Kubernetes deployment (e.g., kubectl apply -f [https://www.getambassador.io/yaml/ambassador/ambassador-rbac.yaml](https://getambassador.io/yaml/ambassador/ambassador-rbac.yaml) if you're using RBAC) and creating a Kubernetes service that points to the deployment.

Once that's done, configuration is done via Kubernetes annotations. One of the advantages of this approach is that the actual metadata about how your service is published is all in one place — your Kubernetes service object.

Ambassador supports a rich set of annotations that map to various features of Envoy. The weight annotation used above will route 10% of incoming to this particular version of the service. Other useful annotations include method, which lets you define the HTTP method for mapping; grpc, for gRPC-based services; and tls which tells Ambassador to contact the service over TLS. The full list is in the configuration documentation.

What's next for Ambassador

At KubeCon NA 2017, one of the themes was how do applications best take advantage of all that Kubernetes has to offer. Kubernetes is evolving at an incredibly fast rate, and introducing new APIs and abstractions. While Ingress resources didn't fit our use cases, we're exploring Custom Resources and the Operator pattern as a potentially interesting place to take Ambassador as this would address the limitations we encountered with ingress. More generally, we're interested in understanding how people would like to build cloud-native applications with Ambassador.

We're also adding more features as more and more users deploy Ambassador in production such as adding arbitrary request headers, single namespace support, WebSockets, and more. Finally, some of our users are deploying Ambassador with Istio. In this set up, Ambassador is configured to handle so-called "north/south" traffic and Istio handles "east/west" traffic.

If you're interested, we'd love to hear from you and contribute. Join our Gitter chat, open a GitHub issue, or just try Ambassador (it's just one line to try it locally with Docker!).

Deploying Java Apps with Kubernetes and the Ambassador API Gateway

kelseyevans — Tue, 20 Feb 2018 18:57:03 +0000

In this article you’ll learn how to deploy three simple Java services into Kubernetes (running locally via the new Docker for Mac/Windows integration), and expose the frontend service to end-users via the Kubernetes-native Ambassador API Gateway. So, grab your caffeinated beverage of choice and get comfy in front of your terminal!

A Quick Recap: Architecture and Deployment

In October last year I extended my simple Java microservice-based "Docker Java Shopping" container deployment demonstration with Kubernetes support. If you found the time to complete the tutorial you would have packaged three simple Java services — the shopfront and stockmanager Spring Boot services, and the product catalogue Java EE DropWizard service — within Docker images, and deployed the resulting containers into a local minikube-powered Kubernetes cluster. I also showed you how to open the shopfront service to end-users by mapping and exposing a Kubernetes cluster port using a NodePort Service. Although this was functional for the demonstration, many of you asked how you could deploy the application behind an API Gateway. This is a great question, and accordingly I was keen to add another article in this tutorial series with the goal of deploying the "Docker Java Shopping" Java application behind the open source Kubernetes-native Ambassador API Gateway.

Figure 1. “Docker Java Shopping” application deployed with Ambassador API Gateway

Quick Aside: Why Use an API Gateway?

I'm confident that many of you will have used (or at least bumped into) the concept of an API Gateway before. Chris Richardson has written a good overview of the details at microservices.io, and the team behind the creation of the Ambassador API Gateway, Datawire, have also talked about the benefits of using a Kubernetes-native API Gateway. In short, an API Gateway allows you to centralise a lot of the cross-cutting concerns for your application, such as load balancing, security and rate-limiting. Running a Kubernetes-native API Gateway also allows you to offload several of the operational issues associated with deploying and maintaining a gateway — such as implementing resilience and scalability — to Kubernetes itself.

There are many API Gateway choices for Java developers, such as the open source Netflix's Zuul, Spring Cloud Gateway, and Mashape's Kong; there are cloud vendors' implementations (such as Amazon's API Gateway); and of course the traditional favourites of NGINX and HAProxy; and finally, also the more modern variants like Traefik. Choosing the best API Gateway for your use case can involve a lot of work — this is a critical piece of your infrastructure, and it will touch every bit of traffic coming into your application. As with any critical tech choice, there are many tradeoffs to be considered. In particular, watch out for potential high-coupling points — for example, I've seen the ability to dynamically deploy "Filters" (Groovy scripts) into Netflix's Zuul enables business logic to become spread (coupled) between the service and the gateway — and also the need to deploy complicated datastores as the end-user traffic increases — for example, Kong requires a Cassandra cluster or Postgres installation to scale horizontally.

For the sake of simplicity in this article I'm going to use the open source Kubernetes-native Ambassador API Gateway. I like Ambassador because the simplicity of the implementation reduces the ability to accidentally couple any business logic to it, and the fact that I can specify service routing via a declarative approach (which I use for all of my other Kubernetes config) feels more "cloud native" — I can also store the routes easily in version control, and send this down the CI/CD build pipeline with all the other code changes.

Getting Started: NodePorts and LoadBalancers 101

First, ensure you are starting with a fresh (empty) Kubernetes cluster. Because I like to embrace my inner-hipster every once in a while, I am going to run this demonstration using the new Kubernetes integration within Docker for Mac. If you want to follow along you will need to ensure that you have installed the Edge version of Docker for Mac or Docker for Windows, and also enabled Kubernetes support by following the instructions within the Docker Kubernetes documentation.

Next clone my "Docker Java Shopfront" GitHub repository. If you want to explore the directory structure and learn more about each of the three services that make up the application, then I recommend having a look at the previous article in this series or the associated mini-book "Containerizing Continuous Delivery in Java" that started all of this. When the repo has been successfully cloned you can navigate into the kubernetes directory. If you are following along with the tutorial then you will be making modifications within this directory, and so you are welcome to fork your own copy of the repo and create a branch that you can push your work to. I don't recommend skipping ahead (or cheating), but the kubernetes-ambassador directory contains the complete solution, in case you want to check your work!

$ git clone git@github.com:danielbryantuk/oreilly-docker-java-shopping.git
$ cd oreilly-docker-java-shopping/kubernetes
(master) kubernetes $ ls -lsa
total 24
0 drwxr-xr-x   5 danielbryant  staff  160  5 Feb 18:18 .
0 drwxr-xr-x  18 danielbryant  staff  576  5 Feb 18:17 ..
8 -rw-r--r--   1 danielbryant  staff  710  5 Feb 18:22 productcatalogue-service.yaml
8 -rw-r--r--   1 danielbryant  staff  658  5 Feb 18:11 shopfront-service.yaml
8 -rw-r--r--   1 danielbryant  staff  677  5 Feb 18:22 stockmanager-service.yaml

If you open up the shopfront-service.yaml in your editor/IDE of choice, you will see that I am exposing the shopfront service as a NodePort accessible via TCP port 8010. This means that the service can be accessed via port 8010 on any of the cluster node IPs that are made public (and not blocked by a firewall).

---
apiVersion: v1
kind: Service
metadata:
 name: shopfront
 labels:
 app: shopfront
spec:
 type: NodePort
 selector:
 app: shopfront
 ports:
 — protocol: TCP
 port: 8010
 name: http

When running this service via minikube, NodePort allows you to access the service via the cluster external IP. When running the service via Docker, NodePort allows you to access the service via localhost and the Kubernetes allocated port. Assuming that Docker for Mac or Windows has been configured to run Kubernetes successfully you can now deploy this service:

(master) kubernetes $ kubectl apply -f shopfront-service.yaml
service "shopfront" created
replicationcontroller "shopfront" created
(master) kubernetes $
(master) kubernetes $ kubectl get services
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP          19h
shopfront    NodePort    10.110.74.43   <none>        8010:31497/TCP   0s

You can see the shopfront service has been created, and although there is no external-ip listed, you can see that the port specified in the stockmanager-service.yaml (8010) has been mapped to port 31497 (your port number may differ here). If you are using Docker for Mac or Windows you can now curl data from localhost (as the Docker app works some magic behind the scenes), and if you are using minikube you can get the cluster IP address by typing minikube ip in your terminal.

Assuming you are using Docker, and that you have only deployed the single shopfront service you should see this response from a curl using the port number you can see from the kubectl get svc command (31497 for me):

(master) kubernetes $ curl -v localhost:31497
* Rebuilt URL to: localhost:31497/
* Trying ::1…
* TCP_NODELAY set
* Connected to localhost (::1) port 31497 (#0)
> GET / HTTP/1.1
> Host: localhost:31497
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 500
< X-Application-Context: application:8010
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Tue, 06 Feb 2018 17:20:19 GMT
< Connection: close
<
* Closing connection 0
{“timestamp”:1517937619690,”status”:500,”error”:”Internal Server Error”,”exception”:”org.springframework.web.client.ResourceAccessException”,”message”:”I/O error on GET request for \”http://productcatalogue:8020/products\": productcatalogue; nested exception is java.net.UnknownHostException: productcatalogue”,”path”:”/”}

You’ll notice that you are getting an HTTP 500 error response with this curl, and this is to be expected as you haven’t deployed all of the supporting services yet. However, before you deploy the rest of the services you’ll want to change the NodePort configuration to ClusterIP for all of your services. This means that each services will only be accessible other the network within the cluster. You could of course use a firewall to restrict a service exposed by NodePort, but by using ClusterIP with our local development environment you are forced not to cheat to access our services via anything other than the API gateway we will deploy.

Open shopfront-service.yaml in your editor, and change the NodePort to ClusterIP. You can see the relevant part of the file contents below:

---
apiVersion: v1
kind: Service
metadata:
 name: shopfront
 labels:
 app: shopfront
spec:
 type: ClusterIP
 selector:
 app: shopfront
 ports:
 — protocol: TCP
 port: 8010
 name: http

Now you can modify the services contained with the productcatalogue-service.yaml and stockmanager-service.yaml files to also be ClusterIP.

You can also now delete the existing shopfront service, ready for the deployment of the full stack in the next section of the tutorial.

(master *) kubernetes $ kubectl delete -f shopfront-service.yaml
service “shopfront” deleted
replicationcontroller “shopfront” deleted

Deploying the Full Stack

With a once again empty Kubernetes cluster, you can now deploy the full three-service stack and the get the associated Kubernetes information on each service:

(master *) kubernetes $ kubectl apply -f .
service "productcatalogue" created
replicationcontroller "productcatalogue" created
service "shopfront" created
replicationcontroller "shopfront" created
service "stockmanager" created
replicationcontroller "stockmanager" created
(master *) kubernetes $
(master *) kubernetes $ kubectl get services
NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
kubernetes         ClusterIP   10.96.0.1       <none>        443/TCP    20h
productcatalogue   ClusterIP   10.106.8.35     <none>        8020/TCP   1s
shopfront          ClusterIP   10.98.189.230   <none>        8010/TCP   1s
stockmanager       ClusterIP   10.96.207.245   <none>        8030/TCP   1s

You can see that the port that was declared in the service is available as specified (i.e. 8010, 8020, 8030) — each pod running gets its own cluster IP and associated port range (i.e. each pods gets its own “network namespace”). We can’t access this port outside of the cluster (like we can with NodePort), but within the cluster everything works as expected.

You can also see that using ClusterIP does not expose the service externally by trying to curl the endpoint (this time you should receive a “connection refused”):

(master *) kubernetes $ curl -v localhost:8010
* Rebuilt URL to: localhost:8010/
* Trying ::1…
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 8010 failed: Connection refused
* Trying 127.0.0.1…
* TCP_NODELAY set
* Connection failed
* connect to 127.0.0.1 port 8010 failed: Connection refused
* Failed to connect to localhost port 8010: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 8010: Connection refused

Deploying the Ambassador API Gateway

Now is the time to deploy the Ambassador API gateway in order to expose your shopfront service to end-users. The other two services can remain private within the cluster, as they are supporting services, and don’t have to be exposed publicly.

First, create a LoadBalancer service that uses Kubernetes annotations to route requests from outside the cluster to the appropriate services. Save the following content within a new file named ambassador-service.yaml. Note the getambassador.io/config annotation. You can use Kubernetes annotations to attach arbitrary non-identifying metadata to objects, and clients such as Ambassador can retrieve this metadata. Can you figure out what this annotation is doing?

The Ambassador annotation is key to how the gateway works — how it routes “ingress” traffic from outside the cluster (e.g. an end-user request) to services within the cluster. Let’s break this down:

“getambassador.io/config: |” — specifies that this annotation is for Ambassador
“ — -” — simply declares how much you love YAML!
“ apiVersion: ambassador/v0” — specifies the Ambassador API/schema version
“ kind: Mapping” — specifies that you are creating a “mapping” (routing) configuration
“ name: shopfront” —is the name for this mapping (which will show up in the debug UI)
“ prefix: /shopfront/” — is the external prefix of the URI that you want to route internally
“ service: shopfront:8010” — is the Kubernetes service (and port) you want to route to

In a nutshell, this annotation states that any request to the external IP of the LoadBalancer service (which will be “localhost” in your Docker for Mac/Windows example) with the prefix /shopfront/ will be routed to the Kubernetes shopfront service running on the (ClusterIP) port 8010. In your example, when you enter http://localhost/shopfront/ in your web browser you should see the UI provided by the shopfront service. Hopefully this all makes sense, but if it doesn’t then please visit the Ambassador Gitter and ask any questions, or ping me on twitter!

With your newfound understanding of Ambassador routing (and world domination of all API Gateways merely a few steps away), you can deploy the Ambassador service:

(master *) kubernetes $ kubectl apply -f ambassador-service.yaml
service “ambassador” created

You will also need to deploy the Ambassador Admin service (and associated pods/containers) that are responsible for the heavy-lifting associated with the routing. It's worth noting that the routing is conducted by a "sidecar" proxy, which in this case is the Envoy proxy. Envoy is responsible for all of the production network traffic within Lyft, and it's creator, Matt Klein, has written lots of very interesting content about the details. You may have also heard about the emerging "service mesh" technologies, and the popular Istio project also uses Envoy.

Anyway, back to the tutorial! You can find a pre-prepared Kubernetes config file for Ambassador Admin on the getambassador.io website (for this demo you will be using the "no RBAC" version of the service, but you can also find an RBAC-enabled version of the config file if you are running a Kubernetes cluster with Role-Based Access Control (RBAC) enabled. You can download a copy of the config file and look at it before applying, or you apply the service directly via the Interwebs:

(master *) kubernetes $ kubectl apply -f https://getambassador.io/yaml/ambassador/ambassador-no-rbac.yaml
service “ambassador-admin” created
deployment “ambassador” created

If you issue a kubectl get svc you can see that your Ambassador LoadBalancer and Ambassador Admin services have been deployed successfully:

(master *) kubernetes $ kubectl get svc
NAME               TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
ambassador         LoadBalancer   10.102.81.42    <pending>     80:31053/TCP     5m
ambassador-admin   NodePort       10.105.58.255   <none>        8877:31516/TCP   1m
kubernetes         ClusterIP      10.96.0.1       <none>        443/TCP          20h
productcatalogue   ClusterIP      10.106.8.35     <none>        8020/TCP         22m
shopfront          ClusterIP      10.98.189.230   <none>        8010/TCP         22m
stockmanager       ClusterIP      10.96.207.245   <none>        8030/TCP         22m

You will notice on the ambassador service that the external-ip is listed as and this is a known bug with Docker for Mac/Windows. You can still access a LoadBalancer service via localhost — although you may need to wait a minute or two while everything deploys successfully behind the scenes.

Let’s try and access the shopfront this now using the `/shopfront/ route you configured previously within the Ambassador annotations. You can curl localhost/shopfront/ (with no need to specify a port, as you configured the Ambassador LoadBalancer service to listen on port 80):

{% gist https://gist.github.com/kelseyevans/1ad64d89409c1deeb5ee985b7f30a1aa %}

That’s it! You are now accessing the shopfront service that is hidden away in the Kubernete cluster via Ambassador. You can also visit the shopfront UI via your browser, and this provides a much more friendly view!

Bonus: Ambassador Diagnostics

If you want to look at the Ambassador Diagnostic UI then you can use port-forwarding. I’ll explain more about how to use this in a future post, but for the moment you can have a look around by yourself. First you will need to find the name of an ambassador pod:

{% gist https://gist.github.com/kelseyevans/a8fd8d73dcbc97191ec71b55514b7d90 %}

Here I’ll pick ambassador-6d9f98bc6c-5sppl. You can now port-forward from your local network adapter to inside the cluster and expose the Ambassador Diagnostic UI that is runing on port 8877.

(master *) kubernetes $ kubectl port-forward ambassador-6d9f98bc6c-5sppl 8877:8877

You can now visit http://localhost:8877/ambassador/v0/diag in your browser and have a look around!

When you are finished you can exit the port-forward via ctrl-c. You can also delete all of the services you have deployed into your Kubernetes cluster by issuing a kubectl delete -f . within the kubernetes directory. You will also need to delete the ambassador-admin service you have deployed.

(master *) kubernetes $ kubectl delete -f . service "ambassador" deleted service "productcatalogue" deleted replicationcontroller "productcatalogue" deleted service "shopfront-canary" deleted replicationcontroller "shopfront-canary" deleted service "shopfront" deleted replicationcontroller "shopfront" deleted service "stockmanager" deleted replicationcontroller "stockmanager" deleted (master *) kubernetes $ (master *) kubernetes $ kubectl delete -f https://getambassador.io/yaml/ambassador/ambassador-no-rbac.yaml service "ambassador-admin" deleted deployment "ambassador" deleted

What's Next?

I'm planning on creating another article soon that discusses how to canary launch/test a service, as Ambassador makes this very easy. Other topics I'm keen to explore is integrating all of this into a CD pipeline, and also exploring how best to set up a local development workflow. Closely related to this, I'm also keen to look into debugging Java applications deployed via Kubernetes.

You can also read more details on Ambassador itself via the docs, including adding auth/security, gRPC support, and TLS termination.

This article originally appeared on the Ambassador blog written by Daniel Bryant.

In search of an effective developer experience with Kubernetes

kelseyevans — Thu, 15 Feb 2018 17:14:06 +0000

At Datawire we are helping many organisations with deploying applications to Kubernetes. Often our most important input is working closely alongside development teams helping them to build effective continuous integration and continuous delivery (CI/CD) pipelines. This is primarily because creating an effective developer workflow on Kubernetes can be challenging -- the ecosystem is still evolving, and not all the platform components are plug-and-play -- and also because many engineering teams fail to realise that in order to "close the loop" on business ideas and hypotheses, you also need to instrument applications for observability. We often argue that the first deployment of an application into production through the pipeline is only the start of the continuous delivery process, not the end as some think.

All of us are creating software to support the delivery of value to our customers and to the business, and therefore the "developer experience" (DevEx) -- from idea generation to running (and observing) in production -- must be fast, reliable and provide good feedback. As we have helped our customer create effective continuous delivery pipelines for Kubernetes (and the associated workflows), we have seen several patterns emerge. We are keen to share our observations on these patterns, and also explain how we have captured some of best patterns within a collection of open source tools for deploying applications to Kubernetes.

From idea to (observable) value

Everything we do as engineers begins with an idea. From this idea a hypothesis emerges -- for example, modifying the layout of a web form will improve conversion, or improving the site's p99 latency will result in more revenue -- and we can extract appropriate metrics for observation -- conversion and page load latency in our example.

Building and packaging for Kubernetes

Once we have agreed our hypothesis and metrics we can then begin to write code and package this ready for deployment on Kubernetes. We have created the open source Forge framework to assist with the entire development process, from automatically creating and managing boilerplate Kubernetes configuration, to allowing us to parameterise runtime properties and resources that can facilitate deploying applications to Kubernetes with a single CLI instruction.

If we are working on a hypothesis that requires "exploration" -- for example, refactoring existing functionality, or solving a technical integration issue -- we often whiteboard ideas and begin coding using techniques like Test-Driven Development (TDD), taking care to design observability (business metrics, monitoring and logging etc) in as we go. If we are working on a hypothesis that requires "experimentation" -- for example, a new business feature -- we typically define Behaviour-Driven Development (BDD)-style tests in order to help keep us focused towards building functionality "outside-in".

Removing friction from the code-deploy-experiment cycle

We attempt to develop within environments that are as production-like as possible, and so frequently we build local services that interact with a more-complete remote Kubernetes cluster deployment. We have created the open source tool Telepresence that allows us to execute and debug a local service that acts as if it is part of the remote environment (effectively two-way proxying from our local development machine to the remote Kubernetes cluster).

We like to "release early, and release often", and so favour running tests in production using canary and dark launches. This way we can expose new functionality to small amounts of real users, and observe their behaviour in relation to our hypothesis. As with any deployment to a production environment, there is obvious a certain amount of risk, and we mitigate this by introducing alerting and automated rollbacks when serious issues are detected. We have created the open source API gateway Ambassador for this purpose.

Smart routing and monitoring with Ambassador API Gateway

Ambassador is built using the popular Envoy proxy that emerged from the work by Matt Klein and his team at Lyft. Ambassador allows "smart routing" of traffic when deploying applications to Kubernetes, and the underlying technology has been proven to operate at-scale within Lyft. Once an application is receiving production traffic we can observe metrics based on our earlier hypothesis. We typically use Prometheus to collect data and Grafana to display the results via dashboards. We've created the open source prometheus-ambassador project to enable the easy export of metrics from Ambassador, for example latency and the number of 5xx HTTP responses codes returned.

And the cycle begins again

Once we have analysed our metrics the development cycle can begin again, either iterating on our existing solution and running additional canary experiments, or if we have proven (or disproven) our original hypothesis we can generate another new idea and hypothesis.

This article originally appeared as part of the Code Faster Guides on Datawire.io written by Daniel Bryant.

Why your development workflow is so important for microservices

kelseyevans — Thu, 25 Jan 2018 20:10:36 +0000

Your development workflow is the process by which your organization develops software. A typical development workflow starts with product definition, and then moves through development, testing, release, and production stages.

The stability vs velocity tradeoff

Organizations tune this workflow for their given business needs and application. Typically, this involves optimizing the workflow to provide the right balance of stability versus velocity. As the application becomes more popular, insuring that updates don't negatively impact users becomes more important. More stringent release criteria, better testing, and development reviews are typical strategies that improve stability. Yet these strategies aren't free, as they reduce velocity.

(Haven't you ever said "we used to ship software so much faster, and now it's slowed down even though we have twice as many engineers?")

Scaling your development workflow

The problem with the development workflow is that no amount of optimization can overcome the fact that there is no single development workflow that works for every part of the application.

The reality is that some parts of your application demand stability, while other parts of your application require velocity. What you really need is multiple workflows that work for different parts of your application.

Microservices

Microservices is distributed development workflow, enabled by splitting your application up into smaller services. By splitting your application into smaller components, you're able to run independent development workflows for each of your services.

You want to run a prototyping workflow for early feature development. As your service matures, you'll want a workflow that supports rapid updates in production. And as it becomes a mission critical service that other services or users really depend on, you'll need a workflow that insures rock-solid stability.

Building a workflow is both easy and hard

The challenge of building a microservices workflow is that you need a standard set of tools and processes that support all these different modes of development. You don't want one set of tools for prototyping, and another set of tools and workflow for production.

In addition, a microservices architecture, the development teams are typically responsible for a service and not just the code. This implies that the development teams need operational skills and capabilities, e.g., monitoring and deployment.

Luckily, Kubernetes has become a de facto standard for running cloud native applications. The Kubernetes ecosystem provides a robust set of operational infrastructure with which to run microservices. So you don't need to start from scratch.

Open Source Tools

The following open source tools may be helpful for optimizing your development workflow on Kubernetes:

What do you think?

What does your development workflow look like? What tools do your developers use, and what pain points do they face?

Monitoring Envoy and Ambassador on Kubernetes with the Prometheus Operator

kelseyevans — Thu, 18 Jan 2018 21:09:25 +0000

In the Kubernetes ecosystem, one of the emerging themes is how applications can best take advantage of the various capabilities of Kubernetes. The Kubernetes community has also introduced new concepts such as Custom Resources to make it easier to build Kubernetes-native software.

In late 2016, CoreOS introduced the Operator pattern and released the Prometheus Operator as a working example of the pattern. The Prometheus Operator automatically creates and manages Prometheus monitoring instances.

The operator model is especially powerful for cloud-native organizations deploying multiple services. In this model, each team can deploy their own Prometheus instance as necessary, instead of relying on a central SRE team to implement monitoring.

Envoy, Ambassador, and Prometheus

In this tutorial, we'll show how the Prometheus Operator can be used to monitor an Envoy proxy deployed at the edge. Envoy is an open source L7 proxy. One of the (many) reasons for Envoy's growing popularity is its emphasis on observability. Envoy uses statsd as its output format.

Instead of using Envoy directly, we'll use Ambassador. Ambassador is a Kubernetes-native API Gateway built on Envoy. Similar to the Prometheus Operator, Ambassador configures and manages Envoy instances in Kubernetes, so that the end user doesn't need to do that work directly.

Prerequisites

This tutorial assumes you're running Kubernetes 1.8 or later, with RBAC enabled.

Note: If you're running on Google Kubernetes Engine, you'll need to grant cluster-admin privileges to the account that will be installing Prometheus and Ambassador. You can do this with the commands below:

$ gcloud info | grep Account
Account: [username@example.org]
$ kubectl create clusterrolebinding my-cluster-admin-binding --clusterrole=cluster-admin --user=username@example.org

Deploy the Prometheus Operator

The Prometheus Operator is configured as a Kubernetes deployment. We'll first deploy the Prometheus operator.

kubectl apply -f prom-operator.yaml

We'll also want to create an additional ServiceAccounts for the actual Prometheus instances.

kubectl apply -f prom-rbac.yaml

The Operator functions as your virtual SRE. At all times, the Prometheus operator insures that you have a set of Prometheus servers running with the appropriate configuration.

Deploy Ambassador

Ambassador also functions as your virtual SRE. At all times, Ambassador insures that you have a set of Envoy proxies running the appropriate configuration.

We're going to deploy Ambassador into Kubernetes. On each Ambassador pod, we'll also deploy an additional container that runs the Prometheus statsd exporter. The exporter will collect the statsd metrics emitted by Envoy over UDP, and proxy them to Prometheus over TCP in Prometheus metrics format.

kubectl apply -f ambassador-rbac.yaml

Ambassador is typically deployed as an API Gateway at the edge of your network. We'll deploy a service to map to the Ambassador deployment. Note: if you're not on AWS or GKE, you'll need to update the service below to be a NodePort instead of a LoadBalancer.

kubectl apply -f ambassador.yaml

You should now have a working Ambassador and StatsD/Prometheus exporter that is accessible from outside your cluster.

Configure Prometheus

We now have Ambassador/Envoy running, along with the Prometheus Operator. How do we hook this all together? Logically, all the metrics data flows from Envoy to Prometheus in the following way:

So far, we've deployed Envoy and the StatsD exporter, so now it's time to deploy the other components of this flow.

We'll first create a Kubernetes service that points to the StatsD exporter. We'll then create a ServiceMonitor that tells Prometheus to add the service as a target.

kubectl apply -f statsd-sink-svc.yaml

Next, we need to tell the Prometheus Operator to create a Prometheus cluster for us. The Prometheus cluster is configured to collect data from any ServiceMonitor with the ambassador:monitoring label.

kubectl apply -f prometheus.yaml

Finally, we can create a service to expose Prometheus to the rest of the world. Again, if you're not on AWS or GKE, you'll want to use a NodePort instead.

kubectl apply -f prom-svc.yaml

Testing

We've now configured Prometheus to monitor Envoy, so now let's test this out. Get the external IP address for Prometheus.

$ kubectl get services
NAME                  CLUSTER-IP      EXTERNAL-IP      PORT(S)          AGE
ambassador            10.11.255.93    35.221.115.102   80:32079/TCP     3h
ambassador-admin      10.11.246.117   <nodes>          8877:30366/TCP   3h
ambassador-monitor    None            <none>           9102/TCP         3h
kubernetes            10.11.240.1     <none>           443/TCP          3h
prometheus            10.11.254.180   35.191.39.173    9090:32134/TCP   3h
prometheus-operated   None            <none>           9090/TCP         3h

In the example above, this is 35.191.39.173. Now, go to http://$PROM_IP:9090 to see the Prometheus UI. You should see a number of metrics automatically populate in Prometheus.

Troubleshooting

If the above doesn't work, there are a few things to investigate:

Make sure all your pods are running (kubectl get pods)
Check the logs on the Prometheus cluster (kubectl logs $PROM_POD prometheus)
Check Ambassador diagnostics to verify Ambassador is working correctly

Get a service running in Envoy

The metrics so far haven't been very interesting, since we haven't routed any traffic through Envoy. We'll use Ambassador to set up a route from Envoy to the httpbin service. Ambassador is configured using Kubernetes annotations, so we'll do that here.

kubectl apply -f httpbin.yaml

Now, if we get the external IP address of Ambassador, we can route requests through Ambassador to the httpbin service:

$ kubectl get services
NAME                  CLUSTER-IP      EXTERNAL-IP      PORT(S)          AGE
ambassador            10.11.255.93    35.221.115.102   80:32079/TCP     3h
ambassador-admin      10.11.246.117   <nodes>          8877:30366/TCP   3h
ambassador-monitor    None            <none>           9102/TCP         3h
kubernetes            10.11.240.1     <none>           443/TCP          3h
prometheus            10.11.254.180   35.191.39.173    9090:32134/TCP   3h
prometheus-operated   None            <none>           9090/TCP         3h

$ curl http://35.221.115.102/httpbin/ip
{
  "origin": "35.214.10.110"
}

Run a curl command a few times, as shown above. Going back to the Prometheus dashboard, you'll see that a bevy of new metrics that contain httpbin have appeared. Pick any of these metrics to explore further. For more information on Envoy stats, Matt Klein has written a detailed overview of Envoy's stats architecture.

Conclusion

Microservices, as you know, are distributed systems. The key to scaling distributed systems is creating loose coupling between each of the components. In a microservices architecture, the most painful source of coupling is actually organizational and not architectural. Design patterns such as the Prometheus Operator enable teams to be more self-sufficient, and reduce organizational coupling, enabling teams to code faster.

Ambassador and Istio: Edge proxy and service mesh

kelseyevans — Fri, 12 Jan 2018 16:15:41 +0000

Ambassador is a Kubernetes-native API Gateway for microservices. Ambassador is deployed at the edge of your network, and routes incoming traffic to your internal services (aka "north-south" traffic). Istio is a service mesh for microservices, and designed to add L7 observability, routing, and resilience to service-to-service traffic (aka "east-west" traffic). Both Istio and Ambassador are built using Envoy.

Ambassador and Istio can be deployed together on Kubernetes. In this configuration, incoming traffic from outside the cluster is first routed through Ambassador, which then routes the traffic to Istio. Ambassador handles authentication, edge routing, TLS termination, and other traditional edge functions.

This allows the operator to have the best of both worlds: a high performance, modern edge service (Ambassador) combined with a state-of-the-art service mesh (Istio). Istio's basic ingress controller, the ingress controller is very limited, and has no support for authentication or many of the other features of Ambassador.

Getting Ambassador working with Istio

Getting Ambassador working with Istio is straightforward. In this example, we'll use the bookinfo sample application from Istio.

Install Istio on Kubernetes, following the default instructions.
Next, install the Bookinfo sample application, following the instructions.
Verify that the sample application is working as expected.

By default, the Bookinfo application uses the Istio ingress. To use Ambassador, we need to:

Install Ambassador. See the quickstart guide.
Update the bookinfo.yaml manifest to include the necessary Ambassador annotations. See below.

Optionally, delete the Ingress controller from the bookinfo.yaml manifest by typing kubectl delete ingress gateway.
Test Ambassador by going to $AMBASSADOR_IP/productpage/. You can get the actual IP address for Ambassador by typing kubectl get services ambassador.

Automatic sidecar injection

Newer versions of Istio support Kubernetes initializers to automatically inject the Istio sidecar. With Ambassador, you don't need to inject the Istio sidecar -- Ambassador's Envoy instance will automatically route to the appropriate service(s). If you're using automatic sidecar injection, you'll need to configure Istio to not inject the sidecar automatically for Ambassador pods. There are several approaches to doing this that are explained in the documentation.

Tutorial: Getting started with Ambassador - a Kubernetes-native API gateway for microservices

kelseyevans — Wed, 03 Jan 2018 19:48:25 +0000

Ambassador is a Kubernetes-native API gateway for microservices built on the Envoy Proxy. Ambassador is designed for self-service. Developers should be able to manage basic aspects of Ambassador without requiring operations. Ambassador accomplishes this by enabling developers to configure it through Kubernetes annotations. This allows developers to easily manage Ambassador using their existing Kubernetes deployment workflow.

In this tutorial, we'll do a quick tour of Ambassador with a demo configuration before walking through how to deploy Ambassador in Kubernetes with a custom configuration.

1. Running the demo configuration

By default, Ambassador uses a demo configuration to show some of its basic features. Get it running with Docker, and expose Ambassador on port 8080:

docker run -it -p 8080:80 --name=ambassador --rm datawire/ambassador:{VERSION} --demo

2. Ambassador's Diagnostics

Ambassador provides live diagnostics viewable with a web browser. While this would normally not be exposed to the public network, the Docker demo publishes the diagnostics service at the following URL:

http://localhost:8080/ambassador/v0/diag/

Some of the most important information - your Ambassador version, how recently Ambassador's configuration was updated, and how recently Envoy last reported status to Ambassador - is right at the top. The diagnostics overview can show you what it sees in your configuration map, and which Envoy objects were created based on your configuration.

3. The Quote of the Moment service

Since Ambassador is an API gateway, its primary purpose is to provide access to microservices. The demo is preconfigured with a mapping that connects the /qotm/ resource to the "Quote of the Moment" service -- a demo service that supplies quotations. You can try it out here:

curl http://localhost:8080/qotm/

This request will route to the qotm service at demo.getambassador.io, and return a quote in a JSON object.

You can also see the mapping by clicking the mapping-qotm.yaml link from the diagnostic overview, or by opening

http://localhost:8080/ambassador/v0/diag/mapping-qotm.yaml

4. Authentication

On the diagnostic overview, you can also see that Ambassador is configured to do authentication -- click the auth.yaml link, or open

http://localhost:8080/ambassador/v0/diag/auth.yaml

for more here. Ambassador uses a demo authentication service at demo.getambassador.io to mediate access to the Quote of the Moment: simply getting a random quote is allowed without authentication, but to get a specific quote, you'll have to authenticate:

curl -v http://localhost:8080/qotm/quote/5

will return a 401, but

curl -v -u username:password http://localhost:8080/qotm/quote/5

will succeed. (Note that that's literally "username" and "password" -- the demo auth service is deliberately not very secure!)

Note that it's up to the auth service to decide what needs authentication -- teaming Ambassador with an authentication service can be as flexible or strict as you need it to be.

5. Ambassador in Kubernetes

So far, we've used a demo configuration, and run everything in our local Docker instance. We'll now switch to Kubernetes, using service annotations to configure Ambassador to map /httpbin/ to httpbin.org.

5.1 Defining the Ambassador Service

Ambassador is deployed as a Kubernetes service. Create the following YAML and put it in a file called ambassador-service.yaml.

Then, apply it to the Kubernetes with kubectl:

kubectl apply -f ambassador-service.yaml

The YAML above does several things:

It creates a Kubernetes service for Ambassador, of type LoadBalancer. Note that if you're not deploying in an environment where LoadBalancer is a supported type, you'll need to change this to a different type of service, e.g., NodePort.
It creates a test route that will route traffic from /httpbin/ to the public httpbin.org service. In Ambassador, Kubernetes annotations (as shown above) are used for configuration. More commonly, you'll want to configure routes as part of your service deployment process, as shown in this more advanced example.

Also, note that we are using the host_rewrite attribute for the httpbin_mapping -- this forces the HTTP Host header, and is often a good idea when mapping to external services. Ambassador supports many different configuration options.

5.2 Deploying Ambassador

Once that's done, we need to get Ambassador actually running. It's simplest to use the YAML files we have online for this (though of course you can download them and use them locally if you prefer!). If you're using a cluster with RBAC enabled, you'll need to use:

kubectl apply -f https://getambassador.io/yaml/ambassador/ambassador-rbac.yaml

Without RBAC, you can use:

kubectl apply -f https://getambassador.io/yaml/ambassador/ambassador-no-rbac.yaml

When Ambassador starts, it will notice the getambassador.io/config annotation on its own service, and use the Mapping contained in it to configure itself. (There's no restriction on what kinds of Ambassador configuration can go into the annotation, but it's important to note that Ambassador only looks at annotations on Kubernetes services.)

Note: If you're using Google Kubernetes Engine with RBAC, you'll need to grant permissions to the account that will be setting up Ambassador. To do this, get your official GKE username, and then grant cluster-admin Role privileges to that username:

$ gcloud info | grep Account
Account: [username@example.org]
$ kubectl create clusterrolebinding my-cluster-admin-binding --clusterrole=cluster-admin --user=username@example.org

5.3 Testing the Mapping

To test things out, we'll need the external IP for Ambassador (it might take some time for this to be available):

kubectl get svc -o wide ambassador

Eventually, this should give you something like:

NAME         CLUSTER-IP      EXTERNAL-IP     PORT(S)        AGE
ambassador   10.11.12.13     35.36.37.38     80:31656/TCP   1m

You should now be able to use curl to httpbin (don't forget the trailing /):

$ curl 35.36.37.38/httpbin/

6. Adding a Service

You can add a service just by deploying it with an appropriate annotation. For example, we can deploy the QoTM service locally in this cluster and automatically map it through Ambassador by creating qotm.yaml with the following:

and then applying it with

kubectl apply -f qotm.yaml

A few seconds after the QoTM service is running, Ambassador should be configured for it. Try it with

$ curl 35.36.37.38/qotm/

7. The Diagnostics Service in Kubernetes

Note that we did not expose the diagnostics port for Ambassador, since we don't want to expose it on the Internet. To view it, we'll need to get the name of one of the ambassador pods:

$ kubectl get pods
NAME                          READY     STATUS    RESTARTS   AGE
ambassador-3655608000-43x86   1/1       Running   0          2m
ambassador-3655608000-w63zf   1/1       Running   0          2m

Forwarding local port 8877 to one of the pods:

kubectl port-forward ambassador-3655608000-43x86 8877

will then let us view the diagnostics at http://localhost:8877/ambassador/v0/diag/.

8. Next

We've just done a quick tour of some of the core features of Ambassador: diagnostics, routing, configuration, and authentication.

Join us on Gitter;
Learn how to add authentication to existing services; or
Learn how to use gRPC with Ambassador; or
Read about configuring Ambassador.
Learn how to use Ambassador with Istio

A development workflow for Kubernetes services

kelseyevans — Tue, 21 Nov 2017 18:46:02 +0000

A basic development workflow for Kubernetes services lets a developer write some code, commit it, and get it running on Kubernetes. It's also important that your development environment be as similar as possible to production, since having two different environments will inevitably introduce bugs. In this tutorial, we'll walk through a basic development workflow that is built around Kubernetes, Docker, and Envoy/Ambassador.

Your cloud infrastructure

This tutorial relies on two components in the cloud, Kubernetes and Ambassador. If you haven't already, go ahead and set them up.

Kubernetes
Ambassador, a self-service API Gateway for Kubernetes

1. A development environment for Kubernetes services

You need a development environment for Kubernetes services. We recommend the following approach:

A containerized build/runtime environment, where your service is always run and built. Containerizing your environment helps insure environmental parity across different development and production environments. It also simplifies the onboarding process for new developers.
Developing your microservice locally, outside of the cluster. You want a fast code/build/test cycle. If you develop remotely, the additional step of deploying to a Kubernetes cluster introduces significant latency.
Deploying your service into Kubernetes once you need to share your service with others (e.g., canary testing, internal development, etc.).

You'll need the following tools installed on your laptop:

git, for source control
Docker, to build and run your containers
kubectl, to manage your deployment
Forge, for deploying your service into Kubernetes
Telepresence, for locally developing your service

Go ahead and install them now, if you haven't already.

2. Deploy service to Kubernetes

In a traditional application, the release / operations team manages the deployment of application updates to production. In a microservices architecture, the team is responsible for deploying service updates to production.

We're going to deploy and publish a microservice, from source, into Kubernetes.

We've created a simple Python microservice that you can use as a template for your service. This template includes:

a Dockerfile that specifies how your development environment and runtime environment are configured and built.
a service.yaml file that customizes deployments for different scenarios (e.g., production, canary, development).
a Kubernetes manifest (k8s/deployment.yaml) that defines how the service is run in Kubernetes. It also contains the annotations necessary to configure Ambassador for the given service.

   git clone https://github.com/datawire/hello-world-python

We're going to use Forge to automate and template-ize the deployment process. Run the Forge configuration process:

   forge setup

The process of getting a service running on a Kubernetes cluster involves a number of steps: building a Docker image, pushing the image to a repository, instantiating a Kubernetes manifest to point to the image, and applying the manifest to the cluster. Forge automates this entire process of deployment:

   cd hello-world-python
   forge deploy

Now, we're going to test the service. Get the external IP address of Ambassador:

   kubectl get services ambassador
   NAME         CLUSTER-IP      EXTERNAL-IP      PORT(S)        AGE
   ambassador   10.11.250.208   35.190.189.139   80:31622/TCP   4d

Access the service via Ambassador:

   curl 35.190.189.139/hello/
   Hello World (Python)! (up 0:03:13)

3. Live coding

When developing, you want a fast feedback cycle. You'd like to make a code change, and immediately be able to build and test your code. The deployment process we just went through adds latency into the process, since building and deploying a container with your latest changes takes time. Yet, running a service in Kubernetes lets that service access other cloud resources (e.g., other services, databases, etc.).

Telepresence lets you develop your service locally, while creating a bi-directional proxy to a remote Kubernetes cluster.

You'd like for your development environment to be identical to your runtime environment. We're going to do that by using the exact same Dockerfile we use for production to build a development image. Make sure you're in the hello-world-python directory, and type:

   docker build . -t hello-world-dev

Now, we can swap the existing hello-world service on Kubernetes for a version of the same service, running in a local container.

   telepresence --swap-deployment hello-world-stable --docker-run \
    --rm -it -v $(pwd):/service hello-world-dev:latest

(Note that Forge has automatically appended a stable suffix to the deployment name to indicate that the service has been deployed with the stable profile specified in the service.yaml.)

Telepresence invokes docker run to start the container. It also mounts the local filesystem containing the Python source tree into the container. Change the "Hello World" message in app.py to a different value:

   def root(): 
    return "Hello World via Telepresence! (up %s)\n" % elapsed()

Now, if we test our service via Ambassador, we'll see that we're now routing to the modified version of our service.

   curl 35.190.189.139/hello/
   Hello World via Telepresence! (up 0:04:13)

Want to learn more?

This article originally appeared on Datawire's Code Faster Guides. Check out the other tutorials in this series:

Or try out the open source projects mentioned in this tutorial:

If you have any questions, reach out to us on Gitter.