Deploying Java Apps with Kubernetes and the Ambassador API Gateway

In this article you’ll learn how to deploy three simple Java services into Kubernetes (running locally via the new Docker for Mac/Windows integration), and expose the frontend service to end-users via the Kubernetes-native Ambassador API Gateway. So, grab your caffeinated beverage of choice and get comfy in front of your terminal!

A Quick Recap: Architecture and Deployment

In October last year I extended my simple Java microservice-based "Docker Java Shopping" container deployment demonstration with Kubernetes support. If you found the time to complete the tutorial you would have packaged three simple Java services — the shopfront and stockmanager Spring Boot services, and the product catalogue Java EE DropWizard service — within Docker images, and deployed the resulting containers into a local minikube-powered Kubernetes cluster. I also showed you how to open the shopfront service to end-users by mapping and exposing a Kubernetes cluster port using a NodePort Service. Although this was functional for the demonstration, many of you asked how you could deploy the application behind an API Gateway. This is a great question, and accordingly I was keen to add another article in this tutorial series with the goal of deploying the "Docker Java Shopping" Java application behind the open source Kubernetes-native Ambassador API Gateway.

Figure 1. “Docker Java Shopping” application deployed with Ambassador API Gateway

Quick Aside: Why Use an API Gateway?

I'm confident that many of you will have used (or at least bumped into) the concept of an API Gateway before. Chris Richardson has written a good overview of the details at microservices.io, and the team behind the creation of the Ambassador API Gateway, Datawire, have also talked about the benefits of using a Kubernetes-native API Gateway. In short, an API Gateway allows you to centralise a lot of the cross-cutting concerns for your application, such as load balancing, security and rate-limiting. Running a Kubernetes-native API Gateway also allows you to offload several of the operational issues associated with deploying and maintaining a gateway — such as implementing resilience and scalability — to Kubernetes itself.

There are many API Gateway choices for Java developers, such as the open source Netflix's Zuul, Spring Cloud Gateway, and Mashape's Kong; there are cloud vendors' implementations (such as Amazon's API Gateway); and of course the traditional favourites of NGINX and HAProxy; and finally, also the more modern variants like Traefik. Choosing the best API Gateway for your use case can involve a lot of work — this is a critical piece of your infrastructure, and it will touch every bit of traffic coming into your application. As with any critical tech choice, there are many tradeoffs to be considered. In particular, watch out for potential high-coupling points — for example, I've seen the ability to dynamically deploy "Filters" (Groovy scripts) into Netflix's Zuul enables business logic to become spread (coupled) between the service and the gateway — and also the need to deploy complicated datastores as the end-user traffic increases — for example, Kong requires a Cassandra cluster or Postgres installation to scale horizontally.

For the sake of simplicity in this article I'm going to use the open source Kubernetes-native Ambassador API Gateway. I like Ambassador because the simplicity of the implementation reduces the ability to accidentally couple any business logic to it, and the fact that I can specify service routing via a declarative approach (which I use for all of my other Kubernetes config) feels more "cloud native" — I can also store the routes easily in version control, and send this down the CI/CD build pipeline with all the other code changes.

Getting Started: NodePorts and LoadBalancers 101

First, ensure you are starting with a fresh (empty) Kubernetes cluster. Because I like to embrace my inner-hipster every once in a while, I am going to run this demonstration using the new Kubernetes integration within Docker for Mac. If you want to follow along you will need to ensure that you have installed the Edge version of Docker for Mac or Docker for Windows, and also enabled Kubernetes support by following the instructions within the Docker Kubernetes documentation.

Next clone my "Docker Java Shopfront" GitHub repository. If you want to explore the directory structure and learn more about each of the three services that make up the application, then I recommend having a look at the previous article in this series or the associated mini-book "Containerizing Continuous Delivery in Java" that started all of this. When the repo has been successfully cloned you can navigate into the kubernetes directory. If you are following along with the tutorial then you will be making modifications within this directory, and so you are welcome to fork your own copy of the repo and create a branch that you can push your work to. I don't recommend skipping ahead (or cheating), but the kubernetes-ambassador directory contains the complete solution, in case you want to check your work!

$ git clone [e-mail]:danielbryantuk/oreilly-docker-java-shopping.git
$ cd oreilly-docker-java-shopping/kubernetes
(master) kubernetes $ ls -lsa
total 24
0 drwxr-xr-x   5 danielbryant  staff  160  5 Feb 18:18 .
0 drwxr-xr-x  18 danielbryant  staff  576  5 Feb 18:17 ..
8 -rw-r--r--   1 danielbryant  staff  710  5 Feb 18:22 productcatalogue-service.yaml
8 -rw-r--r--   1 danielbryant  staff  658  5 Feb 18:11 shopfront-service.yaml
8 -rw-r--r--   1 danielbryant  staff  677  5 Feb 18:22 stockmanager-service.yaml

If you open up the shopfront-service.yaml in your editor/IDE of choice, you will see that I am exposing the shopfront service as a NodePort accessible via TCP port 8010. This means that the service can be accessed via port 8010 on any of the cluster node IPs that are made public (and not blocked by a firewall).

---
apiVersion: v1
kind: Service
metadata:
 name: shopfront
 labels:
 app: shopfront
spec:
 type: NodePort
 selector:
 app: shopfront
 ports:
 — protocol: TCP
 port: 8010
 name: http

When running this service via minikube, NodePort allows you to access the service via the cluster external IP. When running the service via Docker, NodePort allows you to access the service via localhost and the Kubernetes allocated port. Assuming that Docker for Mac or Windows has been configured to run Kubernetes successfully you can now deploy this service:

(master) kubernetes $ kubectl apply -f shopfront-service.yaml
service "shopfront" created
replicationcontroller "shopfront" created
(master) kubernetes $
(master) kubernetes $ kubectl get services
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP          19h
shopfront    NodePort    10.110.74.43   <none>        8010:31497/TCP   0s

You can see the shopfront service has been created, and although there is no external-ip listed, you can see that the port specified in the stockmanager-service.yaml (8010) has been mapped to port 31497 (your port number may differ here). If you are using Docker for Mac or Windows you can now curl data from localhost (as the Docker app works some magic behind the scenes), and if you are using minikube you can get the cluster IP address by typing minikube ip in your terminal.

Assuming you are using Docker, and that you have only deployed the single shopfront service you should see this response from a curl using the port number you can see from the kubectl get svc command (31497 for me):

(master) kubernetes $ curl -v localhost:31497
* Rebuilt URL to: localhost:31497/
* Trying ::1…
* TCP_NODELAY set
* Connected to localhost (::1) port 31497 (#0)
> GET / HTTP/1.1
> Host: localhost:31497
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 500
< X-Application-Context: application:8010
< Content-Type: application/json;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Tue, 06 Feb 2018 17:20:19 GMT
< Connection: close
<
* Closing connection 0
{“timestamp”:1517937619690,”status”:500,”error”:”Internal Server Error”,”exception”:”org.springframework.web.client.ResourceAccessException”,”message”:”I/O error on GET request for \”http://productcatalogue:8020/products\": productcatalogue; nested exception is java.net.UnknownHostException: productcatalogue”,”path”:”/”}

You’ll notice that you are getting an HTTP 500 error response with this curl, and this is to be expected as you haven’t deployed all of the supporting services yet. However, before you deploy the rest of the services you’ll want to change the NodePort configuration to ClusterIP for all of your services. This means that each services will only be accessible other the network within the cluster. You could of course use a firewall to restrict a service exposed by NodePort, but by using ClusterIP with our local development environment you are forced not to cheat to access our services via anything other than the API gateway we will deploy.

Open shopfront-service.yaml in your editor, and change the NodePort to ClusterIP. You can see the relevant part of the file contents below:

---
apiVersion: v1
kind: Service
metadata:
 name: shopfront
 labels:
 app: shopfront
spec:
 type: ClusterIP
 selector:
 app: shopfront
 ports:
 — protocol: TCP
 port: 8010
 name: http

Now you can modify the services contained with the productcatalogue-service.yaml and stockmanager-service.yaml files to also be ClusterIP.

You can also now delete the existing shopfront service, ready for the deployment of the full stack in the next section of the tutorial.

(master *) kubernetes $ kubectl delete -f shopfront-service.yaml
service “shopfront” deleted
replicationcontroller “shopfront” deleted

Deploying the Full Stack

With a once again empty Kubernetes cluster, you can now deploy the full three-service stack and the get the associated Kubernetes information on each service:

(master *) kubernetes $ kubectl apply -f .
service "productcatalogue" created
replicationcontroller "productcatalogue" created
service "shopfront" created
replicationcontroller "shopfront" created
service "stockmanager" created
replicationcontroller "stockmanager" created
(master *) kubernetes $
(master *) kubernetes $ kubectl get services
NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
kubernetes         ClusterIP   10.96.0.1       <none>        443/TCP    20h
productcatalogue   ClusterIP   10.106.8.35     <none>        8020/TCP   1s
shopfront          ClusterIP   10.98.189.230   <none>        8010/TCP   1s
stockmanager       ClusterIP   10.96.207.245   <none>        8030/TCP   1s

You can see that the port that was declared in the service is available as specified (i.e. 8010, 8020, 8030) — each pod running gets its own cluster IP and associated port range (i.e. each pods gets its own “network namespace”). We can’t access this port outside of the cluster (like we can with NodePort), but within the cluster everything works as expected.

You can also see that using ClusterIP does not expose the service externally by trying to curl the endpoint (this time you should receive a “connection refused”):

(master *) kubernetes $ curl -v localhost:8010
* Rebuilt URL to: localhost:8010/
* Trying ::1…
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 8010 failed: Connection refused
* Trying 127.0.0.1…
* TCP_NODELAY set
* Connection failed
* connect to 127.0.0.1 port 8010 failed: Connection refused
* Failed to connect to localhost port 8010: Connection refused
* Closing connection 0
curl: (7) Failed to connect to localhost port 8010: Connection refused

Deploying the Ambassador API Gateway

Now is the time to deploy the Ambassador API gateway in order to expose your shopfront service to end-users. The other two services can remain private within the cluster, as they are supporting services, and don’t have to be exposed publicly.

First, create a LoadBalancer service that uses Kubernetes annotations to route requests from outside the cluster to the appropriate services. Save the following content within a new file named ambassador-service.yaml. Note the getambassador.io/config annotation. You can use Kubernetes annotations to attach arbitrary non-identifying metadata to objects, and clients such as Ambassador can retrieve this metadata. Can you figure out what this annotation is doing?

The Ambassador annotation is key to how the gateway works — how it routes “ingress” traffic from outside the cluster (e.g. an end-user request) to services within the cluster. Let’s break this down:

• “getambassador.io/config: |” — specifies that this annotation is for Ambassador
• “ — -” — simply declares how much you love YAML!
• “ apiVersion: ambassador/v0” — specifies the Ambassador API/schema version
• “ kind: Mapping” — specifies that you are creating a “mapping” (routing) configuration
• “ name: shopfront” —is the name for this mapping (which will show up in the debug UI)
• “ prefix: /shopfront/” — is the external prefix of the URI that you want to route internally
• “ service: shopfront:8010” — is the Kubernetes service (and port) you want to route to

In a nutshell, this annotation states that any request to the external IP of the LoadBalancer service (which will be “localhost” in your Docker for Mac/Windows example) with the prefix /shopfront/ will be routed to the Kubernetes shopfront service running on the (ClusterIP) port 8010. In your example, when you enter http://localhost/shopfront/ in your web browser you should see the UI provided by the shopfront service. Hopefully this all makes sense, but if it doesn’t then please visit the Ambassador Gitter and ask any questions, or ping me on twitter!

With your newfound understanding of Ambassador routing (and world domination of all API Gateways merely a few steps away), you can deploy the Ambassador service:

(master *) kubernetes $ kubectl apply -f ambassador-service.yaml
service “ambassador” created

You will also need to deploy the Ambassador Admin service (and associated pods/containers) that are responsible for the heavy-lifting associated with the routing. It's worth noting that the routing is conducted by a "sidecar" proxy, which in this case is the Envoy proxy. Envoy is responsible for all of the production network traffic within Lyft, and it's creator, Matt Klein, has written lots of very interesting content about the details. You may have also heard about the emerging "service mesh" technologies, and the popular Istio project also uses Envoy.

Anyway, back to the tutorial! You can find a pre-prepared Kubernetes config file for Ambassador Admin on the getambassador.io website (for this demo you will be using the "no RBAC" version of the service, but you can also find an RBAC-enabled version of the config file if you are running a Kubernetes cluster with Role-Based Access Control (RBAC) enabled. You can download a copy of the config file and look at it before applying, or you apply the service directly via the Interwebs:

(master *) kubernetes $ kubectl apply -f https://getambassador.io/yaml/ambassador/ambassador-no-rbac.yaml
service “ambassador-admin” created
deployment “ambassador” created

If you issue a kubectl get svc you can see that your Ambassador LoadBalancer and Ambassador Admin services have been deployed successfully:

(master *) kubernetes $ kubectl get svc
NAME               TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
ambassador         LoadBalancer   10.102.81.42    <pending>     80:31053/TCP     5m
ambassador-admin   NodePort       10.105.58.255   <none>        8877:31516/TCP   1m
kubernetes         ClusterIP      10.96.0.1       <none>        443/TCP          20h
productcatalogue   ClusterIP      10.106.8.35     <none>        8020/TCP         22m
shopfront          ClusterIP      10.98.189.230   <none>        8010/TCP         22m
stockmanager       ClusterIP      10.96.207.245   <none>        8030/TCP         22m

You will notice on the ambassador service that the external-ip is listed as and this is a known bug with Docker for Mac/Windows. You can still access a LoadBalancer service via localhost — although you may need to wait a minute or two while everything deploys successfully behind the scenes.

Let’s try and access the shopfront this now using the `/shopfront/ route you configured previously within the Ambassador annotations. You can curl localhost/shopfront/ (with no need to specify a port, as you configured the Ambassador LoadBalancer service to listen on port 80):

{% gist https://gist.github.com/kelseyevans/1ad64d89409c1deeb5ee985b7f30a1aa %}

That’s it! You are now accessing the shopfront service that is hidden away in the Kubernete cluster via Ambassador. You can also visit the shopfront UI via your browser, and this provides a much more friendly view!

Bonus: Ambassador Diagnostics

If you want to look at the Ambassador Diagnostic UI then you can use port-forwarding. I’ll explain more about how to use this in a future post, but for the moment you can have a look around by yourself. First you will need to find the name of an ambassador pod:

{% gist https://gist.github.com/kelseyevans/a8fd8d73dcbc97191ec71b55514b7d90 %}

Here I’ll pick ambassador-6d9f98bc6c-5sppl. You can now port-forward from your local network adapter to inside the cluster and expose the Ambassador Diagnostic UI that is runing on port 8877.


(master *) kubernetes $ kubectl port-forward ambassador-6d9f98bc6c-5sppl 8877:8877


You can now visit http://localhost:8877/ambassador/v0/diag in your browser and have a look around!

When you are finished you can exit the port-forward via ctrl-c. You can also delete all of the services you have deployed into your Kubernetes cluster by issuing a kubectl delete -f . within the kubernetes directory. You will also need to delete the ambassador-admin service you have deployed.


(master *) kubernetes $ kubectl delete -f .
service "ambassador" deleted
service "productcatalogue" deleted
replicationcontroller "productcatalogue" deleted
service "shopfront-canary" deleted
replicationcontroller "shopfront-canary" deleted
service "shopfront" deleted
replicationcontroller "shopfront" deleted
service "stockmanager" deleted
replicationcontroller "stockmanager" deleted
(master *) kubernetes $
(master *) kubernetes $ kubectl delete -f https://getambassador.io/yaml/ambassador/ambassador-no-rbac.yaml
service "ambassador-admin" deleted
deployment "ambassador" deleted


What's Next?

I'm planning on creating another article soon that discusses how to canary launch/test a service, as Ambassador makes this very easy. Other topics I'm keen to explore is integrating all of this into a CD pipeline, and also exploring how best to set up a local development workflow. Closely related to this, I'm also keen to look into debugging Java applications deployed via Kubernetes.

You can also read more details on Ambassador itself via the docs, including adding auth/security, gRPC support, and TLS termination.

This article originally appeared on the Ambassador blog written by Daniel Bryant.