DEV Community: Peter ONeill

Debugging Go Microservices in Kubernetes with VScode

Peter ONeill — Fri, 09 Apr 2021 14:04:58 +0000

Tutorial: Learn to debug Go microservices locally while testing against dependencies in a remote Kubernetes cluster

Many organizations adopt cloud native development practices with the dream of shipping features faster. Although the technologies and architectures may change when moving to the cloud, the fact that we all still add the occasional bug to our code remains constant. The snag is that many of your existing local debugging tools and practices can’t be used when everything is running in a container or on the cloud.

Easy and efficient debugging is essential to being a productive engineer, but when you have a large number of microservices running in Kubernetes the approach you take to debugging has to change. For one, you typically can’t run all of your dependent services on your local machine. This then opens up the challenges of remote debugging (and the associated fiddling with debug modes and exposing ports correctly). However, there is another way. And the CNCF Telepresence tool enables this path.

This article walks you through using Telepresence to seamlessly connect your local development machine to a remote Kubernetes cluster, allowing you to use your favorite debugging tools with all of your microservices. Giving you the ability to comfortably debug your remote apps with your existing skills.

The Difficulty with Debugging Applications Running in Kubernetes

Splitting your application into microservices introduces a number of challenges. Particularly with debugging. Splitting an application into several, if not dozens, of microservices creates a complex dependency tree that becomes nearly impossible to replicate in staging.

Sure, you can use unit tests with tools like GoMock and GoStub to simulate external dependencies or introduce synthetic data into the mix, but it still leaves you unsure if it will work with data from your actual services.

And after you have deployed your service into your cluster, running a remote debugging session can be tricky to get right, due to the complicated configuration of ports and protocols that need to be set via your Kubernetes service YAML. You often lose the ability to use your favorite debugging tools and strategies.

Step 1: Deploy a Sample Microservice Application

In this article, we’re going to be working with the sample Edgey Corp application written in Go. There are detailed instructions for deploying this application in my previous blog post. Or just apply the YAML from the URL to your Kubernetes cluster, clone the repository, and jump right into the action.

Apply the manifest for the Edgey Corp app to your K8s cluster

kubectl apply -f [https://raw.githubusercontent.com/datawire/edgey-corp-go/main/k8s-config/edgey-corp-web-app-no-mapping.yaml](https://raw.githubusercontent.com/datawire/edgey-corp-go/main/k8s-config/edgey-corp-web-app-no-mapping.yaml)

Git clone the code for the Edgey Corp app to your machine

git clone [https://github.com/datawire/edgey-corp-go.git](https://github.com/datawire/edgey-corp-go.git)

Now that we have the Edgey Corp app running in our Kubernetes cluster, and we have the code on our local machine let’s pop it open with VSCode.

Let’s check the application is running successfully. Run a kubectl get pods to see the pods are up and running for the 3 microservices. verylargedatastore, verylargejavaservice and dataprocessingservice.

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
verylargedatastore-cd998dfc6-k5bkw 1/1 Running 0 19s
verylargejavaservice-77748f79d6-dx4kj 1/1 Running 0 20s
dataprocessingservice-f5b644d95-s98hp 1/1 Running 0 20s

With the sample application up and running we can configure VScode to start debugging.

Step 2: Configure VScode and Delve

VScode is a great all-purpose IDE with countless extensions to get you up and running quickly. We are going to use the Go extension made by the Google team with the Go debugger tool called Delve. Here are links to each tool:

You’ll want to make sure that Delve is both installed to your local system and connected to VScode through the Go extension.

Install Delve to your local system

go install github.com/go-delve/delve/cmd/dlv@latest

Connect Delve to VScode

Open VScode now
Access the command palette with (command+shift+p)
Select Go: install/update tools
Check the box for DLV and any other options you might find useful
Click Ok

Run your Go code in debugging mode

Open the project directory for the Edgy Corp Go App in VScode
Open up the file dataprocessingservice/main.go
Click the debug icon on the right side.
Lastly, click “Run and Debug”

You should now see the debug console starting to log output.

API server listening at: 127.0.0.1:48808
Welcome to the DataProcessingGoService!

Let’s check that it’s working by navigating to localhost:3000 in your browser. You can also navigate to localhost:3000/color to see how the Go application response to the color endpoint.

Step 3: Intercept Your Service with Telepresence

So debugging one microservice in isolation is fine, but it still leaves us assuming what is going to happen when it is connected to the rest of the microservice in our application. Let’s introduce Telepresence here to debug our local Go microservice as part of the larger application.

Install telepresence CLI

macOS

sudo curl -fL [https://app.getambassador.io/download/tel2/darwin/amd64/latest/telepresence](https://app.getambassador.io/download/tel2/darwin/amd64/latest/telepresence) -o /usr/local/bin/telepresence

Linux

sudo curl -fL [https://app.getambassador.io/download/tel2/linux/amd64/latest/telepresence](https://app.getambassador.io/download/tel2/linux/amd64/latest/telepresence) -o /usr/local/bin/telepresence

Make the binary executable

sudo chmod a+x /usr/local/bin/telepresence

Connect your local machine to your Kubernetes cluster.
```
$ telepresence connect
```
Test the connection by cURL-ling our front-end service via its Kubernetes service name.
```
$ curl [http://verylargejavaservice.default:8080/color](http://verylargejavaservice.default:8080/color)
“green”
```
Notice two things here:

A) You are able to refer to the remote Service directly via its internal cluster name as if your development machine is inside the cluster

B) The color returned by the remote DataProcessingService is “green”, versus the local result you saw above of “blue”

Let’s take our connecting to the cluster a step further now by initiating an intercept from our remote dataprocessingservice to our local debug session running on http://localhost:3000. This will send any traffic destined for the remote service down to our debugger

$ telepresence intercept dataprocessingservice --port 3000
Launching Telepresence Daemon v2.1.2 (api v3)
Connecting to traffic manager…
Connected to context peteroneilljr-office-hours ([https://34.67.161.22](https://34.67.161.22))
Using deployment dataprocessingservice
intercepted
Intercept name: dataprocessingservice
State : ACTIVE
Destination : 127.0.0.1:3000
Intercepting : all TCP connections

Let’s try to cURL our front-end service once again. This time we should see our local debugger logging the interaction between the front-end service and our local dataprocessingservice.
```
$ curl [http://verylargejavaservice.default:8080/](http://verylargejavaservice.default:8080/)
```
Awesome! Now that our local debugger is hooked up to the remote cluster we should be all set to start debugging!

Step 4: Step Through Your Breakpoints

Let’s set a breakpoint on the getColor function. Hover over just to the left of the line number and click on the red dot. Once it’s set, visit localhost:3000 in your browser, VScode should pop to the foreground. Click the play button to close the request and the webpage should finish loading.

Now your IDE is processing traffic directly from the cluster, let’s open up our sample application in a web browser:

http://verylargejavaservice.default:8080

VScode will pop open on the breakpoint again. This time let’s click the step button (the one with the clockwise rotating arrow) until the breakpoint reaches the fmt.Println and you see the c: “green” in the variables window. If you try to adjust the variable directly in the IDE variables pane you will receive the following error.

Failed to set variable — literal string can not be allocated because function calls are not allowed without using ‘call’

More details about the bug here: https://github.com/golang/vscode-go/issues/1173

To work around this issue we can adjust the variable in the debug terminal with the command call c = “orange”

Now let’s end the breakpoint and send the info back to the web browser. Click the play button in the top command bar.

Awesome! You should see the verylargejavaservice loading with the orange background colors. We’ve successfully made a request to our front-end service running in our remote cluster, where Telepresence intercepted the traffic sending it through our local debugging session, where we inspected and updated the data to send back the orange color variable.

Introducing Telepresence into the development flow gives us a bridge from our local development environment to our remote Kubernetes cluster. Allowing us to test and debug traffic from the remote cluster as if it was on our local machine, in other words, giving us feedback from our local service to see how it will perform when running in the remote cluster.

Step 5: (Bonus) Clone your Pod’s environment variables to your debug session

In some scenarios, a service can inherit environment variables from the cluster or configuration. In cases like this, cloning the remote deployments environment variables to our local service will be necessary to ensure closer parity between the two environments. To do this we will leverage Telepresence to copy the environment variables from the remote service and save them to a .env file. Then we will tell VScode to add these variables to the debug environment.

To let VScode know that we want to include an environment file, we need to create a launch configuration specifying where the file is going to be.

Stop your last debugging session if it is still running.
Click the Gear Icon in the top right corner to open the launch.json file.
From here click the Add Configuration button.
Select Go: Launch File from the dropdown menu.
Add the line "envFile": "${workspaceFolder}/go-debug.env"

    {
     "name": "Launch with env file",
     "type": "go",
     "request": "launch",
     "mode": "debug",
     "program": "${file}",
     "envFile": "${workspaceFolder}/go-debug.env"
    }

Now that VScode knows where the file is going to be located let’s generate the file with Telepresence. This time run the intercept from the terminal window within VScode to ensure the file is created in the project directory.

$ telepresence intercept dataprocessingservice --port=3000 --env-file=go-debug.env

Finally, start the debug session. Click the debug icon on the right side and click the launch session button.

Now your local process is running with all the same variables as if it was running in the remote cluster.

You can also locally access volumes mounted into your remote Services. This is useful if you are storing configuration, tokens, or other state required for the proper execution of the service. We’ll cover this in more detail in a future tutorial.

Learn More About Telepresence

Today, we’ve learned how to use Telepresence to easily debug a Go microservice running in Kubernetes. Now, instead of trying to mock out dependencies or fiddle around with remote debugging, we can iterate quickly with an instant feedback loop when locally debugging using our favorite IDE and tools.

If you want to learn more about Telepresence, check out the following resources:

Read the docs
Watch the demo video
Read more about Intercepts
Learn about Preview URLs for easy collaboration with teammates
Join our Slack channel to connect with the Telepresence community

Go & Kubernetes: Rapidly Developing Golang Microservices

Peter ONeill — Wed, 03 Mar 2021 19:06:55 +0000

Build a cloud development environment with Telepresence & Golang

Kubernetes is a container orchestration platform that enables its users to deploy and scale their microservice applications at any scale: from one service to thousands of services. Unleashing the power of Kubernetes is often more complicated than it may initially seem — the learning curve for application developers is particularly steep. Knowing what to do is just half the battle, then you have to choose the best tools to do the job. So how do Go developers create a development workflow on Kubernetes that is fast and effective?

Application developers face two unique challenges when trying to create productive development workflows on Kubernetes.

Most development workflows are optimized for local development, and Kubernetes applications are designed to be native to the cloud.
As Kubernetes applications evolve into complex microservice architectures, the development environments also become more complex as every microservice adds additional dependencies. These services quickly start to need more resources than are available in your typical local development environment.

In this tutorial, we’ll set up a development environment for Kubernetes and make a change to a Golang microservice. Normally to develop locally, we’d have to wait for a container build, push to registry and deploy to see the effect of our code change. Instead, we’ll use Telepresence to see the results of our change instantly.

Step 1: Deploy a Sample Microservices Application

For our example, we’ll make code changes to a Go service running between a resource-intensive Java service and a large datastore. We’ll start by deploying a sample microservice application consisting of 3 services:

VeryLargeJavaService: A memory-intensive service written in Java that generates the front-end graphics and web pages for our application
DataProcessingService: A Golang service that manages requests for information between the two services.
VeryLargeDataStore: A large datastore service that contains the sample data for our Edgey Corp store.

Note: We’ve called these VeryLarge services to emphasize the fact that your local environment may not have enough CPU and RAM, or you may just not want to pay for all that extra overhead for every developer.

In this architecture diagram, you’ll notice that requests from users are routed through an ingress controller to our services. For simplicity’s sake, we’ll skip the step of deploying an ingress controller in this tutorial. If you’re ready to use Telepresence in your own setup and need a simple way to set up an ingress controller, we recommend checking out the Ambassador Edge Stack which can be easily configured with the K8s Initializer.

Let’s deploy the sample application to your Kubernetes cluster:

    kubectl apply -f [https://raw.githubusercontent.com/datawire/edgey-corp-go/main/k8s-config/edgey-corp-web-app-no-mapping.yaml](https://raw.githubusercontent.com/datawire/edgey-corp-go/main/k8s-config/edgey-corp-web-app-no-mapping.yaml)

Step 2: Set up your local Go development environment

We’ll need a local development environment so that we can edit the DataProcessingService service. As you can see in the architecture diagram above, the DataProcessingService is dependent on both the VeryLargeJavaService and the VeryLargeDataStore, so in order to make a change to this service, we’ll have to interact with these other services as well. Let’s get started!

Clone the repository for this application from GitHub.

    git clone [https://github.com/datawire/edgey-corp-go.git](https://github.com/datawire/edgey-corp-go.git)

Change directories into the DataProcessingService

    cd edgey-corp-go/DataProcessingService

Start the Go server:

    go build main.go && ./main

See your service running!

    10:23:41 app | Welcome to the DataProcessingGoService!

In another terminal window, curl localhost:3000/color to see that the service is returning blue.

    $ curl localhost:3000/color

    “blue”

Step 3: Rapid Development with Telepresence

Instead of waiting for a container image to build, pushed to a repository, and deployed to our Kubernetes cluster we are going to use Telepresence, an open source Cloud Native Computing Foundation project. Telepresence creates a bidirectional network connection between your local development and the Kubernetes cluster to enable fast, efficient Kubernetes development.

Download Telepresence (~60MB):

    # Mac OS X
    sudo curl -fL [https://app.getambassador.io/download/tel2/darwin/amd64/latest/telepresence](https://app.getambassador.io/download/tel2/darwin/amd64/latest/telepresence) -o /usr/local/bin/telepresence

    #Linux
    sudo curl -fL https://app.getambassador.io/download/tel2/linux/amd64/latest/telepresence -o /usr/local/bin/telepresence

Make the binary executable

    $ sudo chmod a+x /usr/local/bin/telepresence

Test Telepresence by connecting to the remote cluster

    $ telepresence connect

Send a request to the Kubernetes API server:

    $ curl -ik [https://kubernetes.default.svc.cluster.local](https://kubernetes.default.svc.cluster.local)

    HTTP/1.1 401 Unauthorized
    Cache-Control: no-cache, private
    Content-Type: application/json
    Www-Authenticate: Basic realm="kubernetes-master"
    Date: Tue, 09 Feb 2021 23:21:51 GMT

Great! You’ve successfully configured Telepresence. Right now, Telepresence is intercepting the request you’re making to the Kubernetes API server, and routing over its direct connection to the cluster instead of over the Internet.

Step 4: Intercept Your Golang Service

An intercept is a routing rule for Telepresence. We can create an intercept to route traffic intended for the DataProcessingService in the cluster and instead route all of the traffic to the local version of the DataProcessingService running on port 3000.

Create the intercept

    telepresence intercept dataprocessingservice — port 3000

Access the application directly with Telepresence. Visit http://verylargejavaservice:8080. Again, Telepresence is intercepting requests from your browser and routing them directly to the Kubernetes cluster.
1. Now, we’ll make a code change. Open edgey-corp-go/DataProcessingService/main.go and change the value of the color variable from blue to orange. Save the file, stop the previous server instance and start it again with go build main.go && ./main.
2. Reload the page in your browser and see how the color has changed from blue to orange!

That’s it! With Telepresence we saw how quickly we can go from editing a local service to seeing how these changes will look when deployed with the larger application. When you compare it to our original process of building and deploying a container after every change, it’s very easy to see how much time you can save especially as we make more complex changes or run even larger services.

Learn More about Telepresence

Today, we’ve learned how to use Telepresence to rapidly iterate on a Golang microservice running in Kubernetes. Now, instead of waiting for slow local development processes, we can iterate quickly with an instant feedback loop and a productive cloud native development environment.

If you want to learn more about Telepresence, check out the following resources:

Read the docs
Watch the demo video
Read more about Intercepts
Learn about Preview URLs for easy collaboration with teammates
Join our Slack channel to connect with the Telepresence community

In our next tutorial, we’ll use Telepresence to set up a local Kubernetes development environment and then use Delve to set breakpoints and debug a broken service. To be notified when more tutorials are available, make sure to check out our website or follow us on Twitter.

Cloud Development Environments: Using Skaffold and Telepresence on Kubernetes for fast dev loops

Peter ONeill — Thu, 11 Feb 2021 15:59:04 +0000

Build and deploy microservices with Skaffold; Test against remote dependencies with Telepresence

Skaffold is an open source project created by Google. It provides a development framework for Kubernetes based applications. This framework creates a fast, repeatable, and simple local Kubernetes workflow. Put simply, this means that it handles all the hard bits of managing a cloud native local development environment, such as building images and deploying to Kubernetes. With Skaffold handling the hard bits, you save time with every iteration you make in development.

Telepresence is an open source tool created by Ambassador Labs. Telepresence makes Kubernetes developers super productive by letting them code as if their laptop is in their Kubernetes cluster. This way you can query cluster resources and experiment rapidly with other services in realtime.

Together, Skaffold and Telepresence give developers a supercharged development workflow for Kubernetes, where Skaffold handles building and deploying your local service and Telepresence allows you to test it against services in your remote cluster.

In this tutorial, we’ll use Skaffold to build and deploy our local environment. Then, we’ll spin up Telepresence to do some iterations while projecting the local service we are building to a remote cluster. Using Telepresence as part of Ambassador Cloud gives us some additional benefits, like being able to share preview URLs with teammates, so they can view the services running on your local machine as if they were in production. We’ll talk more about this in future blog posts.

CI/CD and the Inner Dev Loop

Continuous integration and continuous deployment — commonly referred to as CI/CD — is the de-facto method used to develop and deploy cloud native applications. On the right side of the diagram above, we have the continuous deployment workflow or outer loop. On the left-side of the diagram we have the development workflow, typically called the inner-dev-loop. This inner-dev-loop is where you write and test code before committing it to a version control system. Because you’re in the midst of the creative process of writing and testing code, you want this feedback loop to be as fast as possible. The faster this loop performs the faster you can refactor your code and test again.

The desire to make the loop faster is why tools like Skaffold and Telepresence, which are designed to save you time in this inner-dev-loop, are a great addition to Cloud Native workflows. With Skaffold managing your local deployments and watching your project directory as you make changes, you get a snappy feedback loop where your code appears in your local-dev-environment every time you save a file.

To make your inner-dev-loop even faster, Skaffold can be paired with Telepresence. With Skaffold managing your local service, and Telepresence safely intercepting traffic from your production cluster, you are able to test your code with nearly instantaneous feedback. On top of that, your code has been tested against real traffic saving you time later from debugging minor differences between dev and prod.

The Skaffold dev workflow

When using Skaffold to help manage your local environment you will find yourself in the Skaffold dev mode quite a bit. At the core of Skaffold dev there is a skaffold.yaml file. Sitting at the root directory of your project, this file contains all the info Skaffold needs to turn your code into containers and those containers into an application on your development cluster. Skaffold doesn’t do this alone though, Skaffold takes advantage of the popular tools you probably already use. Such as Dockerfiles or Webpacks to help build the containers, then Kubectl and Helm to deploy them. Because there are so many tooling options for Cloud Native applications, Skaffold has been designed to be highly modular to allow you to mix-and-match with tools you already use.

You can think of the Skaffold dev mode as a fully automated CI/CD pipeline for your development cluster. It takes all the tools you already use to deploy your application and does it automatically every time you make a change. This pipeline can get complex if needed, but at its primary levels it has 2 main stages: build and deploy. Let’s take a look at how this looks with the ‘react-reload’ example provided in the Skaffold repository.

Build

For those familiar with Docker, the default behaviour is to build a docker image using the provided Dockerfile. While this build stage can use other tools like JIB or Webpacks, the primary goal is to get your code packaged into a container so it is ready to be deployed.

From the example’s Dockerfile we see that it is creating a Node container with the port 8080 exposed. No special instructions are added for Skaffold, it is just a simple Dockerfile.

FROM node:12.16.0-alpine3.10
WORKDIR /app
EXPOSE 8080
CMD [“npm”, “run”, “dev”]
COPY package* ./
# examples don’t use package-lock.json to minimize updates
RUN npm install — no-package-lock
COPY . .

One of the most annoying things about managing containers is when you need to rebuild your images every time a file changes, even if the file doesn’t affect the state of the container. This is one area Skaffold really shines, because Skaffold dev manages the deployment, it can update files used by the container without having to rebuild the image.

However, if you make a change that does affect the container directly, Skaffold will pick up on the change and rebuild the necessary images. Then these newly created images are handed off to the deploy stage, where they are seamlessly integrated into your local dev environment. This seamless integration gives you a very fast turnaround time from code change to code tested.

Deploy

Now on to the Kubernetes bits! Skaffold is such a powerful tool because it allows Developers to focus on writing code and managing kubernetes. Switching between writing code and deploying it to your cluster causes an unnecessary context switch from Dev to Ops. In the deploy stage of the pipeline, Skaffold acts like your personal operations engineer. With all the code neatly packaged up from the build stage, Skaffold creates the deployments and services in your development cluster.

Still working with the ‘react-reload’ example, the deployment and service configurations for the sample application are defined in the manifest files located in k8s folder. Here we can see the react-reload image created in the build stage and service exposing port 8080. Skaffold also supports Helm and Kustomize manifest files.

apiVersion: v1
kind: Service
metadata:
  name: node
spec:
  ports:
  - port: 8080
  type: LoadBalancer
  selector:
    app: node
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: node
spec:
  selector:
    matchLabels:
      app: node
  template:
    metadata:
      labels:
        app: node
    spec:
      containers:
      - name: react
        image: react-reload
        ports:
        - containerPort: 8080

With these manifest files and some of the info in the skaffold.yaml file, Skaffold understands how to apply this configuration. With the default behaviour Skaffold will use kubectl as it is currently configured on your machine. Now that the sample application is up and running, Skaffold switches into watcher mode. Keeping the deployment and services up to date with the code in the project directory automagically. So now, as a developer you can go from writing code to testing code deployed to kubernetes without putting on your hat to be an operations engineer, saving you time and the mental tax from these context switches.

React-Reload with Skaffold Dev for fast build and deploy

Now to put all the pieces together and practice using Skaffold dev mode. I’ll walk you through deploying the ‘react-reload’ example to a local cluster running in Docker. Let’s start off by going through the prerequisites.

Prerequisites

Docker
Docker hub account (Necessary for Skaffold + Telepresence Demo)
Kubectl and Kubernetes cluster (running in docker)
Skaffold

Check requirements

docker version to check that docker is running

$ docker version
Client: Docker Engine — Community
Cloud integration: 1.0.4
===trimmed for brevity===
1. Checked that you are logged into docker
$ docker login
Authenticating with existing credentials…
Login Succeeded
1. kubectl get nodes should return docker-desktop
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
docker-desktop Ready master 2d7h v1.19.3
1. skaffold version to check that the Skaffold binary is available
$ skaffold version
v1.19.0

Example Steps

1. Clone Repo

If you haven’t already done so, clone the react reload example locally and cd into the read-reload directory.

$ git clone git@github.com:GoogleContainerTools/skaffold.git
$ cd skaffold/examples/react-reload

2. Start Skaffold Dev

Run skaffold dev — default-repo <repository-name> in the project directory to kick things off. Remember to replace the repository name with your personal repository.

*Note: We are using an external repository here. When using a local cluster skaffold dev works without an external repository, but we will be using this repository in a later section of the tutorial.

$ skaffold dev — default-repo peteroneilljr
Listing files to watch…
- react-reload
Generating tags…
- react-reload -> peteroneilljr/react-reload:v1.17.1–57-g95092addf-dirty
Checking cache…
- react-reload: Not found. Building
Found [docker-desktop] context, using local docker daemon.
Building [react-reload]…
Sending build context to Docker daemon 11.26kB
Step 1/7 : FROM node:12.16.0-alpine3.10
== trimmed for brevity ===
Step 7/7 : COPY . .
— -> 11fd5f0fcad4
Successfully built 11fd5f0fcad4
Successfully tagged peteroneilljr/react-reload:v1.17.1–57-
95092addf-dirty
Tags used in deployment:
- react-reload -> peteroneilljr/react-reload:11fd5f0fcad4559dbc176cabd46b91a01a82d3dd2b6604f7702f338e6275015e
Starting deploy…
- service/node created
- deployment.apps/node created
Waiting for deployments to stabilize…
- deployment/node is ready.
Deployments stabilized in 2.229471962s
Press Ctrl+C to exit
Watching for changes…
[react]
[react] > react-reload@1.0.0 dev /app
[react] > webpack-dev-server — mode development — hot
[react]
[react] ℹ ｢wds｣: Project is running at [http://0.0.0.0:8080/](http://0.0.0.0:8080/)
[react] ℹ ｢wds｣: webpack output is served from /
== trimmed for brevity ===
[react] [./node_modules/webpack/buildin/module.js] (webpack)/buildin/module.js 497 bytes {0} [built]
[react] ℹ ｢wdm｣: Compiled successfully.

3. Review output

From the output we can see a few of things have happened:

Docker has successfully built the container image
1 service was created in the cluster
1 deployment was created in the cluster

Lastly, you will see the logs report the deployment has stabilized and then starts logging the stdout from the container. React is running the container and watching for changes, as changes are detected they will be reported here.

4. Find project URL

So let’s navigate to our sample app and check it out. Near the beginning of the react output you will see that react reports where the application is running. This should match up with the port defined in Dockerfile and manifest files.

Project is running at [<http://0.0.0.0:8080/>](<http://0.0.0.0:8080/>)

5. Open Sample App

Let’s navigate this to this address in your local web browser. If all went well you should see Hello world! in large text in the middle of the window.

6. Modify HelloWorld.js

Now let’s modify some of the configuration files and watch how Skaffold handles it. Using a text editor or IDE open the file: app/src/components/HelloWorld.js

Let’s make it a bit more interesting and have it say Hello From Skaffold!

import React from ‘react’;
import ‘../styles/HelloWorld.css’;
export const HelloWorld = () => (
<div>
<h1>Hello From Skaffold!</h1>
</div>
);

7. Watch file sync in action

Remember the entire project folder is being watched by Skaffold. So the second you save the changes to this file, Skaffold will kick off the update process. Looking back at your web browser you should see that the changes should already be displayed. And from the log output we see that Skaffold reports 1 file has been synced with the running container.

Syncing 1 files for react-reload:5212d19febc7b8a0e5b623473c502a339b7222034e64024a3c3c45e3ab0e3281
Watching for changes…
[react] ℹ ｢wdm｣: Hash: f8313748da8d23ab6dc6
[react] Version: webpack 4.46.0
[react] Time: 120ms
[react] Built at: 01/14/2021 9:05:52 PM
[react] Asset Size Chunks Chunk Names
[react] fa444694c556f0f903da.hot-update.json 46 bytes [emitted] [immutable] [hmr]
[react] index.html 295 bytes [emitted]
[react] main.fa444694c556f0f903da.hot-update.js 1.39 KiB main [emitted] [immutable] [hmr] main
[react] main.js 1.36 MiB main [emitted] main
[react] Entrypoint main = main.js main.fa444694c556f0f903da.hot-update.js
[react] [./src/components/HelloWorld.js] 210 bytes {main} [built]
[react] + 52 hidden modules
[react] Child html-webpack-plugin for “index.html”:
[react] 1 asset
[react] Entrypoint undefined = index.html
[react] 4 modules
[react] ℹ ｢wdm｣: Compiled successfully.

8. Update dependencies

Now let’s make a change that will cause a bit of a bigger splash; let’s modify the package.json file in the app folder. We can pretend that we need an older version of React. Downgrade the react dependency from 16.13.1 to 16.13.0

“dependencies”: {
    “react”: “¹⁶.13.0”,
    “react-dom”: “¹⁶.13.1”
},

9. Watch automatic redeploy

Unlike the change earlier which did a hot-swap the file javascript file, Skaffold detected the change in dependencies and kicked off a new build for the container image. But we can see docker was able to rebuild the first 4 layers from the cache before copying in the modified package.json file and building the new layer with npm install. Then Docker hands off this newly made image back off to Skaffold where it updates the running deployment.

Generating tags…
- react-reload -> react-reload:v1.17.1–57-g95092addf-dirty
Checking cache…
- react-reload: Not found. Building
Found [docker-desktop] context, using local docker daemon.
Building [react-reload]…
Sending build context to Docker daemon 11.26kB
Step 1/7 : FROM node:12.16.0-alpine3.10
— -> 48ae9395145d
==trimmed for brevity===
— -> ff4ad02e16bb
Step 7/7 : COPY . .
— -> ae2a7b322a8f
Successfully built ae2a7b322a8f
==trimmed for brevity===

10. Conclusion

You should now see your local application running with the updated package dependencies. This alone is a huge time saver, thinking about how often you need to rebuild an image due to a small file change, not to mention redeploying after that. But now let’s look at how we can make this inner-dev-loop even faster; with the addition of Telepresence, we can project our local dev environment into an external cluster.

Skaffold dev and Telepresence for fast, realistic testing in dev

Why introduce Telepresence when we already have a perfectly working development environment? Each of these tools saves time for a very specific piece of the inner-dev-loop, this is what makes them a dynamic duo. And to reiterate the inner-dev-loop is to write code, deploy code, test code and repeat.

Skaffold excels at deploying code to your local development environment, saving you time from running all the operation bits like docker build and kubectl apply. Telepresence on other hand saves time testing code. By intercepting traffic from remote clusters, it’s almost like testing your local services directly in staging or production clusters. Together they create an inner-dev-loop that provides nearly instantaneous feedback on how your code will perform in the production environment. Building on top of example Skaffold is running in Docker.

For today’s example, we’ll be using Telepresence as part of Ambassador Cloud as Ambassador gives us some additional features we’ll want to take advantage of in later parts of this tutorial and future tutorials. Let’s start by projecting this local service into a remote cluster. While it is possible to complete this with another cluster hosted on your local machine, I highly recommend a cluster that is hosted with a Cloud provider. To use Ambassador Cloud, you’ll be asked to log in with GitHub to create a cloud account. Then, you’ll see the instructions to install and configure Telepresence.

Prerequisites

The react-reload demo running from the previous section
A Kubernetes cluster hosted with a Cloud provider
The Telepresence binary, Install instructions can be found once you login here: https://app.getambassador.io/cloud/preview/

Check requirements

Check the demo is still running in your local browser
Create a new terminal window with kubectl connected with your cloud-hosted cluster. I’m setting this new terminal to use a new kubeconfig file, but you can also switch context.

$ export KUBECONFIG=/home/cloud-cluster.yml
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
cloud-cluster Ready master 10d v1.17.3+k3s1
1. Verify telepresence is installed and working
$ telepresence version
Client v2.0.0 (api v3)

Example Steps

1. Upload container image to Docker hub

For Telepresence to intercept traffic we need our sample application to also be running in our Cloud cluster. To do this we need to first upload the container image created by Skaffold to Dockerhub.

Let’s find the image id with docker image ls

$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
peteroneilljr/react-reload e2a7b322a8f36cf26ff3c12aaa156d8176fb42f26a0ceba3e241b6350c6c01c ae2a7b322a8f 2 hours ago 174MB
peteroneilljr/react-reload v1.17.1–57-g95092addf-dirty ae2a7b322a8f 2 hours ago 174MB
peteroneilljr/react-reload 5212d19febc7b8a0e5b623473c502a339b7222034e64024a3c3c45e3ab0e3281 5212d19febc7 3 hours ago 174MB
=== trimmed for brevity ===

We see at least a few different react reload images. The first image created is the one we want to upload, the newer images were created by Skaffold when we made modifications. So let’s give the base image a tag of “latest” and push it to Dockerhub

$ docker image tag 5212d19febc7 peteroneilljr/react-reload:latest
$ docker push peteroneilljr/react-reload
Using default tag: latest
The push refers to repository [docker.io/peteroneilljr/react-reload]
ef61bb2587ca: Pushed
34561a8deb68: Pushed
e2718bad0837: Pushed
3687ae930f2e: Pushed
5947f76477a5: Mounted from library/node
8b86a15dc982: Mounted from library/node
66d69d514191: Mounted from library/node
531743b7098c: Mounted from library/node
latest: digest: sha256:91676ba8810f01b39416f5adaf7cb2c47e82b6a42e58e0735ab6f4f47fb78902 size: 1992

2. Run the sample app in the Cloud cluster

Now that our image is hosted in a public repository we can recreate our sample application in the cloud cluster. Remember the kubernetes manifest? We had one deployment and one service. Let’s create these now.

$ kubectl create deployment node — image=peteroneilljr/react-reload — port=8080
deployment.apps/node created
$ kubectl expose deployment node — type=LoadBalancer — port=8080 — target-port=8080
service/node exposed

3. Check that the sample app is running

Next, let’s find the IP address assigned to our new application.

$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
node LoadBalancer 10.43.95.91 35.223.x.x 8080:32708/TCP 2m52s

And let’s navigate to this address and port in our browser. http://35.223.x.x:8080

4. Intercept traffic

Time to start Telepresence! Let’s start with a telepresence list to verify everything is working as expected.

$ telepresence list
Launching Telepresence Daemon v2.0.0 (api v3)
Need root privileges to run “/usr/local/bin/telepresence daemon-foreground ‘’ ‘’”
Password:
Connecting to traffic manager…
Connected to context default (https://34.123.x.x)
node : ready to intercept (traffic-agent not yet installed)

We should see the node deployment is ready to intercept! So now we are going to intercept the node deployment in the public cluster with the service running on our local port 8080.

$ telepresence intercept node — port 8080
Using deployment node
intercepted
State : ACTIVE
Destination : 127.0.0.1:8080
Intercepting: all connections

Now if you navigate back to your web browser and refresh page, you should see your local deployment being displayed on the public IP address from your Cloud cluster.

5. Testing our inner-dev-loop

With Skaffold and Telepresence working together you’re able to see the changes to our local cluster get populated on the public IP address in real time. Let’s test this out, this time we are going to modify the HelloWorld.css file in the app/src/styles/ folder. Update the h1 color to be ff0000

h1 {
  color: #ff0000;
  text-align: center;
  margin-top: 40vh;
  font-size: 120pt;
}

Once you save this file we should see the heading of the page in your browser pointed at your local cluster change to red nearly instantly, and very shortly after you should notice your public ingress change as well.

6. Sharing our intercept with teammates using Ambassador Cloud

Because WebPack has host checking enabled by default we will need to disable it for this next part.

Open app/webpack.config.js

Add disableHostChecking: true to devServer

devServer:{
  host: ‘0.0.0.0’,
  disableHostCheck: true
},

Save this file and Skaffold will rebuild your local image.

Tag and push this new image to Dockerhub

$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
peteroneilljr/react-reload 82d036eee5887d5660e0a787435f01fddec66c0617c8dc364f45421941aa4f9e 82d036eee588 19 hours ago 173MB
peteroneilljr/react-reload latest 82d036eee588 19 hours ago 173MB
$ docker tag 82d036eee588 peteroneilljr/react-reload
$ docker push peteroneilljr/react-reload

Delete the current pod so that it can respawn with the new configuration.

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
node-86dfc695f5–22psv 1/1 Running 0 13s
$ kubectl delete pod node-86dfc695f5–22psv

Now let’s leave the current intercept

$ telepresence leave node

$ telepresence login
Launching browser authentication flow…
Login successful.

Create a new intercept. This time Telepresence will ask a few questions. Follow the prompts shown below.

$ telepresence intercept node — port 8080
Confirm the ingress to use for preview URL access
Ingress service.namespace [ambassador.ambassador] ? node.default
Port [80] ? 80
Use TLS y/n [n] ? n
Using deployment node
intercepted
State : ACTIVE
Destination : 127.0.0.1:8080
Intercepting: HTTP requests that match all of:
header(“x-telepresence-intercept-id”) ~= regexp(“6e5b52d3–676a-421d-8223–397fb1146162:node”)
Preview URL : [https://nostalgic-colden-2717.preview.edgestack.me](https://nostalgic-colden-2717.preview.edgestack.me)

Now our intercept can be seen and shared with the preview URL.

7. Cleaning up

Just like that we’ve reached the end of our demo, so let’s clean up all the resources we’ve used.

First Telepresence, either leave the intercept if you plan to use it more or uninstall everything. With either approach you’ll see the display quickly return back to the original prompt and color to show the intercept is not longer applied.

$ telepresence leave node
$ telepresence uninstall — everything

Next Skaffold, you can stop skaffold dev by pressing ctr+c within the terminal window.

Lastly, the Cloud cluster. If you’re going to keep using the cluster remove the resources we created, otherwise feel free to delete the whole cluster.

$ kubectl delete svc node

service “node” deleted

$ kubectl delete deploy node

deployment.apps “node” deleted

Conclusion

Now as we can see from this example there are a lot of moving pieces when it comes to writing code for a Cloud Native application. Luckily there are tools like Skaffold that save time managing container images and Kubernetes deployments, and tools like Telepresence & Ambassador to save time debugging by creating a better testing environment.

If you’re considering using Skaffold or just exploring possible tools to make your development workflow better, I hope this article helped out. Drop a comment and let me know what you liked, what you didn’t like, or what you want to hear more about. You can also Tweet at me to let me know what you think @peteroneilljr

Here are some additional resources that may help you get started:

Skaffold documentation
Watch the demo video of Ambassador Cloud and Telepresence
Learn more about Ambassador Cloud and Telepresence
Join our Slack channel