DEV Community: Paul Czarkowski

Deploying and accessing a GCP SQL database via Kubernetes

Paul Czarkowski — Mon, 08 Jun 2020 00:00:00 +0000

Sorry for the long preamble, feel free to just skip straight to the technical details.

Two years ago, faced with the challenge of wanting to reduce the tooling required to deploy infrastructure to Google Cloud I started working on a Google Cloud Operator. The goal of this project was to be able to stand up networks, images, and Virtual Machines for installing Pivotal’s PKS (now Tanzu Kubernetes Grid Integrated).

This worked out pretty well for me and I was able to create/destroy environments at will seeded from a single node GKE cluster running the operator.

Thankfully later some folks at Google started working on an Operator of their own which they named Google Config Connector which sought to solve roughly the same problem. Although their original intent was more around backing services for Kubernetes workloads such as databases.

As they brought in additional features I was able to start offloading some of the work my own operator was doing to GCC, however only recently did they support the full set of services that I require.

Recently I’ve been doing some work around running applications on Kubernetes and been thinking about how I want to handle databases in Dev vs Prod. Using a tool like Helm or ytt I could in theory have a flag that determines whether it should create a Postgres deployment in Kubernetes, or a CloudSQL instance.

I hadn’t actually used the Google Config Connector operator to stand up a Database to be accessed from inside a GKE cluster, so I figured that would be the natural place to start.

Unfortunately I found gaps in both the GCC examples and the CloudSQL documentation.

I could get the database running easily enough but I just could not get applications running on GKE to access it over the Private IP. Finally I found this statement For connecting using private IP, the GKE cluster must be VPC-native and in the same VPC network as the Cloud SQL instance..

Doing some digging it became apparent that GKE clusters are not VPC-native by default. I also found that SQL Instances actually never live in your VPC and you need to allocate an IP Range to peering in that network and set up a Service Network Connection.

With that figured out I was able to successfully create a new GKE cluster, install GCC, and deploy and use a Cloud SQL instance over its private IP.

Deploying a CloudSQL instance via GCC

If you want to refer to a ytt templated version of the manifests created below you can find a fully working example (that also includes Google SQL Proxy) here.

Create GKE Cluster

Before we do anything we need to ensure a few of the Google Cloud APIs are enabled.

gcloud services enable \
  servicenetworking.googleapis.com \
  servicemanagement.googleapis.com \
  iamcredentials.googleapis.com

Some of our commands require your Google Cloud Project ID, you can find it and save it as a variable for later use like so:

PROJECT_ID=$(gcloud config get-value project)

Create a Small GKE Cluster using --workload-pool to enable workload identity and --enable-ip-alias to create a VPC-native cluster:

gcloud container clusters create gcc-cloudsql \
  --num-nodes=1 --zone us-central1-c \
  --cluster-version 1.16 --machine-type n1-standard-2 \
  --workload-pool=${PROJECT_ID}.svc.id.goog \
  --enable-ip-alias

Check the cluster is accessible:

kubectl cluster-info

In order for Applications in GKE to access the Private IP of CloudSQL instances you need to create a VPC Peering Range and a Service Networking Connection.

This only has to be done once per Network in your account.

Create a VPC Peering range:

gcloud compute addresses create cloudsql-peer \
    --global \
    --purpose=VPC_PEERING \
    --prefix-length=16 \
    --description="peering range for CloudSQL" \
    --network=default \
    --project=$PROJECT_ID

Peer that range with our default network:

gcloud services vpc-peerings connect \
    --service=servicenetworking.googleapis.com \
    --ranges=cloudsql-peer \
    --network=default \
    --project=$PROJECT_ID

If this command fails with Cannot modify allocated ranges in CreateConnection then rerun the command but replace connect with update --force.

Deploy Google Config Connector

Create a service account for GCC:

gcloud iam service-accounts create cnrm-system

Bind the roles/owner to the service account:

gcloud projects add-iam-policy-binding ${PROJECT_ID} \
  --member="serviceAccount:cnrm-system@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/owner"

Bind roles/iam.workloadIdentityUser to the cnrm-controller-manager Kubernetes Service Account in the cnrm-system Namespace:

gcloud iam service-accounts add-iam-policy-binding \
  --role="roles/iam.workloadIdentityUser" \
  --member="serviceAccount:${PROJECT_ID}.svc.id.goog[cnrm-system/cnrm-controller-manager]" \
cnrm-system@${PROJECT_ID}.iam.gserviceaccount.com

Download GCC:

gsutil cp gs://cnrm/latest/release-bundle.tar.gz release-bundle.tar.gz

Extract GCC:

tar zxvf release-bundle.tar.gz

Update the manifest with your project id:

sed -i.bak "s/\${PROJECT_ID?}/${PROJECT_ID}/" \
  install-bundle-workload-identity/0-cnrm-system.yaml

Apply the manifest to your cluster:

kubectl apply -f install-bundle-workload-identity/

Wait until GCC is fully online:

kubectl wait -n cnrm-system --for=condition=Ready pod --all

Deploy CloudSQL Instance

Create a Namespace:

kubectl create namespace $PROJECT_ID

Create the Cloud SQL Instance:

kubectl -n $PROJECT_ID apply -f <(cat << EOF
apiVersion: sql.cnrm.cloud.google.com/v1beta1
kind: SQLInstance
metadata:
  name: example
spec:
  region: us-central1
  databaseVersion: POSTGRES_9_6
  settings:
    tier: db-custom-1-3840
    ipConfiguration:
      ipv4Enabled: false
      requireSsl: false
      privateNetworkRef:
        external: default
EOF
)

Create a Cloud SQL User:

kubectl -n $PROJECT_ID apply -f <(cat << EOF
apiVersion: sql.cnrm.cloud.google.com/v1beta1
kind: SQLUser
metadata:
  name: example
spec:
  instanceRef:
    name: example
  password:
    value: "bad-password"
EOF
)

After a few moments check that the SQL Instance is being created:

watch gcloud sql instances list

Once the STATUS changes from PENDING_CREATE to RUNNABLE hit CTRL-C to exit the watch command.

We should now be able to confirm that an Application running in the GKE cluster can access the database. To validate this we can use the postgres:13-alpine image.

Run the postgres image as a Pod with an interactive shell:

kubectl run -ti --restart=Never --image postgres:13-alpine --rm psql -- sh

Inside that shell run psql:

Change the IP to match the IP from the gcloud sql instances list command. Enter the password bad-password when prompted

$ psql -h ip.add.re.ss --username=example -d postgres
Password for user example: bad-password
psql (13beta1, server 9.6.16)
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.

Conclusion

That’s it! Super easy and powerful way to get a managed database provisioned and usable without ever leaving Kubernetes.

If you want to make things even more Kubernetes friendly you can use Google’s SQL Proxy inside your Kubernetes cluster which helps manage your SSL Connections to the databases as well as being able to use predictable names rather than hunting for IP addresses.

How to broadcast to Twitch and Zoom with OBS

Paul Czarkowski — Wed, 13 May 2020 00:00:00 +0000

If you prefer a Video Tutorial, you can watch me screencast it on youtube, otherwise read on below.

Recent world events (COVID-19) has completely turned the Developer Advocacy role on its head and many of us are scrambling to find new ways to effectively reach audiences remotely.

Many of us have been reaching for tools like Zoom for smaller groups such as Meetups and streaming platforms like Twitch / YouTube Live for reaching larger groups.

Many of us are turning to Open Broadcaster Studio (OBS) to turn our PCs into virtual production studios capable of composing multiple artifacts such as windows, audio, and webcams into layered streams. The most basic use case for OBS is to provide the ability to display onscreen code or a terminal session with a webcam image overlayed in the corner

OBS can record to a high definition video file on your local machine as well as broadcast it out to Twitch, YouTube, or any other RTSP capable platform.

It should come as no surprise that streaming high quality video from multiple sources can be quite taxing and it often makes sense to have OBS running on a separate computer (refurbished Dell desktops are perfect for this) with a video capture device (like an Elgato HD60) capturing video from your primary machine.

Using OBS its fairly simple to record and broadcast a presentation, or live coding session. However it feels more like a webinar than it does a meetup or conference presentation as the interaction is very one way.

By hooking up Zoom and OBS bidirectionally you can have the reach of a streaming platform at the same time as the more interative nature of having real live people in the (virtual) room with you. Having even just a few people acting as an audience and being able to talk to them and have them talk back and see their body language and reactions makes all the difference.

Unfortunately trying to share a screen and audio through multiple systems is complicated and isn’t possible straight out of the box and we need to solve a few problems:

Rebroadcast video from Zoom through to your stream
Rebroadcast video from OBS through to Zoom
Mix audio from multiple inputs through to multiple outputs (without causing feedback loops)

1. Rebroadcast video from Zoom through to your stream

Multiple Monitors

If you have multiple monitors you can dedicate one of them to Zoom and add a Display Capture source in OBS for that whole display and set zoom to fullscreen on that monitor.

Note: I was on a Zoom between two machines and only one camera, so pretend Not Paul is a very handsome gentleman.

I’m not a fan of the layout of fullscreen zoom calls, and I prefer the flexibility of capturing it as a window, which also makes it possible to do this with a single monitor.

Single Monitor / Window Capture

In OBS you can add a Window Capture and select your Zoom meeting. This will capture just the Zoom window itself. You can adjust the window size to suit the resolution that you’re outputting to.

Note: The window capture settings are OS dependent, on Windows it would display the name and executable file for the window you want to share.

One Caveat is that Zoom likes to switch to fullscreen when somebody shares their screen. Make sure to deselect the Enter full screen when a participant shares screen in the Share Screen Settings to disable this behavior.

You might be tempted to try and share your screen to Zoom, however Zoom behaves painfully when you do that and you’ll most likely lose your capture of the Zoom window. Instead you should add the OBS virtual camera (see below) which lets you use the contents of your OBS scene as a camera in Zoom.

2. Rebroadcast video from OBS through to Zoom

OBS has a plugin that lets you create a virtual camera which you can use as your Zoom camera. The people in your Zoom call will see exactly the same video that you’re streaming without having to share your screen.

You can download the virtual camera plugin from here however it currently only supports Windows. (There’s a video4linux you can use if you’re on Linux) for the same effect.

With the Virtual Camera plugin installed and enabled you can the select it as a camera in Zoom.

You can see the remote camera now showing the host’s OBS scene of their webcam nested with their screen share.

3. Mix audio from multiple inputs through to multiple outputs

Without causing feedback loops.

Setting up the video was relatively straight forward. Audio is the complicated beast. So far I have only solved this on Windows, but am quite certain you could solve it in Linux using the JACK Audio Connection Kit (JACK).

If you’re running a Mac you should be able to use Rogue Amoeba’s Loopback app to perform similar (and maybe even more advanced) audio filtering. Thanks Matty Stratton for the tip.

Basically we need to be able to selectively route certain inputs to certain outputs. For instance we want zoom audio to go into OBS, and we want OBS audio to go into Zoom, but we don’t want them to loop eachothers audio back and create a feedback loop.

In order to solve this, you need a Mixer that can mix three discreet inputs (Zoom, Desktop, Mic) to three discreet outputs (Speakers, Zoom, OBS).

You also need virtual audio devices that can act as patch cables.

There’s likely a number of different audio solutions out there to solve this, I’ll show you what I found works really well and while not free, is extremely affordable.

This is where VB Audio comes in. It’s a website with an interesting assortment of audio software for Windows, most of which is either Free or Donationware (read the licenses carefully before using it in a commercial setting).

VB Audio offers virtual cables that we can use to patch the sound correctly. They have up to 5 cables that you can purchase via Donation. In order to wire up our sound we’ll use VB-CABLE A+B and VB-CABLE C+D which gives us 4 virtual cables.

All audio sent to VB-CABLE inputs is sent to the corresponding VB_CABLE output.

Pay for, download, and install both bundles.

You will still need a Mixer capable of mixing the inputs correctly. Thankfully VB Audio also provides VoiceMeeter. It comes in three flavors, VoiceMeeter, VoiceMeeter Banana, and VoiceMeeter Potato.

The regular VoiceMeeter does not have enough inputs and outputs, so you’ll want to use either Banana or Potato. I used Banana as it has exactly the right number of inputs.

Pay for (Donate), download and install VoiceMeeter Banana.

Now you have the Mixer and the Virtual Cables we can hook everything up together.

Configure VoiceMeeter Banana

First open up VoiceMeeter Banana. You’ll see there are Three Hardware Inputs and Three Hardware outputs.

Hardware Inputs:

You can rename the Inputs by right clicking on their names, and you can select their devices underneath.

Rename Hardware input 1 to Zoom and set it to WDB: CABLE-B Output.
Rename Hardware input 2 to Mic and set it to your microphone device. Mine is Microphone (Yeti).
Rename Hardware input 3 to Desktop and set it to WDB: CABLE-D Output.

Hardware Outputs:

*Set the outputs by clicking the A1, A2, A3 dropdowns next to where it says HARDWARE OUT *

A1 -> Speakers / Headphones
A2 -> WDM: Cable-A Input (Zoom)
A3 -> WDM: Cable-C Input (OBS)

Mixing:

Under the Hardware Inputs you can see a Volume selector and a sound activity bar. Beside that you can see options, the outputs to send to as well as Mute.

Zoom -> A1 (Speakers), A3 (OBS)
Mic -> A1 (Speakers), A2 (Zoom), A3 (OBS)
Desktop -> A1 (Speakers), A2 (Zoom), A3 (OBS)

*I usually set the Desktop audio volume to about -33 so that random noises don’t drown out people speaking.

Configure Desktop Audio

In your status bar select the Speaker Icon and configure it to output to Cable-D Input

Configure Zoom audio

In Zoom, go to Audio Settings. Set Speaker to Cable-B Input and set Microphone to Cable-A Output.

Configure OBS settings

in OBS under Settings -> Audio Disable all Audio devices. Then set Mic/Auxillary Input to CABLE-C Output.

Then in the OBS Audio Mixer rename the Mic/Auxillary Input to VoiceMeeter and mute all other devices such as webcams.

Test it out

You should now have everything wired up correctly and just need to test it out.

When you configured the Mic input in VoiceMeeter you chose to send it to all three outputs. This means that you should hear your Mic input through your speakers/headphones.

If you don’t like to listen to yourself you can deselect A1 for the Mic input. This is an easy toggle to see if the audio is working.

You can also use the volume meters in the various apps to confirm they match. For instance when you speak in the Microphone you should see the Mic meter, and the A1 , A2 , A3 meter move at the same time. You should also see activity in the OBS mixer and Zoom.

You should hit Start Recording in OBS and get on a zoom with a friend (or a second computer) and verify sound is working in both directions, and that all of the audio ends up in the OBS recording.

Conclusion

Assuming you didn’t hit any issues following this guide you now have your system configured to Stream to both Twitch (or YouTube) and Zoom at the same time, mixing their audio and video appropriately.

This gives you a powerful way to not just record content, but to stream it and share it with both intimate groups on Zoom and Larger audiences on Twitch.

One of the best parts of this setup is how I can easily host someone else on my twitch stream and have them share their desktop and do demos, live coding etc.

I can also run workshops through zoom and people on my twitch stream can also follow along.

Streaming from Zoom to Twitch (or YouTube)

Paul Czarkowski — Wed, 01 Apr 2020 00:00:00 +0000

Zoom meetings are great for a small number of people, and if you’re willing to buy a Webinar license you can also use it for a full on webinar (although there are some interesting restrictions to webinars, like you can’t have breakout rooms).

Twitch is a great streaming platform, originally used for streaming video games, its now more commonly being used to stream live coding and another technology related things that involve sharing your screen.

Traditionally you’d use software called Open Broadcaster Software (OBS) which you can compose multiple video/audio inputs into a single video that can be streamed to Twitch. This is how folks will compose their xbox, a chat screen and a webcam together in their stream.

However OBS is quite CPU hungry (I think a GPU might help) and by the time you run OBS on a standard machine its tough to run anything else without starving OBS of resources and causing stuttering on the stream.

Its made worse if you want to have multiple people on the stream and you end up running something like Zoom which allows you all to chat, and one or more of you to share your screen via zoom, then you just push Zoom itself through OBS.

But if you’re already streaming video via Zoom, wouldn’t it be great if you could just use Zoom to stream directly to Twitch and cutout the middle man? Turns out you can!

Setting up Zoom to stream to Twitch

Preamble aside, its fairly easy, but completely unintuitive to stream Zoom to Twitch. So lets talk it through. First of all you do need a paid [Pro, Business, Education, or Enterprise] account, hopefully you have that through your workplace.

Log into your organization’s Zoom, and hit the Settings button on the left hand menu. Scroll down to Advanced settings and turn Allow live streaming meetings on. Check either the YouTube or the Custom Live Streaming Service , the latter which we’ll configure for Twitch.

Since I’m bad at remembering things, I throw the following in the instructions text box so I remember when setting up the meeting how to do it.

stream url: rtmp://live.twitch.tv/app
stream key: <your twitch account stream key>
live streaming url: https://twitch.tv/accountname

Then just hit Save. You’re now ready to stream. You can either set up the actual stream live in a Zoom meeting, or you can Schedule a meeting and configure it there. Let’s take a look at the latter.

Schedule a Zoom meeting with Twitch Streaming

In your Zoom account on the zoom website click Schedule a meeting.

Set a name, the start time, duration etc and click Save.

This will drop you to the settings page for the meeting you just scheduled and on the bottom of that page is the Live Streaming configuration setting. Click configure live stream settings.

Here you enter the details following the instructions you conveniently placed in the Stream settings:

You can find the twitch key at the following URL https://dashboard.twitch.tv/u//settings/channel

Hit Save. Now when you join the Meeting it will automatically configure the Live Stream, but you’ll still need to start the Stream by clicking the … More button and select the streaming option from there.

If you didn’t configure live streaming for the meeting, you can actually configure it here as long as you’ve enabled streaming in Settings

Zoom will redirect you to a browser and configure the stream before redirecting it to your twitch streaming page.

You can close this twitch streaming window once its started or leave it open. If you leave it open turn off its sound or you’ll create some fun echo loopbacks.

Here’s the Twitch view:

Here’s the Zoom View: