DEV Community: Chris Harris

Container Apps & Metabase Saved Big Bucks

Chris Harris — Thu, 10 Nov 2022 17:53:30 +0000

I've got another post that I wrote on my Substack that I think is particularly relevant to small businesses with their data in the cloud. While my journey took place on Azure, the same approach can apply to any cloud provider.

I cover my experience starting with Azure Kubernetes Services and ending up on Azure Container Apps and the $180 per month it saved me!

Substack: Serverless Metabase Saved Big Bucks

PostgreSQL to DuckDB - There and Quack Again

Chris Harris — Sun, 23 Oct 2022 14:12:14 +0000

Edit -
I'm using this platform to primarily talk about things related to my day job, but this particular post isn't as closely tied to it as the other posts I've made, so I moved it to a new substack:
https://affordabledata.substack.com/p/postgresql-to-duckdb

Make Better Decisions the Easy Way: Deploy Metabase with Azure Container Apps

Chris Harris — Wed, 12 Oct 2022 17:18:33 +0000

With a few simple steps, you can deploy Metabase on Microsoft Azure using Azure Container Apps. This process works for any Docker container hosted on Docker Hub, not just Metabase, so you can try it with your containers.

Background

I work on improving the Developer Experience on Microsoft Azure and I love spending time in the data engineering and data analysis space. I recently have enjoyed working with Metabase, and deploying it on Azure makes it easy to create some amazing visualizations for my data that is already in the Azure cloud.

There are several options for running container-based apps on Azure, but Azure Container Apps (ACA) is one of the easiest to get started with.

The set up

Prerequisites you will need:

Azure CLI
An Azure subscription. If you don't already have one, you can create one for free.
Docker Desktop

Deploy Metabase to Azure Container Apps

Create the Container App and deploy the Metabase image
```
az containerapp create -g metabase-rg -n metabase --image [myname]metabaseacr.azurecr.io/metabase --environment metabase-aca-env --target-port 3000 --ingress 'external' --cpu 2 --memory 4.0Gi
```
Note: If prompted to install the containerapp extension, choose Y.

You will see the steps as they are running, like 'Creating resource group 'your-resource-group-name' and Creating Containerapp your-app-name....

When the command is complete you will see a line that starts with Browse to your Container App at: http://metabase. You will use this URL to open your new Metabase app, but if you do it right now, you'll likely see a stream timeout error because Metabase needs more resources than the default .5 CPU and 1GB memory Azure Container Apps allocates. The next step will help you update the resources and get the app running.
Update the resources for your Container App
To increase the CPU and memory resources allocated to your new Container App, just run the following command:
```
az containerapp update --resource-group metabase-rg --name metabase --cpu 1 --memory 2.0Gi
```

Open Metabase and explore

Paste the URL from Step 1 in a browser to start enjoying Metabase!

If you need that URL again, you can use this command to quickly find it:
```
az containerapp show -g metabase-rg -n metabase --query properties.configuration.ingress.fqdn
```
Use the Metabase Getting Started Guide or search for Metabase on YouTube if you need help getting started.

Cleanup the resources

When you are done exploring Metabase, don't forget to delete the resources you created if you aren't going to use them any longer.

az group delete -g metabase-rg

Next Step - Set up your Container App for production use

Metabase has a nice doc for configuring Metabase to be used in production: How to Run Metabase in Production. One of the steps is to migrate from the internal H2 database to PostgreSQL, and you can use Azure Database for PostgreSQL for this purpose.

You will need to set some environment variables in your Container App to tell Metabase which database to talk to. Azure Container Apps lets you do this easily through the Azure Portal or through the Azure CLI.

If This step is interesting to you, let me know and I'll add another article.

Conclusion

I'm still learning about Metabase myself, but I've been super impressed so far. I would love to hear what you think, and if you run into problems, don't hesitate to drop into the Metabase Discussion and bring your questions.

I want to give a shout-out to the Metabase Discussion site. I've had a number of questions about deploying and using Metabase that I've posted on the site. Most of my questions have been answered within hours and no question has gone unanswered. Amazing support!

And if you want a little more information on running Metabase in a Docker container, Metabase has a very nice doc that got me off to a good start: Running Metabase on Docker

Apache Superset and Azure - multi-container application deployment

Chris Harris — Tue, 10 May 2022 17:13:24 +0000

In this article you’ll learn the steps to deploy Apache Superset on Azure using Azure Kubernetes Service.

Background

I work on improving the Python Developer Experience on Microsoft Azure and I've spoken with lots of Python developers from around the world. One thing I've heard repeatedly is that many Python devs like to run production code in containers. And another thing I've learned is many apps require multiple containers.

I also like to do some data analysis on the side and recently ran across Apache Superset which describes itself as a "modern data exploration and data visualization platform". Coincidentally, Superset has a lot of Python code and can be deployed in containers (nine of them at current count!)

There are several options for running container-based apps on Azure. Because Superset requires several containers (9 at current count) I decided to use Azure Kubernetes Service (AKS). In a future post I plan to share how you can use Azure's new managed service for containers, Azure Container Apps, instead.

While I did not find an existing document or post to help me complete my journey in a straightforward manner, here are several docs that helped:

The set up

Some of the syntax below may vary with your choice of OS or shell.

Prerequisites you will need:

Git
Azure CLI
Helm CLI
An Azure subscription. If you don't already have one, you can create one for free.
Docker Desktop

Get the Superset app and create your AKS Cluster

Clone the Apache Superset repo

git clone https://github.com/apache/superset

cd superset

Sign in to the Azure CLI
You can sign in by using the az login command
```
az login
```

Create a new AKS cluster with ACR integration
Azure Container Registry (ACR) is kind of like Docker Hub. It's a place to store your container images so you can run your application in your Azure Kubernetes Service (AKS) cluster. We will use it to store the Superset container images.
Use az acr create to create an ACR named [yourname]supersetacr in a resource group called supersetrg.
Replace [yourname] below with a name of your choice.

# Create a Resource Group to hold your ACR and AKS resources
# Feel free to use a location closer to you
az group create supersetrg --location westus2

# Create an Azure Container Registry
az acr create -n [yourname]supersetacr -g supersetrg --sku basic

# Create an AKS cluster with ACR integration
az aks create -n supersetaks -g supersetrg --generate-ssh-keys --attach-acr [yourname]supersetacr

When you create the ACR, you will see a blob of JSON. Here are a couple of important values to notice:

...
"location": "westus2",
"loginServer": "[yourname]supersetacr.azurecr.io",
"name": "[yourname]2supersetacr",
...
"provisioningState": "Succeeded",
"publicNetworkAccess": "Enabled",
"resourceGroup": "supersetrg",
"sku": {
  "name": "Basic",
  "tier": "Basic"
},
...

It will take a few minutes to create the AKS cluster. When it's done you will see an even larger blob of JSON. Here are a couple of important values to notice in this one:

...
  "osDiskSizeGb": 128,
  "osDiskType": "Managed",
  "osSku": "Ubuntu",
  "osType": "Linux",
...
  "provisioningState": "Succeeded",
...
  "vmSize": "Standard_DS2_v2",
...
"azurePortalFqdn": "supersetak-supersetrg-2223f9-06aacbbd.portal.hcp.westus2.azmk8s.io",
...
"fqdn": "supersetak-supersetrg-2223f9-06aacbbd.hcp.westus2.azmk8s.io",
...
"kubernetesVersion": "1.22.6",
...
"nodeResourceGroup": "MC_supersetrg_supersetaks_westus2",
...

Additional ways to integrate ACR with AKS: 3 Ways to integrate ACR with AKS

Push the Superset container images into your ACR
Superset uses two container images, which you can see in the superset repo in the /helm/superset/values.yaml file:

image: 
  repository: apache/superset
...
initImage: 
  repository: busybox

While ACR can technically pull images directly from Docker Hub, the throttling that Docker Hub has recently implemented means it could take a while (sometimes a long while) for the images to get into your ACR. Instead we're going to pull the images to your machine and then push them to ACR. Don't forget to replace [yourname] with the name you chose previously.

# First - login to your ACR so Docker can push to it
az acr login -n [yourname]supersetacr.azurecr.io

# Pull the Superset image
docker pull apache/superset
# Tag the image using your ACR login server name 
docker tag apache/superset [yourname]supersetacr.azurecr.io/superset
# Push the image to ACR
docker push [yourname]supersetacr.azurecr.io/superset

# Pull the Busybox image
docker pull busybox
# Tag the image using your ACR login server name
docker tag busybox [yourname]supersetacr.azurecr.io/busybox
# Push the image to ACR
docker push [yourname]supersetacr.azurecr.io/busybox

Create a my_values.yaml file to override defaults
In the previous step we gave the images new tags and now we need to create override the defaults in /helm/superset/values.yaml so our new images will be used.
Add the following to a new file called my_values.yaml, being sure to replace [yourname] with the name you chose previously:
```
image:
  repository: [yourname]supersetacr.azurecr.io/superset

initImage:
  repository: [yourname]supersetacr.azurecr.io/busybox
```
While we're in the my_values.yaml file, in order to expose the Superset website, we need to change service type from ClusterIP to LoadBalancer and to make it easier to browse we will set the port to 80. Add these lines to my_values.yaml as well:
```
# Set type to 'LoadBalancer' so we can browse it
service:
  type: LoadBalancer
  port: 80
```
And unless you're planning on deploying to production, I recommend that you load up some examples to play with. Just make one more addition to the my_values.yaml file:
```
# Load Superset Examples
init:
  loadExamples: true
```
Here is the entire my_values.yaml file:
```
image:
  repository: [yourname]supersetacr.azurecr.io/superset

initImage:
  repository: [yourname]supersetacr.azurecr.io/busybox

# Set type to 'LoadBalancer' so we can browse it
service:
  type: LoadBalancer
  port: 80

# Load Superset Examples
init:
  loadExamples: true
```

Deploy Superset to your AKS cluster

Get the credentials to your AKS cluster so helm deploy to it
Use the az aks get-credentials command to download credentials for your AKS cluster:
```
az aks get-credentials -n supersetaks -g supersetrg
```
Deploy to AKS using helm
First update your helm chart dependencies using the helm dependency update command:
```
helm dependency update helm/superset
```
Then install your helm chart using the helm install command:
```
helm upgrade --install --values my_values.yaml superset helm/superset
```

Check it out!

View your AKS cluster in the Azure Portal and open Superset
Some Azure services make it very easy for you to quickly jump from your CLI to view the resource in the Azure web portal; fortunately, AKS is one of those services.

Open the portal to your AKS cluster with the az aks browse -n supersetaks -g supersetrg command.
This will take you to the Workloads page where you will see the status of your Kubernetes workloads. Give it five minutes or so for everything to get deployed and to turn green.

Then click on Services and ingresses.

On the Services and ingresses page, you will find the External IP for your Superset website:

When you click on the External IP and add the port number :8088, if everything worked perfectly, you will see the Superset login page below where you can use admin and admin to login.

Conclusion

While I haven't tried out everything yet, I have connected to an external database and created charts and dashboards. I'm happy to say that performance has been great so far and I'm looking forward to seeing what I can create!
Here's one of the dashboards you can explore when you load the examples mentioned above:

Cleaning up the resources

Since AKS is not free, when you are done testing Superset you may want to delete the resources.

When you created your ACR, you created a resource group called supersetrg. And when you created your AKS cluster another resource group was also created with a name similar to this: MC_supersetrg_supersetaks_westus2.

You will need to delete both of these resource groups to prevent incurring additional costs.

# View all of your resource groups
az group list -o table

# Delete the resource groups for your AKS cluster
az group delete -g supersetrg
az group delete -g MC_supersetrg_supersetaks_westus2

Next steps

Consider creating an Azure Database for PostgreSQL and connecting it to Superset
Once you have a database connected, you can upload some data to Superset from a CSV or Excel file
Learn how to Explore Data in Superset
Learn how to Create your First Dashboard