DEV Community: David Truxall

Parameter Position in OpenAPI

David Truxall — Wed, 27 Nov 2024 00:54:00 +0000

OpenAPI provides a terrific UI to use for testing, learning and using your APIs. My team has built some APIs using Quarkus, and used their implementation of the SmallRye OpenAPI library. The problem I’ve run into is that the parameters of any given API call come out in alphabetical order, not the order specified in the code. Here’s a sample of the code with some OpenAPI metadata:

@GET
@Path("/range")
@Operation(summary = "Get a list of resources inclusive for a range of dates")
public RangeResult getRecordsByRange(
        @Context UriInfo urlInfo,
        @Parameter(description = "The starting date", required = true, example = "yyyy-MM-dd")
        @QueryParam("startDate") String startDate,
        @Parameter(description = "The ending date, inclusive of records generated on that day", required = true, example = "yyyy-MM-dd")
        @QueryParam("endDate") String endDate,
        @Parameter(description = "The current data page, using a zero-based index. Default value is zero (the first page)", required = false, example = "2")
        @QueryParam("page") int page,
        @Parameter(description = "The number of records per page (max 1000). Default value is 100.", required = false, example = "100")
        @QueryParam("pageSize") int pageSize) throws ApiException {
}

Unfortunately, that code results in this:

This doesn’t make sense to me for multiple reasons. First, the endDay parameter is before the startDay parameter. Second, those same parameters are required and they occur first and last. To make this more intuitive for the user, I’d want to group the required parameters at the beginning, and ensure that startDay comes before endDay.

The solution was suggested in a Github issue for Quarkus, I did not find the solution elsewhere, and it certainly was not intuitive. The answer was to not place the @Parameter metadata inline with the function parameters (like all the OpenAPI examples I could find). By separating them from the function definition and placing them above the function, OpenAPI respected the order. I changed the code to this:

@GET
@Path("/range")
@Operation(summary = "Get a list of resources inclusive for a range of dates")
@Parameter(description = "The starting date", required = true, example = "yyyy-MM-dd")
@Parameter(description = "The ending date, inclusive of records generated on that day", required = true, example = "yyyy-MM-dd")
@Parameter(description = "The current data page, using a zero-based index. Default value is zero (the first page)", required = false, example = "2")
@Parameter(description = "The number of records per page (max 1000). Default value is 100.", required = false, example = "100")
public RangeResult getRecordsByRange(
        @Context UriInfo urlInfo,
        @QueryParam("startDate") String startDate,
        @QueryParam("endDate") String endDate,
        @QueryParam("page") int page,        
        @QueryParam("pageSize") int pageSize) throws ApiException {
}

That change resulted in the below image, which is exactly what I wanted:

Exposing Kafka in Docker Desktop Kubernetes

David Truxall — Wed, 03 Apr 2024 00:06:00 +0000

In an effort to get more fluent with Kubernetes, I’m using it instead of Docker/Docker compose during my local development. I’m using Docker Desktop on a Mac, so it’s going to be slightly different than minikube or on Windows.

In most cases I can take a Docker Compose file and recreate it as a services and deployments that my code can interact with. But in the case of Kafka, there are a couple complexities that I needed to iron out to get it running. The two main problems were exposing the Kafka UI web app from inside the cluster, and exposing Kafka in a way that both Kafka UI and applications running outside the cluster (maybe from an IDE or command-line) can access topics.

The first problem is to expose Kafka both inside and outside the cluster. Inside the cluster it gets used by Kafka UI, and outside the cluster it gets used by the code I’m creating. The trick is in both the Service and Deployment definitions. In the Service definition, I set the type to NodePort, but I define ports for both internal and external access, with only the external access using a nodeport:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: kafka-broker
  name: kafka-service
  namespace: dev
spec:
  type: NodePort
  ports:
  - port: 9092
    name: inner
    targetPort: 9092
    protocol: TCP
  - port: 30092
    name: outer
    targetPort: 30092
    nodePort: 30092
    protocol: TCP
  selector:
    app: kafka-broker

Setting up the broker is more complex. First, we make an alias for PLAINTEXT:

- name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP value: "INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT"

Then, make sure the container is exposed on two ports:

ports:         
- containerPort: 9092         
- containerPort: 30092

Port 9092 is the standard Kafka port, and is used inside the cluster. Port 30092 is in the accepted range for nodeports, and is the port used outside the cluster. Now, we tell Kafka to listen on those ports using the aliases, and advertise listeners both inside the cluster and using the domain we initially defined:

- name: KAFKA_INTER_BROKER_LISTENER_NAME
  value: "INSIDE"
- name: KAFKA_LISTENERS
  value: "INSIDE://:9092,OUTSIDE://:30092"
- name: KAFKA_ADVERTISED_LISTENERS
  value: "INSIDE://$(KAFKA_SERVICE_SERVICE_HOST):9092,OUTSIDE://kafka.example.net:30092"

Here’s the full Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: kafka-broker
  name: kafka-broker
  namespace: dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka-broker
  template:
    metadata:
      labels:
        app: kafka-broker
    spec:
      hostname: kafka-broker
      containers:
      - env:
        - name: KAFKA_BROKER_ID
          value: "1"
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: $(ZOOKEEPER_SERVICE_SERVICE_HOST):2181
        - name: KAFKA_INTER_BROKER_LISTENER_NAME
          value: "INSIDE"
        - name: KAFKA_LISTENERS
          value: "INSIDE://:9092,OUTSIDE://:30092"
        - name: KAFKA_ADVERTISED_LISTENERS
          value: "INSIDE://$(KAFKA_SERVICE_SERVICE_HOST):9092,OUTSIDE://kafka.example.net:30092"
        - name: KAFKA_LISTENER_SECURITY_PROTOCOL_MAP
          value: "INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT"
        - name: KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR
          value: "1"
        - name: KAFKA_TRANSACTION_STATE_LOG_MIN_ISR
          value: "1"
        - name: KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR
          value: "1"
        image: confluentinc/cp-kafka:7.6.0
        imagePullPolicy: IfNotPresent
        name: kafka-broker
        ports:
        - containerPort: 9092
        - containerPort: 30092

The next problem, exposing Kafka UI, is not unique to Kafka, but is a problem for any web app you run inside the Docker Desktop cluster. I didn’t want to use port forwarding. While I know that works, it’s not as convenient as just accessing a URL.

The first thing I set up was URL resolution in my hosts file. I edited my /etc/hosts file and added the following line:

127.0.0.1 kafka.example.net

I have taken to usingexample.net for my cluster resources, and example.com for my Docker Compose resources. Both those domains are reserved and won’t conflict with actual domains.

The next step is to install an nginx ingress into my local cluster:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.10.0/deploy/static/provider/cloud/deploy.yaml

Now I can set up Kafka UI:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: kafka-ui-service
  name: kafka-ui-service
  namespace: dev
spec:
  ports:
    - name: kafka-ui-port
      port: 80
      targetPort: 8080
  selector:
    app: kafka-ui
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: kafka-ui
  name: kafka-ui
  namespace: dev
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kafka-ui
  template:
    metadata:
      labels:
        app: kafka-ui
    spec:
      hostname: kafka-ui
      containers:
      - env:
        - name: KAFKA_CLUSTERS_0_NAME
          value: "local"
        - name: KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS
          value: $(KAFKA_SERVICE_SERVICE_HOST):9092
        image: provectuslabs/kafka-ui:master
        imagePullPolicy: IfNotPresent
        name: kafka-ui
        ports:
        - containerPort: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kafka-ui-ingress
  namespace: dev
spec:
  ingressClassName: nginx
  rules:
  - host: kafka.example.net
    http:
      paths:
      - backend:
          service:
            name: kafka-ui-service
            port:
              number: 80
        path: /
        pathType: Prefix

I’ve placed thecomplete yaml file in a repo, and this post sent me in the right direction for my solution.

Critical Thinking in Software Design

David Truxall — Wed, 07 Sep 2022 13:42:15 +0000

Inc published a great article a while back which enumerates four secrets of great critical thinkers. I find the concepts in this article are totally relevant to designing software or software architecture even though that's not what the article is about. So I’m adding my programmer's take on these four secrets:

Slow Down

One of my main gripes about meetings is making a decision on the spot. At least for me, my best thinking occurs somewhere else after the fact. I need time to consider the implications and possibly see alternatives before I can be certain it’s the correct decision. So to make the most effective decisions, postpone immediate decisions as often as possible. Make the time to explore the alternatives.

Break from the Pack

This is the basis of one of the main arguments by Ted Neward in his keynote at Codemash 2012 that I already covered. The “best practices” are found in books, on the web, in corporate documents. Generally, these can get the job done. But I doubt it’s always the best or right solution to a problem. It’s certainly the safest. If that’s the goal, then go that way. But in business we are always looking for an advantage, and I doubt the best practice gives you the advantage. More likely, it’s a starting point which you should use to find the best solution. Slow down and think about how the best practice solves the problem and if that is really the solution you need.

Encourage Disagreement

I have long used this as a metric in my interview process. I want team members who will express their opinions. Usually the best answer to a problem is not the idea of a single person, but an amalgamation of ideas or compromise between multiple ideas. If no one can disagree with the boss/tech lead/manager then you might as well adopt only the best practices and call it done.

Engage with Maverics

I think this one is a little harder in an enterprise environment, since often a big corporation is not the home for maverics. I think what you need to do here is actively engage with people whose viewpoints are not aligned with yours. People who think differently about a problem may provide you with some insight you may have missed or not considered properly. I have always thought of code reviews as a source for this. I approach code reviews as a way to see someone else’s problem solving at work. I’m not interested in telling them how they did it wrong, but seeing what they did well and adapt that to my problem solving toolkit.

I try to revisit these topics and remind myself to practice these concepts as I do my work, it's too easy to get lost in the moment writing code. I hope you can do the same.

GitHub Setup on a New Mac

David Truxall — Tue, 23 Aug 2022 22:00:00 +0000

I recently acquired a new Mac, and needed to set up git with access to Github on the new computer. I decided to document the process.

First, create a new ssh key, adding a password:

ssh-keygen -t rsa -b 4096

This key ends up in the .ssh folder of your home directory: ~/.ssh/rsa_id

Now, copy the public key to the clipboard to add it to GitHub:

cat ~/.ssh/id_rsa.pub | pbcopy

In GitHub, navigate to the account settings, and choose SSH and GPG keys from the left navigation menu. Click the New SSH Key button. Type a Title for the key, and paste the clipboard contents into the Key field. Click the Add SSH Key button to save it.

Configure git locally, using the email address from the GitHub account:

git config --global user.email "<YourEmail@somewhere.com>"
git config --global user.name "<YourUserName>"

Instead of typing a password for every use of git, store the password in the keychain locally. In the .sshfolder, create a file named config:

touch ~/.ssh/config

Open the config file in an editor, and add the following content:

Host * UseKeychain yes AddKeysToAgent yes

The next push to GitHub will prompt for the password, and from that point on it’s stored in keychain and git won’t prompt for the password any more.

Since I am using GitHub, I use a GPG key to verify my identity when I commit and push code. I can see my commits are verified, protecting me against someone spoofing pushes to my repos:

The first step is creating a GPG key. I installed and used Mac GPG Tools to create my key. After installation the program launches. Create a new key using a strong password and the same email address used for GitHub:

Choose a strong passphrase:

After creating a passphrase for the key, install the public key at GitHub. First, copy the public key to a file using the email address used while creating the key, then copy the file contents to the clipboard:

gpg --export --armor YourEmail@somewhere.com > public-key.asc
cat public-key.asc | pbcopy

In GitHub, navigate to account settings, and choose SSH and GPG keys from the left navigation menu. Click the New GPG Key button. Type a Title for the key, and paste the clipboard contents into the Key field. Click the Add GPG Key button to save it.

Next, configure git to use the GPG key. First, find the ID for the key:

gpg --list-secret-keys --keyid-format=long

The output should look like this:

/Users/username/.gnupg/pubring.kbx

sec rsa4096/EAF3888888888888E 2022-07-17 [SC] [expires: 2026-07-17]
919488888888888888888888888888888888888E
uid [ultimate] Your Name YourEmail@somewhere.com
ssb rsa4096/A888888888888884 2022-07-17 [E] [expires: 2026-07-17]

On the line below the one with your email address, copy the text after the rsa4096/ and before the date generated. Use that ID to configure git to sign commits:

git config --global user.signingkey A888888888888884

Now, when ready commit some code destined for GitHub, add a new parameter to the command: -S

git commit -S -m "A clear commit message"

The first time committing with the new parameter, Mac GPG prompts for the passphrase set on the key above. You can choose to save the passphrase in the keychain for future commits.

Now my machine is set up to work with GitHub and verify my identity on my commits.

Deploy A Vue App in Docker Without Root

David Truxall — Wed, 17 Aug 2022 00:44:26 +0000

So you’re going to deploy your Vue app in a Docker container. That great!! Containers are a fantastic way to deploy your app. When I deploy Vue apps, I choose nginx as the web server. nginx is available as a Docker image from Dockerhub, so you don’t need to do much to get started. Unfortunately the default implementation runs in the context of the root user. This can be a security problem, especially if the container gets breached. The attacker is now running as root.

Unfortunately, it’s not quite as simple as just changing the user in the Dockerfile. The reason the nginx image runs as root is that in Linux, the user must be root in order to run the app on port 80 or 443. We can make the changes to the container to make this possible, but the changes are complex. Luckily we are using a container, so the actual port the web server runs on in the container is just not relevant. So we can run the app in the context of a non-root user on any other port (like 8080 for instance). When running the container, we can map back to port 80 or 443 for production deployments if we need to expose the app directly to the Internet. In my case, the SSL/TLS certificate is hosted either in a reverse proxy or a Kubernetes ingress, so I am not including the certificate in my Docker images.

The first thing we need to change is the main configuration file for nginx. We want it to listen on another port, this time it’s going to be 8080. The rest of the configuration is a default setting:, but it could be there if we are exposing the app directly on port 443:

server {
  listen 8080;
  server_name localhost;

  location / {
    root /usr/share/nginx/html;
    index index.html index.htm;
    try_files $uri $uri/ /index.html;
  }

  error_page 400 500 502 503 504 /50x.html;
  location = /50x.html {
    root /usr/share/nginx/html;
  }
}

Next, we need to change the user context nginx runs under. Luckily, the nginx folks have thought about this, and already created a user called nginx right in the default container, so there’s no system-level user configuration necessary. Here’s the complete Dockerfile:

FROM nginx:1.19

RUN rm -f /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d

RUN chown -R nginx:nginx /var/cache/nginx && \
    chown -R nginx:nginx /var/log/nginx && \
    chown -R nginx:nginx /etc/nginx/conf.d

RUN touch /var/run/nginx.pid && \
    chown -R nginx:nginx /var/run/nginx.pid

USER nginx

COPY dist /usr/share/nginx/html

EXPOSE 8080

Let’s look at the relevant parts of the Dockerfile. There are a few directories where the nginx user must have ownership rights for logging, caching and configuration, as well as the process ID file:

RUN chown -R nginx:nginx /var/cache/nginx && \
    chown -R nginx:nginx /var/log/nginx && \
    chown -R nginx:nginx /etc/nginx/conf.d

RUN touch /var/run/nginx.pid && \
    chown -R nginx:nginx /var/run/nginx.pid

We set the user context next, so nginx runs under this user:

USER nginx

Then the Dockerfile copies the contents of the dist folder into the image. This is the output from building our Vue app with npm:

COPY dist /usr/share/nginx/html

And lastly we set the port, which can’t be 80 or 443:

EXPOSE 8080

Now our Dockerfile is set to create a container that is not running with root privileges. The app can be run over 80 or 443 using Docker, a Kubernetes ingress, or even a reverse proxy, with a smaller amount of risk than using the defaults.

Breaking Production

David Truxall — Thu, 24 Jun 2021 15:00:03 +0000

Recently, an intern at HBO Max mistakenly sent a test email to thousands of users in production. Twitter was on fire with memes and jokes, but I found the threads about developers’ experiences breaking production to be much more interesting. Most long-time developers experience the gut-wrenching experience of breaking production, and I’m no different. My story is more complicated than I could get in a Tweet, so I’m telling the story here.

The year was 2003. I was working at a company in the Detroit area migrating a classic ASP application to ASP.Net. The site “just had constant errors” that no one could elaborate on, and needed a thorough overhaul. The company decided to use the need to rewrite code as an opportunity to move to .Net since Microsoft was moving way from Classic ASP at that time. Unfortunately for me, there was no logging in the existing system, and all the exceptions were unhandled so my team had no idea what was breaking or when it was happening. To try and get a grip on the problem, we wrote code to hook into the global error handler and have the server send everyone on the team an email with the error details so we could start to understand the problems and frequency by getting real-time alerts. There were no Sentry.io/Crashlytics/LogRocket services or the like at that time, so we built our own. We were testing this feature, and had not rolled it out to production yet. It was only in our development environment.

The day was August 14, 2003. That afternoon, there was a severe blackout across the northeastern US. When it was clear the power was not coming back on, the company sent us home. For us the blackout lasted days. And this is where my failure begins.

The system I was working on was backed by an Oracle database, which had a dedicated administrator. The web servers had a different person administering them. During the blackout, the data center, which was located at our facility in Detroit, was running off of generators to keep services available to users outside the blackout area. The database administrator decided to shut down the development database server to conserve energy. But the web server admin kept the development web server running. Unbeknownst to me, the web server admin was running a tool that pinged the home page of the development web site every 8 seconds to make sure it was still alive. Unfortunately, the home page of the web site accessed the database, which was turned off. This caused an unhandled exception on the home page. We already know the site was not handling any exceptions. So the error fired the global exception handler and sent me and my team an email about the error. Every 8 seconds. For two days, because no one on my team was working during the blackout to see the emails.

We returned to the office when power was restored. No one in the company could get email though. That system was still down. Turns out it was down because of my code. The handler sent 56,000 emails during the blackout and filled the disk of the Novell GroupWise email server. Back in 2003 disks were no where near the size we use today. The email administrator was furious. No administrative tool existed for her to remove all those emails. My team had to sit for hours and hours and hours deleting emails using the desktop client. Which was limited to only selecting 100 messages at a time. We certainly did our penance.

You might be thinking that the circumstances of the blackout caused the problem, not really a mistake in the code. But the fault was mine. I should have considered throttling in the error handler. It should have stopped sending the same error message repeatedly. There is no value seeing the same error over and over and over, especially over a short period of time. This was of course the first thing I fixed after deleting all the emails. It’s a hard-earned lesson I’ll never forget.

I may have broken production at other times, but nothing as dramatic and difficult to recover from as this. If you are a junior person reading this, I can assure you that some day your code will break something. But you are not alone, it happens to all of us. I hope your breaks are minor and less consequential, but I know what you are feeling. Feel it and then take that lesson to heart and you’ll become a better developer.

Secure a Vue.js App with KeyCloak

David Truxall — Tue, 30 Mar 2021 10:00:00 +0000

In a recent project I needed to secure a long-standing Vue.js application with Keycloak, a terrific open source identity and access management tool. Unfortunately, the Vue sample in Keycloak’s documentation is flawed. The sample refreshes the token every minute, whether or not the user performs any actions. I don’t think this behavior is very secure and certainly not going to meet realistic use cases. With this article I’m demonstrating a method more in line with the features of Vue.js.

The use case I’m describing here is one where some pages are public and some pages are protected via authentication. I’ve also added in authorization to the sample. So a user wanting to access protected content will log in using Keycloak (authentication) and also need the correct role assigned in Keycloak (authorization). This use case is similar to having a protected admin section of a public web site. I know there is much more to proper client than what I show here. This simple example is intended to be a starting point. This short video demonstrates the behavior we’ll code in this post.

TLDR; The Steps

Configure a Client in KeyCloak
Capture the Client ID, role name for the Client ID, and Realm name from Keycloak
Create users in Keycloak
Create a plugin in the Vue app
Integrate the plugin into app startup
Set routes to be authenticated
Create a navigation guard that manages authentication and authorization to protected resources.

Keycloak Configuration

If you have not set up or configured Keycloak previously, check out my article demonstrating how to run Keycloak locally using Docker. I’m assuming you are running a Keycloak instance, have admin rights to the Keycloak administration app and have already created a Realm. The main tasks to complete for this example are creating a Client application registration and adding users in Keycloak. If you know how to do this just skip down to the code section, otherwise let’s get to work with Keycloak.

Create A Client in Keycloak

A Client in Keycloak is a way to register an app to be secured. The Vue app we create needs to have a corresponding Client ID. Make sure to keep track of the Realm name for later in our Vue code. Click the Clients link in the left navigation pane under the Realm you are using to see the existing clients and create a new one.

Figure 1. Add a new client to Keycloak

Click the Create button at the top left of the table of existing Clients to add a new Client for your app. In my case, I’m going to create a Vue app to demo logging in with Keycloak.

Figure 2. Name the client and set the root URL

Set the client id as the name of your app, and set the Root URL to the URL you are using for local development of your Vue application. In my case, the URL is http://localhost:8080. Click the Save button. Make sure you get the protocol correct. If you get an http/https mismatch you’ll have issues later on.

In the Client details page, make sure the Client Protocol is set to openid-connectand the Access Type is set to public. Check that the following are set properly:

Create A Role

Each Client needs one or more roles. If you don’t assign roles and check them in your application, any user from your Realm will be able to log into the app. You should create roles to control user access. Click the Roles tab in your Client details page. In the Roles tab, click the Add Role button at the upper right of the table. Give your role a name and save it. We’ll need the role name for later in our Vue app.

Figure 3. Create a role for the new client

Create Users

We need users to log into our application (Client). For this demo we’ll create two users. One user with the role we just created and one user without. We will create a user and then assign a password and role. Click the Users link in the left navigation pane to get started.

Figure 4. Add a Keycloak user

You won’t see any users listed, even if you added some previously. Click the Add user button in the top right of the table.

Figure 5. Add user details

Enter a username, and their actual name and email. Click the Save button. Let’s give them a password.

Figure 6. Set the password for the user

On the user details page, click the Credentials tab. Give the user a password. If you don’t want the user to be forced to change their password, turn the Temporary switch to off. I’ll do this when I’m running locally for development, but not when making accounts for actual users in a non-development environment. Click the Set Password button.

Assign A Role

Since the application (Client) has a role, we need to assign that role to the user so they can log into that application. Click the Role Mappings tab.

Figure 7. Assign a client role to a user

Select your client created earlier in the Client Roles drop down list. The role created earlier should be there. Select that role and click the Add selected > button. The user is good to go now.

Create a second user following the same steps, but do not assign any role to this user. Now, let’s get coding in Vue.

Vue Setup

You can find the sample code I wrote for this article at GitHub. I made a simple application with the Vue-cli and edited that app.

We will use the library provided by Keycloak to build our authentication plumbing. It’s very important that the version of the library you install matches the version of the Keycloak instance you are working with. You can install it using npm, my Keycloak instance is version 12.0.4:

npm install --save keycloak-js@12.0.4

You can also find the JavaScript library in your Keycloak instance. It’s located at <yourServerUrl>/auth/js/keycloak.js

Create A Plugin

In order to make the Keycloak object accessible throughout the application, I created a Vue plugin. The plugin is going to create a global $keycloak object we can reference anywhere in the app. The code for the plugin looks like:

import Vue from 'vue'
import **_Keycloak_** from 'keycloak-js'

const options = {
  url: 'http://localhost:8001/auth/',
  realm: 'local-dev',
  clientId: 'vue-demo'
}

const _keycloak = Keycloak(options)

const **_Plugin_** = {
  install(Vue) {
    Vue.$keycloak = _keycloak
  }
}

**_Plugin_**.install = Vue => {
  Vue.$keycloak = _keycloak
  **_Object_**.defineProperties(Vue.prototype, {
    $keycloak: {
      get() {
        return _keycloak
      }
    }
  })
}

Vue.use( **_Plugin_** )

export default **_Plugin_**

The important bits to note are in the options object. The url must be the Keycloak base server URL using the /auth/directory. The realm and clientId came from the configuration of the Client in Keycloak steps above.

The next step is to initialize the Keycloak object from the plugin before starting the Vue app in main.js. So your main.js file should look something like this:

import Vue from 'vue'
import App from './App.vue'
import **_router_** from './router'
import **_authentication_** from "@/plugins/authentication"

Vue.config.productionTip = false
Vue.use( **_authentication_** )

Vue.$keycloak
  .init({ checkLoginIframe: false })
  .then(() => {
    new Vue({
      **_router_** ,
      render: h => h(App)
    }).$mount('#app')
  })

This only sets up Keycloak. It does not yet protect secured content. If you want your app to always force a login and have no public content, change the .init function to use login-required upon onLoad:

.init({ onLoad: 'login-required', checkLoginIframe: false })

Create A Navigation Guard

In order to secure the pages we want behind a login, we have to create a navigation guard, a feature in Vue intended for just this purpose. But, we need a way to tell which pages are the unsecured and secured pages. We do this by setting a meta tag in our Vue router configuration file:

const routes = [
  {
    path: '/',
    name: 'Home',
    component: Home,
    meta: {
      isAuthenticated: false
    }
  },
  {
    path: '/secured',
    name: 'Secured',
    meta: {
      isAuthenticated: true
    },
    component: () => import('../views/Secured.vue')
  },
  {
    path: '/unauthorized',
    name: 'Unauthorized',
    meta: {
      isAuthenticated: false
    },
    component: () => import('../views/Unauthorized.vue')
  }
]

In the above code the home page and unauthorized message page are not secured. But the page named ‘Secured’ is secured. The navigation guard will check this property and redirect users to login when necessary. So we create this function in the Vue router which is a global navigation guard:

**_router_**.beforeEach((to, from, next) => {
  if (to.meta.isAuthenticated) {
    // Get the actual url of the app, it's needed for Keycloak
    const basePath = **_window_**.location.toString()
    if (!Vue.$keycloak.authenticated) {
      // The page is protected and the user is not authenticated. Force a login.
      Vue.$keycloak.login({ redirectUri: basePath.slice(0, -1) + to.path })
    } else if (Vue.$keycloak.hasResourceRole('vue-demo-user')) {
      // The user was authenticated, and has the app role
      Vue.$keycloak.updateToken(70)
        .then(() => {
          next()
        })
        .catch(err => {
          **_console_**.error(err)
        })
    } else {
      // The user was authenticated, but did not have the correct role
      // Redirect to an error page
      next({ name: 'Unauthorized' })
    }
  } else {
    // This page did not require authentication
    next()
  }
})

The navigation guard handles four use cases:

The page requires authentication and the user is not authenticated
The page requires authentication, the user is authenticated and has the correct role (authorized). Update their token.
The page requires authentication, the user is authenticated but not authorized. Redirect them to an error page.
The page does not require authentication

This is all the code necessary for simple login behavior. You can see in the sample code I’ve included Login and Logout buttons and those are really one-line functions from the Keycloak object.

Now we have an app demonstrating simple authentication and authorization using Keycloak. There are three places we added Keycloak integration into an app: a Vue plugin, wiring up the plugin in main.js, and in the Vue router. The sample could certainly be expanded upon and would need more functionality, especially if the app is calling API’s. This article shows how to refresh the token when using Axios using an interceptor. I hope this more Vue-centric sample helps out some folks trying to get their Vue app integrated with Keycloak.

Keycloak for Local Development

David Truxall — Tue, 30 Mar 2021 10:00:00 +0000

Keyloak is an open source identity management tool. If you are learning how to make your app use authentication, it’s an ideal tool if you don’t have any other source. It’s straightforward to learn and setup, yet has pretty sophisticated features to grow into. For this article, we are going to set up Keycloak to run locally using Docker.

TLDR; The Steps

Assuming you have Docker installed locally, these are the general steps to set Keycloak up for local use:

Pull and launch the Docker image
Set up a realm
Create a client (app registration)
Create a role for the users of the client
Create users
Assign users to the role

Docker

We will run the Docker container locally. The admin account gets created when creating the container. For this demonstration I’m just going to use admin/admin for the username and password. Don’t do that if you install Keycloak outside of your box, create a strong password. Run this Docker command on the command-line to launch Keycloak:

docker run --name keycloak-dev -d -e KEYCLOAK_USER=admin -e KEYCLOAK_PASSWORD=admin -p 8001:8080 quay.io/keycloak/keycloak:18.0.1

For this sample I’m using version 12.0.4, which will probably be out-of-date when you try this. So things can change, be aware these steps may no longer match in the future. The admin user gets created by the environment variables. The -d flag runs the container in the background and frees up your command line. Keycloak will start up using an embedded H2 database. If you destroy the container you’ll lose all your configuration. In a real deployment I’d use a container running PostGREs to store the data. For local development the embedded database is fine and much quicker to set up.

Keycloak will be running on port 8001 at http://localhost:8001. You’ll have to wait a bit for Keycloak to spin up. You can always check the log using:

docker logs keycloak-dev

When you see a message similar to “Keycloak 12.0.4 (WildFly Core 13.0.3.Final) started in 18938ms” you know it’s ready.

Set Up a Realm

Open http://localhost:8001 in your browser. You should see the staring screen.

Figure 1: Keycloak starting web page

Click the Administration Console link. Log in with the credentials used to create the Docker image above. Let’s create a Realm, which is a container for the apps you want to protect behind Keycloak. At first login you will be in the master realm, and you should not use that for your apps.

Figure 2: Add a new realm to Keycloak

Click the drop-down arrow next to the Master realm name in the left navigation pane, then click the Add Realm button. Give the realm a name and remember it for later, you’ll need it when configuring an app to authenticate using Keycloak.

Create A Client

Click the Clients link in the left navigation pane.

Figure 3: Add a new client to Keycloak

Click the Create button at the top left of the table of existing Clients to add a new Client for your app. In my case, I’m going to create a Vue app to demo logging in with Keycloak.

Figure 4: Name the client and set the root URL

Set the client id as the name of your app, and set the Root URL to the URL you are using for local development of your application. In my case, http://localhost:8080. Click the Save button.

Figure 5: Set the client details

In the Client details page, make sure the Client Protocol is set to openid-connectand the Access Type is set to public. Check that the following are set properly:

Save the details and lets add a Role to our Client.

Create A Role

Figure 6: Create a role for the new client

In the Roles tab, click the Add Role button at the upper right of the table. Give your role a name and save it.

Create Users

We need users to log into our application (Client). We will create a user and then assign a password and role. Click the Users link in the left navigation pane.

Figure 7: Add a Keycloak user

You won’t see any users listed, even if you added some. Click the Add user button in the top right of the table.

Figure 8: Add user details

Enter a username, and their actual name and email. Click the Save button. Let’s give them a password.

Figure 9: Set the password for the user

Assign A Role

Since the application (Client) has a role, we need to assign that role to the user so they can log into that application. Click the Role Mappings tab.

Figure 10: Assign a client role to a user

Select your client created earlier in the Client Roles drop down list. The role created earlier should be there. Select that role and click the Add selected > button. The user is good to go now.

The next step is to wire up an application to log in using this local instance of Keycloak.

A Computer Virus and Impostor Syndrome

David Truxall — Wed, 24 Mar 2021 16:38:42 +0000

2020 was the 20th anniversary of the Love Letter/Love Bug (ILOVEYOU) virus. If you’ve never heard of it, Sophos posted a good summary. I was working at a New Horizons in my first job as a web developer. When it hit, I happened to be in the office of one of the IT admins. We were talking, then he looks at his screen and asks: “Why would Gary send me an email saying he loves me?”. Another similar message arrived from someone else. Then another. I saw the light bulb go off in his head. He bolted out of his chair, ran down the hall to the server room and powered off the Exchange server. The virus was in the system for only a couple minutes and had spread to around 120 accounts. This was the first time I saw disaster recovery in action.

For me, the most interesting moment arrived after the fact, when I got a printout of the virus code. It was written in Visual Basic for Applications (VBA). This was the same programming language I was using in my job. I was creating an intranet using Active Server Pages (Classic ASP) which used VBA. I had also been writing Word and Excel macros to automate some company processes. When I looked at the printout, I thought to myself: “I could have written that!”. I didn’t want to write a virus, but I knew enough VBA to understand the code. It was a validating moment for me as a programmer. Someone else’s code that had done so much (even though it was damaging) was understandable to me. At that time I frequently read Dr. Dobb’s Journal and those articles filled me with impostor syndrome, the programming discussed in those articles was mostly over my head. The virus code was a turning point for me.

Every programmer starts like this, impostor syndrome creeps around the corner frequently. There is so much to know and understand and it feels overwhelming at times. I’ve always referred to it as “drinking from the firehose“. Know that it gets better over time. At some point I realized I was understanding the articles in Dr. Dobbs. That didn’t happen overnight, but I did put in the work to get there. I hope your validating moment arrives, and know that I am rooting for you.

Misleading CORS Errors

David Truxall — Fri, 11 Dec 2020 21:02:01 +0000

This post is about a problem with CORS (cross-origin resource sharing) in Chrome. CORS is one of the security mechanisms built into browsers to prevent other sites from consuming your content or APIs unless specifically allowed. If you are a native mobile developer or a back-end developer consuming APIs you may never encounter CORS errors, because CORS only applies to the browser. If you need some background on CORS, Mozilla has an excellent write-up. I’m going to discuss an issue I’ve encountered with misleading CORS errors in Chrome and ways to work around the error to discover the actual problem.

At some point Chrome changed the way CORS is reported in the developer tools, perhaps as far back as 2019. The current behavior in Chrome is that CORS errors take precedence over network errors. So today if your front-end application has a problem with the back-end service, it might report the problem as CORS when it is actually something else entirely. I am not the only developer that encountered this problem. If you read the error message (good developers read errors closely, don’t they?), you will end up going down the wrong path to solve the problem. Here’s the error I see in Chrome:

Access to XMLHttpRequest at ‘https://my.long.url.com’ from origin ‘http://localhost:8080’ has been blocked by CORS policy: No ‘Access-Control-Allow-Origin’ header is present on the requested resource.

It looks like a CORS error, but the underlying issue ended up being something else entirely. In my case, the problem was that the service layer was down. There is a reverse proxy in front of the service, and the reverse proxy was correctly returning an HTTP 503 response, because the service application behind the proxy had crashed. But that HTTP response lacked the Access-Control-Allow-Origin header. It was missing because the underlying application was setting those headers, not the reverse proxy. Since the application was down, the headers were not set. This causes Chrome to display the CORS error in the developer console instead of the network error.

Finding the Actual Problem

Because I was confident that there was no CORS misconfiguration in the server application, I had to look elsewhere for the answer. My first choice is to try a different browser. Another browser sometimes displays the same errors differently. I tried Safari. Same result. Edge? Same result. FireFox? A-ha! I saw the 503 error in the JavaScript console and also in the network tab in FireFox.

FireFox also reported the CORS error along with the network error. The JavaScript code still failed and kept the content of the 503 response from reaching the application code, which is the expected behavior for CORS errors. But the 503 error information was in the developer console. This is much more helpful to me as a developer. I assume that Chrome and the other browsers view this as a security issue and just deny the request when the header is missing, even in the developer console.

Another approach to finding the real issue would be to use a tool not affected by CORS, such as curl or PostMan. I used curl adding the -v flag and see the 503 response in the headers returned from the call:

As another alternative, I could even start an instance of Chrome without the CORS protections enabled using the terminal on my Mac :

open -n -a /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --args --user-data-dir="/tmp/chrome_dev_test" --disable-web-security

This also worked and I could now see the 503 error in the Chrome developer console. I do not like to develop using Chrome with security disabled because it ultimately hides issues like CORS which can occur as legitimate errors. But it’s a good tool for troubleshooting.

Perhaps we could fix the problem by configuring the reverse proxy to add the missing Access-Control-Allow-Origin in the case of 5xx HTTP responses? Or maybe configure the reverse proxy to add the CORS-related headers all the time instead of the underlying application? I’m not sure, finding information about this situation has been difficult.

The bottom line is this: If you see a CORS error when there were none previously and believe CORS is configured correctly on the back end, use some other tool to ensure you are troubleshooting the correct error.

Add Vulnerability Scanning to your Vue CI

David Truxall — Tue, 13 Oct 2020 18:53:06 +0000

DevOps is a term developers are familiar with by now. The term DevSecOps is now becoming popular, and gets lots of press. The “sec” part stands for security, but adding the “sec” is not so obvious or simple. One approach we can take to add some “sec” is to make sure the libraries our code depends on are safe. This step is only one facet of “sec” in DevSecOps. Ultimately the vulnerabilities in our dependencies are the library author’s responsibility, but we can take steps in our own project to understand the risks we face when using a library, and be informed so we can make choices to mitigate that risk.

This post demonstrates how to use the OWASP Dependency-Check tool to scan your project’s dependencies for known vulnerabilities in a CI/CD pipeline. Check out my previous post for more detail on this tool. This post outlines how to add dependency-check to a Vue.js project and apply the tool during a CI/CD build with Jenkins and publish the results to SonarQube. The files used in this post are found in this repo.

Getting Set Up

My team uses Jenkins for CI/CD. The simplest way to add dependency-check to your CI is to add it as a plugin for Jenkins. This takes care of any dependencies and lets you add it as a step in your CI, and publish the results on your job’s page.

On the other hand, I’m not a fan of Jenkins plugins. Troublesome builds on a Jenkins server are difficult to debug, and plugins make it harder. I prefer to be able to run build scripts locally, so my team routinely uses shell scripts for our builds, and Jenkins becomes a job runner and publisher of information instead of a collection of plugins cobbled together to create a build job. This adds some complexity as you need to install components on the server directly that may have been installed by a Jenkins plugin. And of course you’ll need those components installed locally as well. But if you have ever been stuck in a seemingly endless commit-push-build cycle trying to get a broken job to complete on the server you can appreciate this approach. I can run a script locally and make changes until it works before trying to publish the job on Jenkins. If it fails on Jenkins the culprit is likely server configuration and not something with the build script. When you use a Jenkins plugin it’s difficult to tell the difference between those two kinds of errors.

Look at my previous post for guidance on installing OWASP Dependency Check. You’ll have to install it on the build server and locally if you want to troubleshoot scripts on your machine.

The output of the Dependency Check scan can be published to Jenkins if you have the plugin installed. You can also publish results to SonarQube, a terrific static analysis tool for your code. In my case, we use SonarQube so that’s the route I’m describing. Since the test app is a Vue.js web application, we need to install a scanner for SonarQube. In my project I add:

npm install --save-dev sonarqube-scanner

Since it’s a development dependency it won’t be added to the final build code. Our build script needs to communicate with SonarQube, so there are environment variables set to take care of credentials. You can set them on the server directly or use a Jenkins plugin to set them for the job or build node.

SONAR_URL=<YOUR SONAR URL>
SONAR_USER=<YOUR SONAR USERNAME>
SONAR_PASSWORD=<YOUR SONAR PASSWORD>

To run the script locally create a file called .env.local and add the values above to the file. The build script will pick up that file and set the environment variables for the duration of the script run.

There is also a sonar-project.properties file in the project. I use this file to set the defaults for SonarQube and set the thresholds for SonarQube to report the severity based on the number of identified vulnerabilities.

sonar.host.url=${env.SONAR_URL}
sonar.exclusions=dist/ **/*,reports/*,coverage/** /*
sonar.login=${env.SONAR_USER}
sonar.password=${env.SONAR_PASSWORD}
sonar.dependencyCheck.jsonReportPath=reports/dependency-check-report.json
sonar.dependencyCheck.htmlReportPath=reports/dependency-check-report.html
sonar.dependencyCheck.severity.blocker=9.0
sonar.dependencyCheck.severity.critical=7.0
sonar.dependencyCheck.severity.major=4.0
sonar.dependencyCheck.severity.minor=0.0
sonar.projectKey=demo-web

In the SonarQube application itself you also need to enable the Dependency Check plugin in the Marketplace section of the app’s Administration area:

The dependency-check scanner creates output as XML, JSON, and HTML, but it’s up to you to choose which formats to generate. I create all three as output, you could do less depending on your circumstance. The Jenkins plugin wants XML and HTML in order to publish on your job’s page. SonarQube wants JSON and HTML to publish on that platform.

The Build Script

These are the steps our build script will take:

npm install
Run unit tests
Build for production
Create a Docker container
Deploy the container
Scan the code for vulnerabilities
Upload the results to an instance of SonarQube

#!/bin/bash

echo 'Clean up build directory'
git clean -fdx

echo 'npm tasks'
# Install dependencies
npm install

# Run unit tests
npm run test:unit

npm run build

# Create a Docker image
echo 'Docker tasks'
imageName="demo-web:1.0."

if [-z ${BUILD_NUMBER+x}];
    then
    echo "var is unset"
    imageName="${imageName}1"
    echo "build set to one";
    else
    echo "var is set to '$var'"
    imageName="${imageName}${BUILD_NUMBER}"
    repoName="${repoName}${BUILD_NUMBER}"
fi

docker build -t "${imageName}" .

# Remove running instances
echo 'Stop running container'
docker stop demo_web || true

echo 'Remove existing container'
docker rm demo_web || true

echo 'Start a new container'
# Start a container with this image
docker run --name demo_web --restart=always -v /data/log/demo-web:/var/log/nginx/log:rw -p 80:80 -p 443:443 -d "${imageName}"

# Clean up unused images
echo 'Remove old images'
# docker image prune -f

# Scan with dependency-check
echo 'Scan dependencies'
mkdir reports || true
dependency-check --scan ./ -f JSON -f HTML -f XML -o reports

# Set environment variables for Sonar credentials (local only)
set -a
. ./.env.local

echo 'SonarQube analysis'
# Run SonarQube scanner
./node_modules/sonarqube-scanner/dist/bin/sonar-scanner

Now you can create a job in Jenkins and use the script to build, deploy, and analyze our code. You can test this running the build script locally on the command-line too. If you are really intent on Jenkins, you could easily recreate that build script as a Jenkinsfile, but you won’t be able to test it locally.

The Results

When dependency-check runs, the results end up in the reports directory as specified in the build script. In the sonar-project.properties file we told the sonar scanner where to find these files. The scanner uploads them to SonarQube and integrates them into the results view:

The scan found three vulnerabilities and gave us a failing grade of “D” and kept our app from passing the quality gate set in SonarQube. Click the number of vulnerabilities link to see the detail:

The three libraries identified were not libraries we added directly with npm. The three vulnerable libraries discovered in this example are:

decompress:4.2.1
yargs-parser:10.1.0
yargs-parser:13.1.2

They are transitive dependencies of some other library we installed. We need to track them down. At the command-line we can run npm list <libraryname>to find out. I tried it for yargs-parser which was listed in two of the vulnerabilities reported by dependency-check:

The yargs-parser library actually occurred in three dependencies, all using different versions, two of which (versions 10.1.0 and 13.1.2) had a vulnerability. Luckily for us these occurred in our development dependencies, so they were not included in the built application. If they do end up as a dependency that gets deployed, you need to take some actions:

Report the problem as an issue in the library’s repo
Better yet, fix the problem and create a pull request at the library’s repo
Try changing the version in your package-lock.json file. Then re-run npm install and test your application. If we follow the links in the report we find out that versions 13.1.2, 15.0.1, 18.1.1 or later are not (as) vulnerable. There is no guarantee that the later version will not break the library depending on it though. YMMV.
Find a different library to solve the problem

OWASP Dependency Check adds awareness of your Vue app’s dependencies for vulnerabilities in your CI/CD process. You can’t fix problems you are not aware of, and this tool provides awareness, and my example shows how you surface that awareness in SonarQube. Add the OWASP Dependency Check tool and publishing steps to your CI/CD process and build safer software.

Using OWASP Dependency Check to scan a Vue app

David Truxall — Tue, 04 Aug 2020 17:59:37 +0000

In 2020 we know security is important. I strive to be a more secure developer all the time. There are many avenues to achieve this, and one of them is understanding the vulnerabilities of the libraries you include in your project. That’s the goal for this article.

This post demonstrates how to use the OWASP Dependency-Check tool to scan your project’s dependencies for known vulnerabilities. It supports many platforms, but in this instance I’m going to run it against a simple Vue.js application. My team is also running it against Android apps and Node.js web services. There is a Github repo with the code I’m scanning for this demonstration.

OWASP dependency-check depends on Java 8+ being installed, so make sure that is already set up in your environment. After Java is installed, installing dependency-check varies by platform. I’ll cover Mac, Windows, and command-line Linux.

Installing locally on my Mac was pretty straightforward as dependency-check can be managed with HomeBrew:

brew install dependency-check

Installing on Windows is slightly more complicated. Download the zip file, decompress it to a location you choose, then add that location to your path.

Installing dependency-check on Linux was also complicated. I decompressed the zip file in my user’s home directory, then created a symbolic link for dependency-check in /usr/local/bin. This avoids permission issues for the user running the tool.

The dependency-check scanner runs from the command-line and creates output as XML, JSON, or HTML, but it’s up to you to choose which formats to generate. I’ll create all three as output, you could do less depending on your circumstance.

To scan my Vue app, I’m going to open my terminal and navigate to my app’s directory. The scanner is looking at your dependencies so we need to make sure they are installed. I’m also going to create a directory for the scan output called “vulnerabilities” then run the scanner:

mkdir vulnerabilities
npm install
dependency-check --scan ./ -f JSON -f HTML -f XML -o vulnerabilities

The scan will take some time. The tool downloads the most recent reports of vulnerabilities, so the first run takes a while to execute. The reports are cached for a while so this won’t happen on every run. A build today that is vulnerability-free may not be so tomorrow if a new vulnerability is reported. The tool will update itself when you run it:

Once the scan is complete, there will be three files in the “vulnerabilities” directory. Other tools like Jenkins and SonarQube can consume the XML or JSON results generated during the scan. In another post I’ll go over adding dependency-check to a continuous integration build using Jenkins and SonarQube. Open dependency-check-report.html to see the person-readable results. The scan found four vulnerabilities:

None of those were libraries listed in our package.json, so those must be transitive dependencies of one of the libraries we installed. We need to track them down. At the command-line we can run npm list <libraryname>to find out. I tried it for yargs-parser which was listed in two of the vulnerabilities reported by dependency-check:

The yargs-parser library actually occurred in three dependencies, all using different versions, two of which (versions 10.1.0 and 13.1.2) had a vulnerability. Luckily for us these occurred in our development dependencies, so they were not included in the built Vue application and won’t be deployed. If they do end up as a dependency that gets deployed, you need to take some actions:

Report the problem as an issue in the library’s repo
Better yet, fix the problem and create a pull request at the library’s repo
Try changing the version in your package-lock.json file. Then re-run npm install and test your application. If we follow the links in the report we find out that versions 13.1.2, 15.0.1, 18.1.1 or later are not (as) vulnerable. There is no guarantee that the later version will not break the library depending on it though. YMMV.
Find a different library to solve the problem

OWASP Dependency Check is a straightforward tool you can use to scan your Vue app’s dependencies for vulnerabilities. Knowledge of these vulnerabilities makes you a more secure developer. You can’t fix problems you are not aware of, and this tool provides awareness. Run this tool against your app, and see what you learn. Better yet, add the tool to your CI/CD process and help the whole team.