DEV Community: Kyle Welsby

Deploying Google Cloud Platform Credentials to Koyeb

Kyle Welsby — Wed, 22 Dec 2021 23:33:07 +0000

Let's say you have an administration application you're deploying to Koyeb and you store assets on Google Cloud Storage.
You're going to need to get GOOGLE_APPLICATION_CREDENTIALS to your deployment somehow while keeping your credentials, secret.

Here is a quick guide to that I hope helps you get your awesome application interfacing with Google Cloud Platform in no time.

Assuming you've already gotten a JSON file containing your credentials the following steps make use of Base64 encoding and decoding to store and transfer the contents of the credentials.

1. Encode your key to Base64



base64 --input gcloud-key.json

You'll have the contents of the file encoded as a Base64 string. You can now add this to your Koyeb environment variables as a secret value.

2. Add the encoded Base64 as a secret variable



GCP_SA_KEY="aGkga295ZWI="

We're saving it as this variable for later use.

4. Add plain text env variable



GOOGLE_APPLICATION_CREDENTIALS="./gcloud-key.json"

While we're in the the environment variable settings, we'll need to add a plain text variable to tell our application what file to look for that contains the Google Application Credentials.

3. Update your build command



echo -n $GCP_SA_KEY | base64 --decode > gcloud-key.json | yarn build

We'll update the build command to echo the base64 string and pipe the string to the base64 command to be decoded and written to the file gcloud-key.json.

Then we'll run our normal build command yarn build.

Thats it, the next time your application builds the Google Application Credentials will be available to your application and you can happily interface with your Google Cloud Platform account.

Hope this helped, happy hacking.

Nuxt Content custom markdown block

Kyle Welsby — Thu, 26 Nov 2020 16:06:15 +0000

During a live-stream on 25th November 2020 I was tackling a feature on my project where I have Markdown within front-matter data I would like to render to the page correctly.

Here I'm quickly sharing some findings after sleeping on the problem and finding an elegant solution to the problem.

tl;dr

Utilise the content:file:beforeInsert hook in nuxt.config.js.

Setup

Assuming you've got a nuxt.js project with @nuxt/content module installed

It is expected you'll have a content file like the following, containing Markdown within a property.

# content/home.md
---
title: Hello World
specialNotice: |
  This is **Special Notice** to 
  everyone using 
  [Markdown](https://en.wikipedia.org/wiki/Markdown).
---

Configuration

With the above content, we can see the specialNotice which is formatted using Markdown. So we're going to pre-parse this field and make it accessible for the <nuxt-content /> helper to render the HTML output.

Within nuxt.config.js we're going to add into the hook section a content:file:beforeInsert to convert the Markdown content to JSON ATS which is like a node tree that can be rendered to the page.

// nuxt.config.js
export default {
  // ...
  hooks: {
    // Doc: https://content.nuxtjs.org/advanced#contentfilebeforeinsert
    'content:file:beforeInsert': async (document, database) => {
      // search for markdown containing 
      // only `specialNotice` property.
      if (document.extension === '.md' && 
          document.specialNotice) { 
        // Replace Markdown string in database 
        // with the JSON ATS version
        document.specialNotice = await database 
          .markdown
          .toJSON(document.specialNotice) 
      }
    }
  },
  // ...
}

Displaying

<!-- pages/index.vue -->
<template>
  <div>
    <h1>{{ document.title }}</h1>
    <nuxt-content :document="document.specialNotice" />
  </div>
</template>
<script>
export default {
  async asyncData ({ $content }) {
    const document = await $content('home').fetch()
    return {
      document
    }
  }
}
</script>

There we have it; Your page should contain the special notice rendered in HTML.

Follow me on Twitch, YouTube and Twitter for regular live-streams and chat.

Community-driven data modifications using Cloud Functions

Kyle Welsby — Wed, 18 Dec 2019 16:11:26 +0000

Prelude

I'm working on a side project around my paid client work and weekly Soulection Tracklists publications called TracksPlayed (a spin-off from Soulection Tracklists).

With this new project, I'm exploring new technologies and processes to improve features I wish I had in Soulection Tracklists.

Here is an insight into a process that I think will allow the community to suggest changes to data (I'm also open to suggestions in the comment section if you have other solutions).

For this; will assume you're using Firebase FireStore (a real-time NoSQL database by Google), but can easily be adapted to work with MongoDB and PostgreSQL projects.

Suggestions Collection

Let's start off with a database collection called suggestions here we will have a structure like the following.

{
  "path": "string",
  "key": "string",
  "value": "string",
  "createdAt": "datetime",
  "approvedAt": "datetime",
  "committedAt": "datetime"
}

Let's take a moment to explain each part of this schema.

property	type	description
`path`	`String`	A reference to the document we want to change for example, `product/1`, the path will contain the collection and document ID so we can find it later.
`key`	`String`	The value within the document, we will want to change the given value. Let's say a someone wants to suggest a `name` change to `product/1` the key would be `name` which leads us to;
`value`	`String`	The content that is suggested to be changed to.
`createdAt`	`String`	Optional, but useful to know when the suggestion was created.
`approvedAt`	`DateTime`	A date and time when the suggestion is approved. We'll explain this a bit later.
`committedAt`	`DateTime`	Once the change has been applied we'll mark this with the time the action was applied and finished. Useful to know the change has been firmly committed, or allow us to re-run the job safely.

Suggestions Moderation

Depending on your use case you can have a site admin moderation view or a per account moderation view which would be scoped to only the suggestions of the path match any of the documents the account has access to.

I'm not going to cover the per account moderation view in order to keep this writing short. Feel free to explore this yourself.

We'll assume a simple administration view where we will fetch all documents in the suggestions collection that has a null in the approvedAt column.

Approving the suggestion will set the approvedAt value to the current date new Date().

Handling Suggestions

On Firebase we can set a Firebase Function to listen for changes in FireStore when we update a document. In MongoDB or PostgreSQL you can add triggers, and execute a Cloud Function/Lambda.

We'll aim to do to the following in the Cloud Function;

Set the value of the suggested path key to the value given.
Set the committedAt to the current time.
Ensure we do not handle the approval multiple times

Start with a simple function to listen to the suggestions documents.

exports.onSuggestionApproved = functions.firestore.document('suggestions/{documentId}')
  .onUpdate((snap, context) => {
    // ... do something when the document is updated
  })
})

setting the value of the suggestion

We'll use the path to get the document reference and can set the key and value.

let ref = db.doc(data.path)
ref.set({
  [data.key]: data.value,
}, { merge: true })

Finishing the function

The ref.set returns a Promise, and we will have two promises to wait for. We need to add them to an array and wait for them all to resolve before completing the function.

Change each instance of ref.set to push to a promises array

let promises = []
promises.push(
  ref.set(/* ... */)
)

Return once all the promises are successfully resolved.

return Promise.all(promises)

Putting it all together

exports.onSuggestionApproved = functions.firestore.document('suggestions/{documentId}')
  .onUpdate((snap, context) => {
    const before = snap.before.data()
    const data = snap.after.data()
    let promises = []
    /**
     * @todo add approvedBy to ensure the changing document can be modified by a member of the account
     */
    if (!before.approvedAt && data.approvedAt) {
      let ref = db.doc(data.path)
      promises.push(
        ref.set({
        [data.key]: data.value,
        }, { merge: true })
      )
      promises.push(
        snap.after.ref.set({
          committedAt: new Date()
        }, { merge: true })
      )
    }
    return Promise.all(promises)
  })

Now we have a simple Cloud Function that listens for changes in a Firestore document and will trigger changes in a corresponding document once the approvedAt is defined.

Notable Concerns

In the final code, you can see a @todo. The power of this function allows any changes to any document. To avoid people setting themselves as administrators or committing changes to documents they have not authorisation on, we need to add a permission check.
This is a conditional check that would have to be made before the document is changed.

Conclusion

Allowing your community to make changes to database documents allows a whole wealth of crowd-sourced information.
This example allows the functionality without having to enable or replicate the code for each place we want to implement the feature on.

Having a suggestions column allows changes to be staged and moderated before they are committed. Also retains a history of the submitted changes for auditing should that be of interest to you.

If you have any better solutions or questions please comments below.

Writing tests for Vue.js Storybook

Kyle Welsby — Sat, 19 Oct 2019 22:46:50 +0000

Over the last couple of weeks, I've found new joy with writing my Vue.js components within Storybook as a tool to visualise all the possible permutations of a given Component in isolation from the target application.

It's all fair game writing your code, hitting save and seeing the change in the browser and visually observing everything works as expected. That's not good enough! I want unit-tests to ensure my components functionality is what I expect. ✅

In this guide, I'll show you how to install Jest to your Storybook project and examples of tests for Vue.js components.

Getting started

If you already have Storybook and Vue.js installed to your project, please skip to Installing Jest.

Let's get you quickly started with Storybook and Vue.js by creating a new project folder where your stories will reside.

Make a new folder; here we'll call it design-system but you can call it whatever you like.

mk ./design-system
cd ./design-system

Now we'll install our main dependencies Vue.js and Storybook.

note: My personal preference is the Single File Component style of Vue.js for ease of understanding between projects.

npm init -y # initialize a new package.json quicly
npm install --save vue
npm install --save-dev vue-loader vue-template-compiler @babel/core babel-core@^7.0.0-bridge.0 babel-loader babel-preset-vue
npx -p @storybook/cli sb init --type sfc_vue

Hooray! We've got Storybook installed with a couple of Vue.js examples to start.

Let's boot the Storybook server and see what we got.

npm run storybook

That is great and all, but now we'll want to set up Jest. 😄

Installing Jest

Let's get stuck right in and install all the dependencies required.

npm install --save-dev jest vue-jest babel-jest @babel/core @babel/preset-env @vue/test-utils

Configure Babel by creating a babel.config.js file in the root of the project.

// babel.config.js
module.exports = {
  presets: [
    '@babel/preset-env'
  ]
}

Configuration for Jest will need to be added too by creating a jest.config.js file in the root of the project.

// jest.config.js
module.exports = {
  moduleFileExtensions: ['js', 'vue', 'json'],
  transform: {
    '^.+\\.js$': 'babel-jest',
    '.*\\.(vue)$': 'vue-jest'
  },
  collectCoverage: true,
  collectCoverageFrom: ['<rootDir>/src/**/*.vue'],
  transformIgnorePatterns: ["/node_modules/(?!@babel/runtime)"],
  coverageReporters: ["text-summary", "html", "lcov", "clover"]
}

Finally, we'll need to update the package.json scripts to reference Jest as our test runner.

// package.json
{
  "name": "storybook-vue",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "jest",
    "storybook": "start-storybook -p 6006",
    "build-storybook": "build-storybook"
  },
  ...
}

Before we continue, let's give our installation a quick run to ensure everything is looking ok.

We'll have to run Jest with --passWithNoTests as we haven't written any tests yet.

note: the double dashes -- on their own are intentional to allow the arguments to be passed through to the inner command.

npm run test -- --passWithNoTests

We should see the following output.

npm run test -- --passWithNoTests

> storybook-vue@1.0.0 test ~/code/design-system
> jest "--passWithNoTests"

No tests found, exiting with code 0

=============================== Coverage summary ===============================
Statements   : Unknown% ( 0/0 )
Branches     : Unknown% ( 0/0 )
Functions    : Unknown% ( 0/0 )
Lines        : Unknown% ( 0/0 )
================================================================================

Great!, everything looks like it's wired up ok for Jest to be happy, now let's write some tests. 🤖

Writing our first test

Given we set up the project fresh and ran the initialise command in Storybook, we should have some simple example stories waiting for us in src/stories.

For example, our project structure would look something like this.

tree -I 'node_modules|coverage'
.
|-- babel.config.js
|-- jest.config.js
|-- package-lock.json
|-- package.json
`-- src
    `-- stories
        |-- 0-Welcome.stories.js
        |-- 1-Button.stories.js
        |-- MyButton.vue
        `-- Welcome.vue

2 directories, 8 files

Create a new file in the src/stories directory called MyButton.test.js so we can write our first tests for MyButton.vue.

In this test file, we'll import the MyButton.vue component and @vue/test-utils.

// src/stories/MyButton.test.js
import Component from './MyButton.vue';
import { shallowMount } from "@vue/test-utils";

describe('MyButton', () => {
  let vm
  let wrapper
  beforeEach(() => {
    wrapper = shallowMount(Component)
    vm = wrapper.vm
  })
})

Looking at our MyButton.vue file, we'll see in the <script> block a method called onClick.

// src/stories/MyButton.vue (fragment)
export default {
  name: 'my-button',

  methods: {
    onClick () {
      this.$emit('click');
    }
  }
}

This method, when called, will emit a click event to any parent consuming components. So testing this will require us to spy on $emit, and we will expect $emit to be called with click.

Our test will look like the following.

// src/stories/MyButton.test.js (fragment)
describe('onClick', () => {
  it('emits click', () => {
    vm.$emit = jest.fn()
    vm.onClick()
    expect(vm.$emit).toHaveBeenCalledWith('click')
  })
})

Here's a full example of our MyButton.vue.js test file.

// src/stories/MyButton.test.js
import { shallowMount } from "@vue/test-utils";
import Component from './MyButton.vue';

describe('MyButton', () => {
  let vm
  let wrapper
  beforeEach(() => {
    wrapper = shallowMount(Component)
    vm = wrapper.vm
  })

  describe('onClick', () => {
    it('emits click', () => {
      vm.$emit = jest.fn()
      vm.onClick()
      expect(vm.$emit).toHaveBeenCalledWith('click')
    })
  })
})

Brilliant! We can run our tests and see how we're doing.

npm run test

> storybook-vue@1.0.0 test ~/code/design-system
> jest

 PASS  src/stories/MyButton.test.js
  MyButton
    onClick
      ✓ emits click (15ms)


=============================== Coverage summary ===============================
Statements   : 25% ( 1/4 )
Branches     : 100% ( 0/0 )
Functions    : 33.33% ( 1/3 )
Lines        : 25% ( 1/4 )
================================================================================
Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        2.921s
Ran all test suites.

🎉 Congratulations you've just written our first test for our Storybook project!

... but what is that in the Coverage summary? 25% of the lines are covered? That has to be improved.

Improving code coverage

As we did with our first test, we'll create a new file for the other component Welcome.test.js in the src/stories directory.

The contents of Welcome.vue is a little more involved with props and having to preventDefault.

// src/stories/Welcome.vue
const log = () => console.log('Welcome to storybook!')

export default {
  name: 'welcome',

  props: {
    showApp: {
      type: Function,
      default: log
    }
  },

  methods: {
    onClick (event) {
      event.preventDefault()
      this.showApp()
    }
  }
}

Let's cover the natural part first, methods as with the tests in MyButton.test.js we can copy most of this code across.

As our code stipulates, we'll need to spy on the given property showApp to ensure it is called and the event we provide will have to include preventDefault.

// src/stories/Welcome.test.js (fragment)
describe('onClick', () => {
  it('calls showApp', () => {
    let showApp = jest.fn()
    wrapper.setProps({
      showApp
    })
    let event = {
      preventDefault: jest.fn()
    }
    vm.onClick(event)
    expect(showApp).toHaveBeenCalled()
    expect(event.preventDefault).toHaveBeenCalled()
  })
})

Testing props have a subtle difference to it as we need to fully mount the component to access the $options where props are defined.

// src/stories/Welcome.test.js (fragment)
describe("props.showApp", () => {
  it('logs message', () => {
    wrapper = mount(Component)
    vm = wrapper.vm
    let prop = vm.$options.props.showApp;

    let spy = jest.spyOn(console, 'log').mockImplementation()
    prop.default()
    expect(console.log).toHaveBeenCalledWith('Welcome to storybook!')
    spy.mockRestore()
  })
})

Ensure to import mount from @vue/test-utils

// src/stories/Welcome.test.js (fragment)
import { shallowMount, mount } from "@vue/test-utils";

You would notice we're using jest.spyOn() to mock the implementation of console.log to allow us to assert .toHaveBeCalledWith and then restore the console.log to its initial application once our test has completed.

Here is a full example of the test file.

// src/stories/Welcome.test.js
import { shallowMount, mount } from "@vue/test-utils";
import Component from './Welcome.vue';

describe('Welcome', () => {
  let vm
  let wrapper
  beforeEach(() => {
    wrapper = shallowMount(Component)
    vm = wrapper.vm
  })

  describe("props.showApp", () => {
    it('logs message', () => {
      wrapper = mount(Component)
      vm = wrapper.vm
      let prop = vm.$options.props.showApp;

      let spy = jest.spyOn(console, 'log').mockImplementation()
      prop.default()
      expect(console.log).toHaveBeenCalledWith('Welcome to storybook!')
      spy.mockRestore()
    })
  })

  describe('onClick', () => {
    it('calls showApp', () => {
      let showApp = jest.fn()
      wrapper.setProps({
        showApp
      })
      let event = {
        preventDefault: jest.fn()
      }
      vm.onClick(event)
      expect(showApp).toHaveBeenCalled()
      expect(event.preventDefault).toHaveBeenCalled()
    })
  })
})

We can rerun our tests and fingers crossed the coverage should be vastly improved. 🤞

npm test

> storybook-vue@1.0.0 test ~/code/design-system
> jest

 PASS  src/stories/MyButton.test.js
 PASS  src/stories/Welcome.test.js

=============================== Coverage summary ===============================
Statements   : 100% ( 4/4 )
Branches     : 100% ( 0/0 )
Functions    : 100% ( 3/3 )
Lines        : 100% ( 4/4 )
================================================================================

Test Suites: 2 passed, 2 total
Tests:       3 passed, 3 total
Snapshots:   0 total
Time:        2.404s
Ran all test suites.

That is Awesome, well done! 🚀

Notes

With most code challenges, I usually battle through small problems along the way. Here I like to give credit to where I have found solutions to the issues I have experienced while getting the project setup.

Using Jest with Babel as documented required adding babel-core@7.0.0-bridge.0 to the development dependencies to ensure it works well with Babel 7.

You'll notice in the jest.config.js I included a transformIgnorePatterns definition. Although the current code doesn't demand too much from Core.js, I added this definition. It will save some headake later on in your development, avoiding the no descriptive SyntaxError: Unexpected identifier issues.

Thank you for reading, I hope this helped you get your Vue.js Storybook project to the next level.
🙏

Unit testing Firebase Firestore & Cloud Functions

Kyle Welsby — Mon, 30 Sep 2019 09:49:11 +0000

A personal project of mine has me open the pandora's box of all fun and new technology. I usually am not able to use in my daytime job — Firebase Firestore and Cloud Functions (Lambdas for you AWS folk). 🤖

I challenged myself to write a function that takes a payload of data and creates a record in Firebase. Along with this challenge, I wanted to wrap my functionality with unit-tests as a new stretch goal.

The official Firebase Cloud Functions documentation is easy to read and understand for very basic use-cases. I wanted to go the extra mile beyond the primary examples. 😄

Code

Here I have a simple function that listens to a Firestore document created event. It will invoke the Cloud Function to take the data, check if it exists and if not, create an associating record.

const functions = require('firebase-functions');
const admin = require('firebase-admin');

admin.initializeApp();
let db = admin.firestore();

exports.onEpisodeTrackCreated = functions.firestore.document('episodes/{episodeId}/tracks/{trackIndex}')
  .onCreate((snap, context) => {
    const data = snap.data()

    if (!data.name) throw new Error('Missing `name` parameter')

    const name = data.name.trim()
    let tracksRef = db.collection('tracks')

    return tracksRef.where('name', '==', name).get()
      .then(snapshot => {
        if (snapshot.empty) {
          return tracksRef.add({
            name: name
          })
        }
        let doc
        snapshot.forEach(snapDoc => {
          doc = snapDoc
        })
        return doc
      })
        .then((doc) => {
          snap.ref.set({
            trackId: doc.id
          }, { merge: true })
          return doc
        })
  })

Test Setup

Install firebase-functions-test and Jest; a popular "batteries included" testing framework.

npm install --save-dev firebase-functions-test jest

We'll need to create a test folder where we will store the unit-tests for our functions.

Next, I updated the package.json with the test script to call.

"scripts": {
"test": "jest test/"
}

Firebase Cloud Functions can run in Online and Offline modes. Online mode means it will interact with your Firebase account, create/destroy data. Offline mode will result in us stubbing our calls, and this is the preferred option in my opinion for this writing.

Initialize the SDK in offline mode by not defining any configuration options.

const test = require('firebase-functions-test')();

Let us continue with writing our unit test that invokes the function and should successfully resolve with the async/await otherwise it will throw an error.

const test = require('firebase-functions-test')();
const functions = require('../index.js');

describe('onEpisodeTrackCreated', () => {
  it('successfully invokes function', async () => {
    const wrapped = test.wrap(functions.onEpisodeTrackCreated);
    const data = { name: 'hello - world', broadcastAt: new Date() }
    await wrapped({
      data: () => ({
        name: 'hello - world'
      }),
      ref:{
        set: jest.fn()
      }
    })
  })
})

What happens when we run the test now? 🤔


 FAIL  tests/index.test.js
  onEpisodeTrackCreated
    ✕ successfully invokes function (832ms)

  ● onEpisodeTrackCreated › successfully invokes function

    Could not load the default credentials. Browse to https://cloud.google.com/docs/authenti
cation/getting-started for more information.

      at GoogleAuth.getApplicationDefaultAsync (node_modules/google-auth-library/build/src/auth/googleauth.js:161:19)
      at GoogleAuth.getClient (node_modules/google-auth-library/build/src/auth/googleauth.js:503:17)
      at GrpcClient._getCredentials (node_modules/google-gax/src/grpc.ts:150:20)
      at GrpcClient.createStub (node_modules/google-gax/src/grpc.ts:295:19)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        1.91s

😢 this is not good.

Thinking about this error a bit more, we do have quite a bit going on in our code. This error is telling us something about the credentials. Perhaps it is to do with the initializeApp on the firebase-admin? 🤔

We'll mock that and see what happens next.

jest.mock('firebase-admin', () => ({
  initializeApp: jest.fn()
}))

And the result...

 FAIL  tests/index.test.js
  ● Test suite failed to run

    TypeError: admin.firestore is not a function

      3 | 
      4 | admin.initializeApp();
    > 5 | let db = admin.firestore();
        |                ^
      6 | 
      7 | exports.onEpisodeTrackCreated = functions.firestore.document('episodes/{episodeId}/tracks/{trackIndex}')
      8 |   .onCreate((snap, context) => {

      at Object.firestore (index.js:5:16)
      at Object.require (tests/index.test.js:23:19)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        1.757s

Brilliant, this is a better position to be in. Because we're calling out to firestore but we've completely mocked the implementation this is as expected.

Now to complete the mocking for this test. 😅

const mockQueryResponse = jest.fn()
mockQueryResponse.mockResolvedValue([
  {
    id: 1
  }
])

jest.mock('firebase-admin', () => ({
  initializeApp: jest.fn(),
  firestore: () => ({
   collection: jest.fn(path => ({
     where: jest.fn(queryString => ({
       get: mockQueryResponse
     }))
   })) 
  })
}))

And the final run. 😬

 PASS  tests/index.test.js
  onEpisodeTrackCreated
    ✓ successfully invokes function (3ms)

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        1.026s

Brilliant. 🙌

I really hope this solution helps you with testing your next project.

Sources

Of course, this result did not come about organically, it took a great deal of searching through the internet for relevant solutions through the coding phase.