DEV Community: David Wells

Promisified node.js file system utilities

David Wells — Fri, 15 May 2020 16:46:29 +0000

Originally published on davidwells.io

Node introduced the promisify utilities in back in version 8.

Using promisify you can use the node.js file system utilities with async/await and promises.

There are a couple of bits missing from the core node.js filesystem.

Below are the promisified file system calls I use a bunch in projects.

I've added createDir, fileExists, and deleteDir to smooth over some of the core fs methods that can be a little awkward to work with.

/* ./utils/fs.js */
const fs = require('fs')
const mkdirp = require('mkdirp')
const rimraf = require('rimraf')
const { promisify } = require('util')

const fileExists = (s) => new Promise(r => fs.access(s, fs.F_OK, e => r(!e)))

module.exports = {
  // Check if file exists
  fileExists: fileExists,
  // Read file
  readFile: promisify(fs.readFile),
  // Write file
  writeFile: promisify(fs.writeFile),
  // Recursively create directory
  createDir: mkdirp,
  // Read directory file names
  readDir: promisify(fs.readdir),
  // Recursively delete directory & contents
  deleteDir: promisify(rimraf),
}

Install mkdirp and rimraf npm packages for deleteDir & createDir

npm install mkdirp rimraf

Then you can use it in your project

const path = require('path')
const { fileExists, readFile } = require('./utils/fs')

async function doIt(filePath) {
  const exists = await fileExists(filePath)  
  let contents
  if (exists) {
    contents = await readFile(filePath)
  }
  return contents
}

const file = path.resolve('foo-bar.json')
doIt(file).then((contents) => {
  console.log('Yay 🎉', contents)
})

Hope this helps!

Let me know if you have other file system utilities you like to use in the comments below.

Analytics tip: How to trigger custom logic for utm campaign parameters

David Wells — Fri, 04 Oct 2019 16:37:00 +0000

Url utm tracking tokens are very commonly used for measuring the effectiveness of marketing campaigns. They are very useful when running ads, sending emails, and tracking traffic from external sources. For more on what utm token are checkout this post.

When using google analytics, these utm tokens are automatically saved into google analytics. Yay 🎉

But, what if we want to save the campaign data in other tools? Or what if we aren't using google analytics at all and want to collect these metrics?

We can use the analytics npm package to do this!

The `campaign` event

In the analytics lifecycle, there is a campaign event that is emitted if there are utm parameters found in the current URL.

This campaign event can be hooked into with a plugin or a listener.

Using a custom plugin

Create a custom plugin
Attach the plugin to analytics when it's initialized

import Analytics from 'analytics'

/* 1. Create the plugin & functionality you want to trigger */
const customPlugin = {
  // All plugins have a NAMESPACE
  NAMESPACE: 'save-campaign-data',
  // Attach a function to the 'campaign' event
  campaign: ({ payload }) => {
    console.log('utm data', payload.campaign)
    // Send data elsewhere, save to localStorage etc.
  }
}

/* 2. Attach the plugin to the plugins array when you initialize analytics */
const analytics = Analytics({
  app: 'app-name',
  plugins: [
    customPlugin,
    // ... other plugins
  ]
})

After attaching the plugin, anytime there is a utm paramters in a url this functionality will trigger.

For example:

?utm_source=a&utm_medium=b&utm_term=c&utm_content=d&utm_campaign=e

Will fire the campaign event with the payload

const customPlugin = {
  NAMESPACE: 'save-campaign-data',
  campaign: ({ payload }) => {
    console.log('utm data', payload.campaign)
    /*{
      source: 'a',
      medium 'b',
      term: 'c',
      content 'd',
      name: 'e'
    }*/
  }
}

Using a Listener

You can also react to the campaign event via a listener anywhere in your application code.

Initialize analytics
Set a listener with on or once to listen for the campaign event

import Analytics from 'analytics'

const analytics = Analytics({
  app: 'app-name',
  plugins: [
    // ... other plugins
  ]
})

/* Attach a listener to campaign */
analytics.on('campaign', ({ payload }) => {
  console.log('utm data', payload.campaign)
  // Send data elsewhere, save to localStorage etc.
})

You can also use the once listener to ensure a callback is only fired once.

/* Alternatively Attach a listener to campaign */
analytics.once('campaign', ({ payload }) => {
  console.log('utm data', payload.campaign)
  // Send data elsewhere, save to localStorage etc.
})

For more information on how to use the analytics library for page tracking, custom events, & identifying visitors. See the docs!

Simplify form handling in your app using valid HTML elements

David Wells — Thu, 05 Sep 2019 15:56:58 +0000

Originally published on davidwells.io

If your website or app does something (anything), you most likely have this flow somewhere:

User enters information into form
Run Input validation (optional)
User submits form
Data sent to backend for processing 🚀

To handle this flow, we'll need to handle form inputs in our code.

Often times, in the land of JS fatigue, I see devs jump for framework specific libraries or larger "form state" solutions to address form handling.

There are many many libraries to tackle form problems formsy React, redux-forms, final-form, formik, the list goes on...

Aside: These libs offer a number of features & support various edge cases etc. Use them if you need them! Kudos to the respective open source authors for their time & awesomeness 🎉

But... I'm always wondering why folks are not using simple HTML forms & grabbing values from the DOM.

In my experience with building large web apps these larger "form state" solutions have turned out to be more than I need.

This post is about using simple valid HTML form inputs for form handling.

This approach works with any framework or with plain ol' HTML.

Why you no HTML?

HTML forms have existed since the dawn on the web. Those input nodes are battle tested and work quite well. In this post, we are going to use them to grab form values and do our interesting app stuff.

Using native form event listeners work great to solve this problem but the developer ergonomics around the DOM APIs and grabbing values from different types of inputs is the tricky bit.

<html>
  <body>
    <!-- The awesome app form -->
    <form id="baz" action="/" method="POST">
      <label>
        <span>Email address</span>
        <input name="email"type="email" required />
      </label>

      <button type="submit" name="submit" class="button">
        Submit Form
      </button>
    </form>

    <!-- The codez -->
    <script>
    /* Get form selector */
    const form = document.querySelector('form[id=baz]')

    /* Make awesome app logic function */
    function doStuff(event) {
      // 1. Get event.target
      // 2. Loop over DOM nodes
      // 3. Get current node values
      // 4. Do app logic
    }

    // Attach the listener
    form.addEventListener('submit', doStuff, false)

    // Lastly Remember to remove listener if in SPA
    form.removeEventListener('submit', doStuff, false)
    </script>
  </body>
</html>

The code above isn't too bad but it can be quite cumbersome to loop over form inputs and grab values from different types of form elements.

What if we need to attach to multiple forms? Logic starts to add up.

How can we simplify?

There are two approaches to solving this.

1. Using get-form-data

The first is using get-form-data, it's very tiny and let's you grab all values from a valid form element.

const form = document.querySelector('form[id=baz]')

/* Make awesome app logic function */
function doStuff(event) {
  const data = getFormData(form)
  console.log(JSON.stringify(data))
  // Do stuff with the form data
}

// Attach the listener
form.addEventListener('submit', doStuff, false)

It's super simple and works well in react.

2. Using analytics-util-forms

The second, is the form utilities library. This takes things 1 step further and will attach the event listeners to one or more forms for you.

Form utilities is a tiny lib (2.6kB) for reading values from valid HTML forms. I created this form-utils library was created for use in some upcoming analytics plugins.

Form utilities lib exposes these methods onSubmit, onChange, & listen.

You can listen to individual input changes, or full on "user clicked the submit button" events.

Checkout the example HTML for more on how to use:

And read on to see the API.

Listening to form submissions with form-utils

Listen to form submissions & do stuff with inputs.

This will incept form submissions & fire a custom callback before submitting the form normally.

import { onSubmit } from 'analytic-util-forms'

// Add to single form
const formOne = document.querySelector("form[id=one]")
onSubmit(formOne, (event, data) => {
  console.log('form', event.target)
  console.log('form data', JSON.stringify(data, null, 2))
})

// Add to single form with options
onSubmit('form[id=two]', {
  /* Turn on debug to disable submissions and see values */
  debug: true,
  /* Turn off sensitive values filter */
  disableFilter: false,
  //* // Exclude field by name or regex pattern of name attribute
  excludeFields: [
    /private/,
    'shhhh'
  ],
  /* Custom filter function. Return false to exclude data */
  filter: (fieldName, value) => {
    if (fieldName === 'hello') {
      return false
    }
    // credit card number
    if (value.match(/^\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}$/)) {
      return false
    }
    return true
  }
}, (event, data) => {
  console.log('form', event.target)
  console.log('form data', JSON.stringify(data, null, 2))
})

// Remove onSubmit listener
const cleanUpFuntion = onSubmit('form[id=three]', (event, data) => {
  console.log('form', event.target)
  console.log('form data', JSON.stringify(data, null, 2))
})
cleanUpFuntion() // <-- call function to clean up listener


// Listen to all forms on page
onSubmit('all', (event, data) => {
  console.log('form', event.target)
  console.log('form data', JSON.stringify(data, null, 2))
})

Listening to form field changes with form-utils

Listen to form changes & do stuff with inputs.

import { onChange } from 'analytic-util-forms'

// Add to single form with no options
const formOne = document.querySelector("form[id=one]")
onChange(formOne, (event, data) => {
  console.log('form', event.target)
  console.log('form data', JSON.stringify(data, null, 2))
})

// Add to single form with options
onChange('form[id=two]', {
  /* Turn on debug to disable submissions and see values */
  debug: true,
  /* Turn off sensitive values filter */
  disableFilter: false,
  //* // Exclude field by name or regex pattern of name attribute
  excludeFields: [
    /private/,
    'shhhh'
  ],
  /* Custom filter function. Return false to exclude data */
  filter: (fieldName, value) => {
    if (fieldName === 'hello') {
      return false
    }
    // credit card number
    if (value.match(/^\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}$/)) {
      return false
    }
    return true
  }
}, (event, data) => {
  console.log('form', event.target)
  console.log('change data', JSON.stringify(data, null, 2))
})

// Remove onChange listener
const cleanUpFuntion = onChange('form[id=three]', (event, data) => {
  console.log('form', event.target)
  console.log('change data', JSON.stringify(data, null, 2))
})
cleanUpFuntion() // <-- call function to clean up listener

// Listen to all forms on page
onChange('all', (event, data) => {
  console.log('form', event.target)
  console.log('form data', JSON.stringify(data, null, 2))
})

Listening to submission & change events with form-utils

Listen will attach onChange & onSubmit listeners to forms

import { listen } from 'analytic-util-forms'

// Add to single form with no options
const formOne = document.querySelector("form[id=one]")
listen(formOne, (event, data, type) => {
  console.log('listen type', type)
  console.log('listen form', event.target)
  console.log('listen form data', JSON.stringify(data, null, 2))
})

// Listen to all forms with options
listen({
  /* Turn on debug to disable submissions and see values */
  debug: true,
  /* Turn off sensitive values filter */
  disableFilter: false,
  /* Custom functionality handler for onSubmit */
  onSubmit: (event, data) => {
    console.log('submit form', event.target)
    console.log('submit data', JSON.stringify(data, null, 2))
  },
  onChange: (event, data) => {
    console.log('change form', event.target)
    console.log('change data', JSON.stringify(data, null, 2))
  },
  /* Include only specific forms. This negates 'all'
  includeForms: [
    'form[id=content-form]',
    window.document.forms[1]
  ],
  /**/
  /* Exclude forms by selectors or node.
  excludeForms: [
    'form[name=two]',
    window.document.forms[2]
  ],
  /**/
  //* // Exclude field by name or regex pattern of name attribute
  excludeFields: [
    /private/,
    'shhhh'
  ],
  /**/
  //* // Custom filter function
  filter: (fieldName, value) => {
    if (fieldName === 'hello') {
      return false
    }
    // credit card number
    if (value.match(/^\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}$/)) {
      return false
    }
    return true
  }
  /**/
})

Installing

To install the form helpers in your project run

npm install analytics-util-forms

Keepin' things small

In an effort to keep bundle sizes as tiny as possible, I always checkout bundlephobia for respective package sizes.

Keep things as small as you can and budget dependancies accordingly!

Wrapping up

How you handle form values are ultimately up to you. If other libs or abstractions work for you, great! Keep on truckin 🚚. They handle additional things like validation, dirty inputs, and a bunch of other use cases.

Props to insin, for the get-form-data library for inspiration and the analytics-util-forms this uses under the hood.

A practical approach for clean analytics

David Wells — Mon, 26 Aug 2019 16:06:36 +0000

Originally posted on davidwells.io

Collecting clean company wide metrics is hard.

When every app, site & tool your company makes is instrumented in a different way, by a different developer, the downstream effects can be rather troublesome.

It’s the classic case of garbage in, garbage out.

This document outlines a proposal for a clean approach to metric collection across all projects of an organization.

This approach has worked quite well at Serverless & Netlify for a number of data science, product, and marketing activities.

Let’s jump into it!

Why Standardize?

As companies & product grow, it’s critical that instrumentation is in place to track how things are evolving and how things are improving against KPIs.

The first step towards clean data is standardizing around a naming convention and ensuring the right tooling in place to enforce these conventions.

Standardizing from the beginning of a project can help ensure that data is reliable, readable, & will scale to changes in product direction.

These naming conventions for analytics are important for:

Programmatically dealing with data
Organizing + searching for data in downstream analytics UI
Data consistency & reliability
Ensuring event names scale to diverging product lines
Streamlining on-boarding for new members on the team

How?

In order to achieve the benefits listed above, there there a couple considerations to take into account.

The pattern established must scale to fit multiple products/touch-points
The pattern must be parseable by humans and machines
All products/touch-points must adhere to the pattern
Validation + enforcement are also quite important. We’ll cover that in the implementation section below.

So what's a good naming convention look like?

Event Naming Conventions

After quite a bit of research on what “sane” naming conventions might look, what shook out was a slightly modified version of from segment’s "object-action framework" from their guide for clean data.

Pattern

Context => Object => Action

The format answers these questions:

Where is the event from? Context
What is the event effecting? Object
What was the action taken? Action

Some examples of how this might look in various contexts:

App => site => deployed
App => function => invoked
Site => docs => rated
Site => docs => searched
CLI => user => loggedIn
API => user => passwordReset

The context of where the events originate is generically short but still descriptive enough for folks to understand where they originated from.

Some examples of context context would be app (web ui), site (marketing site), cli (the command line interface), api (the backend api service), etc.

These will vary depending on your application/org structure.

Format/Syntax

The syntax is, of course, up to you.

In our case, all events are formatted like this with words camelCased

productName:objectName_actionName

Here are some examples:

site:newsletter_subscribed
app:site_deployed
cli:user_login
api:site_created

The above syntax achieves a couple things. First and foremost, it’s human readable and easy for folks to tell exactly where an event originated.

For example, app:user_emailChanged happened in the main app, effected the user, and tells me exactly what they did (changed their email).

Prefixing the context, with product name, also makes it much easier to digest data in downstream marketing tools.

It’s easy to filter down to all events that originated in your site, in your app, CLI, whatever. This gives product & marketing folks a very clear picture of what data is available for activation & retention activities.

Conventions

These are recommended conventions to follow:

Prefix with all events with the product name
camelCase everything
Separate product name & the Object/Action with :
Separate the Object and Action with _
i.e. productName:objectName_actionName
Don’t abbreviate unnecessarily
Avoid programmatically create event names
For page loads, use “viewed”, not “visited”, “loaded”, etc.

Implementation

It’s very important to optimize these collection points as soon as you can. Retrofitting a haphazard analytics implementation is much more of a challenge. It could require more cross team coordination & investigative work to make sure downstream systems will still operate as expected.

Here are some practical next steps for getting things in a "cleaner" state:

1. Map out existing projects & properties

First things first, collect and map out objects & actions across properties. What product lines exist? How might things evolve? What critical happy paths & events would be valuable to track?

2. Decide on naming conventions

Then, figure out your naming conventions that make sense for your organization.

3. Communicate changes to org

Then, talk to other stakeholders and agree on naming conventions.

It’s nice for everyone to say “yes we will do it like this” but even better to bake naming convention validation directly into the code. This will prevent new folks from firing poorly named events and polluting downstream data science efforts. Think of this as "linting" for analytics.

Making it impossible for bad data to get through will make everyone’s life easier as they are trying to do things with it down the line.

4. Implement & ship

How you enforce these conventions is entirely up you you.

I'd recommend using standard libraries across your organization & leveraging some existing tooling to make this easier.

The analytics library makes validation easy via its plugin middleware architecture.

Every page view, custom event, and identify call is passed through a middleware chain that allows for adding safe guards to ensure data flowing into third party tools is clean.

When application code calls analytics.track() the data flows through the middleware chain and if events are invalid, you can automatically format them to meet standards or warn the developer to conform to the conventions.

Tracking events flow through this chain

Flip on debug mode and you can watch as events flow through as you navigate your application.

Using analytics plugin

Adding validation checks everywhere you track events can be a large task. That's one of the reasons why the analytics library is a nice abstraction layer to help do this in a single spot.

Instead of going into 20 different files and adding if/else statements, we can simply add the events validation plugin.

The naming convention described in this post is codified in the plugin and will noOp any malformed events.

import Analytics from 'analytics'
import eventValidation from 'analytics-plugin-event-validation'
import customerIOPlugin from 'analytics-plugin-customerio'

const analytics = Analytics({
  app: 'awesome-sauce',
  plugins: [
    eventValidation({
      /* Context of where events are coming from */
      context: 'app',
      /* Allowed objects */
      objects: [
        'sites', // example app:sites_cdConfigured
        'user',  // example app:user_signup
        'widget' // example app:widget_created
      ],
      /* throw on validation errors if using in only dev env */
      // throwOnError: true
    }),
    customerIOPlugin({
      siteId: '123-xyz'
    }),
  ]
})

// Event names must now conform to this format:
analytics.track('app:sites_whatever')
analytics.track('app:user_action')
analytics.track('app:widget_deleted')

Bring your own convention

If you aren't a fan of the proposed event syntax content:object_action, you can roll your own analytics plugin and bring your own validation logic.

Here is an example:

import Analytics from 'analytics'
import googleAnalytics from 'analytics-plugin-ga'

/* Bring your own plugins */
const customValidationPlugin = {
  NAMESPACE: 'company-xyz-event-validation',
  trackStart: ({ payload, abort }) => {
    /* Your custom event validation logic */
    if (!payload.event.match(/^foo/)) {
      /* Abort the call or throw error in dev mode */
      return abort('Event name does not meet validation requirements')
    }
  }
}

const analytics = Analytics({
  app: 'app-name',
  plugins: [
    // Attach custom validation plugin
    customValidationPlugin,
    googleAnalytics({
      trackingId: 'UA-121991123',
    })
  ]
})

// valid event
analytics.track('foo button clicked')

// invalid event
analytics.track('blahblah bad event name')
// ^ no ops or throws depending on your validation logic in `trackStart`

For more on writing custom plugins see the docs

Research & links

There are tons of great resources out there to learn more about advanced analytics.

See below resources for additional analytics information:

5 advanced npm package.json config tips

David Wells — Tue, 06 Aug 2019 18:14:54 +0000

Originally published on davidwells.io

Node, npm, and package.json are super awesome.

Together they give devs a rich dev environment & build system out of the box.

Over my years as a node dev, I've gathered some advanced patterns for making npm scripts & package.json even more awesome, so I thought I'd share.

Hope these help ya. Also, leave a comment below if you have!

1. Using NPM script `pre` & `post` lifecycle

npm has built in lifecycle methods for your package scripts. When you use pre and post with commands, they will run before and after the given script.

An Example

{
  "name": "npm-lifecycle-example",
  "scripts": {
    "prefoo": "echo prefoo",
    "foo": "echo foo",
    "postfoo": "echo postfoo"
  }
}

Running npm run foo will run these three commands in order

prefoo,
foo,
then postfoo

Will result in:

echo prefoo
echo foo
echo postfoo

Use this for prebuild and postbuild (and whatever else 🌈). This is very handy for baking in deploy processes & chaining npm commands together.

2. Passing an argument into a command

You can dynamically pass values into an npm script.

To do this, you need to use the $npm_config_{key} value in your command.

Example:

{
  "name": "passing-flags-into-command",
  "scripts": {
    "pass-arg": "node scripts/lol.js --name=$npm_config_name",
  }
}

Then use it:

npm run pass-arg --name woot

In this example we are running lol.js with the arg.

const argv = require('minimist')(process.argv.slice(2));
console.log(argv.name);
// woot

This is super handy for makeshift CLI tools within your projects.

3. Using values from package.json `config` key

To avoid repeating configuration in a bunch of different scripts, You can reference values from the package.json file config field.

{
  "config": {
    "ENDPOINT": "my-value"
  },
  "scripts": {
    "set-env": "REACT_APP_ENDPOINT=$npm_package_config_ENDPOINT react-scripts start"
  }
}

This is handy for certain occasions

4. Using `cross-env` for cross platform scripts

The setting environment variables in a script is pretty easy, however they aren't cross platform compatible (cough cough windows)

The cross-env pkg from npm allows you sets ENV vars for command to use so anyone running your scripts will be happy.

Example:

{
  "name": "using-cross-env",
  "scripts": {
    "cross-env": "cross-env NODE_ENV=prod OTHERFLAG=xyz webpack --config webpack.js",
  }
}

This will set NODE_ENV=prod and OTHERFLAG=xyz to process.env variables for webpack to use (for example).

Props to kentcdodds for making this gem.

5. Passing arguments to other npm commands

Sometimes we need to pass additional arguments to an existing npm script.

Rather than altering the existing script every time we want the arg, or duplicating the script over and over, we can leverage the -- separator.

Example json

{
  "name": "passing-flags-example",
  "scripts": {
    "my-script": "esw src/components",
    "pass-flags-to-other-script": "npm run my-script -- --watch",
  }
}

This example below will pass the --watch flag to the my-script command

npm run pass-flags-to-other-script

Will run:

esw src/components --watch

This is super handy for setting up test scripts with a normal mode and a watch mode.

Putting it all together

Here is are all the methods combined into a MEGA json.

{
  "name": "advanced-package-json",
  "version": "0.1.0",
  "private": true,
  "config": {
    "SESSION_ENDPOINT": "my-value"
  },
  "dependencies": {
    "react": "^15.6.1",
  },
  "scripts": {
    "pass-arg": "node scripts/lol.js --name=$npm_config_name",
    "set-env": "REACT_APP_ENDPOINT=$npm_package_config_ENDPOINT react-scripts start",
    "cross-env": "cross-env NODE_ENV=prod OTHERFLAG=xyz webpack --config webpack.js",
    "my-script": "esw src/components",
    "pass-flags-to-other-script": "npm run my-script -- --watch",
  },
  "devDependencies": {
    "react-scripts": "^1.1.4",
    "cross-env": "^1.1.4"
  }
}

Do you know any other `package.json` tips?

Let me know in the comments below.

Go forth and

npm init

How to align images in markdown

David Wells — Mon, 29 Jul 2019 19:48:11 +0000

Originally posted on davidwells.io

Markdown is awesome. It's a fantastic way to write content quickly without the overhead of formatting with a WYSIWYG editor. If you are unfamiliar with .md files checkout the basics here & here.

Writing text in markdown is super quick and easy, but what about aligning images?

Read on to learn how! The full code can be found in this github gist

Image alignment in markdown

Normal markdown image tags don't allow for any alignment properties and thats a bummer when you are trying to make your README.md file pretty on github.

<!-- No alignment options -->
![GitHub Logo](/images/logo.png)

Luckily, we can use html image tags to make enhance our docs.

<!-- Alignment options!!!!! -->
<img align="left" width="100" height="100" src="http://www.fillmurray.com/100/100">

Left alignment

This is the code you need to align images to the left:

<img align="left" width="100" height="100" src="http://www.fillmurray.com/100/100">

Right alignment

This is the code you need to align images to the right:

<img align="right" width="100" height="100" src="http://www.fillmurray.com/100/100">

Center alignment example

Wrap images in a p or div to center.

<p align="center">
  <img width="460" height="300" src="http://www.fillmurray.com/460/300">
</p>

Markdown Formatting on steroids

If you like this, you might enjoy markdown-magic. I built it to automatically format markdown files and allow folks to sync docs/code/data from external sources.

Regex to match markdown links in Javascript

David Wells — Thu, 25 Jul 2019 16:50:12 +0000

Original posted on davidwells.io

Handy little regular expression to parse links from markdown files.

Links in markdown look like this:

[View the analytics docs](https://getanalytics.io/)

To parse .md links programmatically with a regular expression use this pattern:

/* Match only links that are fully qualified with https */
const fullLinkOnlyRegex = /^\[([\w\s\d]+)\]\((https?:\/\/[\w\d./?=#]+)\)$/

/* Match full links and relative paths */
const regex = /^\[([\w\s\d]+)\]\(((?:\/|https?:\/\/)[\w\d./?=#]+)\)$/

const string = "[View the analytics docs](https://getanalytics.io/)"

const myMatch = string.match(regex)

console.log(myMatch)
/* ["[View the analytics docs](https://getanalytics.io/)", "View the analytics docs", "https://getanalytics.io/", index: 0, input: "[View the analytics docs](https://getanalytics.io/)", groups: undefined] */

// de-structure the array
const [ full, text, url ] = myMatch

console.log(text)
// 'View the analytics docs'

console.log(url)
// 'https://getanalytics.io/'

See the demo on regex101

Parsing all links out of file

To grab all the links of of a file, you can use this:

const regexMdLinks = /\[([^\[]+)\](\(.*\))/gm

// Example md file contents
const mdContents = `
Lorem ipsum dolor sit amet, consectetur adipiscing elit..

[hello link](/admin/table_edit/table_edit.cfm?action=edit&table_name=organizationsXcategories)

Lorem ipsum dolor sit amet, consectetur adipiscing elit..

[otherLink](https://google.com)

Lorem ipsum dolor sit amet, consectetur adipiscing elit..

[third link](https://google.com)
`

const matches = mdContents.match(regexMdLinks)
console.log('links', matches)

const singleMatch = /\[([^\[]+)\]\((.*)\)/
for (var i = 0; i < matches.length; i++) {
  var text = singleMatch.exec(matches[i])
  console.log(`Match #${i}:`, text)
  console.log(`Word  #${i}: ${text[1]}`)
  console.log(`Link  #${i}: ${text[2]}`)
}

Comment below if you have any questions or updates to this snippet!

7 Serverless Auth Strategies for Protecting Gated Functions

David Wells — Thu, 25 Jul 2019 16:32:10 +0000

Originally posted on davidwells.io

How do we keep certain functions protected & scoped down to specific users (i.e. admin/paid users)?

In this post we will walk through the different strategies available for authorizing access to your serverless functions.

The code in this repo covers primarily AWS lambda functions but the general strategies can apply to any Functions as a service provider.

Pick your auth provider

There are a boatload of services that provide out of the box auth for your app. It's recommended to use one of these mainly because it's quite easy to mess up some piece of the security chain rolling your own auth.

Some options out there include:

Rolling your own auth is ill advised and against the serverless idea: "Focus on value to customers not the plumbing"

Choose a provider and proceed!

Choose a strategy

There are many ways to protect for your functions.

Inline auth checking
Middleware
"Legacy" middleware
Auth decorators
Custom authorizers
Proxy level
Single use access token
If you know of more please leave a comment on the post!

The list below will walk through them and the pros/cons of each.

1. Inline auth checking

Inlined function authentication happens inside your function code

We do a check on the auth headers or the body of the request to verify the user is okay to access the function.

const checkAuth = require('./utils/auth')

exports.handler = (event, context, callback) => {
  // Use the event data auth header to verify
  checkAuth(event).then((user) => {
    console.log('user', user)
    // Do stuff
    return callback(null, {
      statusCode: 200,
      body: JSON.stringify({
        data: true
      })
    })
  }).catch((error) => {
    console.log('error', error)
    // return error back to app
    return callback(null, {
      statusCode: 401,
      body: JSON.stringify({
        error: error.message,
      })
    })
  })
}

Benefits of this approach:

it's easy to do for a single function. Ship it!

Drawbacks of this approach:

this authentication method is hard to share across multiple functions as your API grows and can lead to non DRY code
Caching can be a challenge and if your authentication is an expensive operation or takes a while this can result in a slower UX and cost you more in compute time.

2. Middleware

Next up we have the middleware approach to authentication. This is still happening at the code level but now your logic that verifies the user is allowed to access the function is abstracted up a level into reusable middleware.

MiddyJs does a great job at enabling a sane middleware approach in lambda functions

const middy = require('middy')
const authMiddleware = require('./utils/middleware')

const protectedFunction = (event, context, callback) => {
  // Do my custom stuff
  console.log('⊂◉‿◉つ This is a protected function')

  return callback(null, {
    statusCode: 200,
    body: JSON.stringify({
      data: 'auth true'
    })
  })
}

exports.handler = middy(protectedFunction).use(authMiddleware())

Our middy middleware looks like this:

const checkAuth = require('./auth')

module.exports = function authMiddleware(config) {
  return ({
    before: (handler, next) => {
      checkAuth(handler.event).then((user) => {
        console.log('user', user)
        // set user data on event
        handler.event.user = user
        // We have the user, trigger next middleware
        return next()
      }).catch((error) => {
        console.log('error', error)
        return handler.callback(null, {
          statusCode: 401,
          body: JSON.stringify({
            error: error.message
          })
        })
      })
    }
  })
}

You can also instrument this yourself as seen in the movie demo(link here)

3. "Legacy" middleware

This middleware approach is using a familiar web framework with express PR flask and using their an auth module from their ecosystem.

In the case of express, you can use passport strategies in a lambda function

const express = require('express')
const cors = require('cors')
const bodyParser = require('body-parser')
const compression = require('compression')
const morgan = require('morgan')
const serverless = require('serverless-http')
const customLogger = require('./utils/logger')
const auth0CheckAuth = require('./utils/auth0')

/* initialize express */
const app = express()
const router = express.Router()

/*  gzip responses */
router.use(compression())

/* Setup protected routes */
router.get('/', auth0CheckAuth, (req, res) => {
  res.json({
    super: 'Secret stuff here'
  })
})

/* Attach request logger for AWS */
app.use(morgan(customLogger))

/* Attach routes to express instance */
const functionName = 'express'
const routerBasePath = (process.env.NODE_ENV === 'dev') ? `/${functionName}` : `/.netlify/functions/${functionName}/`
app.use(routerBasePath, router)

/* Apply express middlewares */
router.use(cors())
router.use(bodyParser.json())
router.use(bodyParser.urlencoded({ extended: true }))

/* Export lambda ready express app */
exports.handler = serverless(app)

Benefits of this approach:

Leverage existing code to GSD

Cons to this approach:

This takes a step backwards in the "serverless" approach to doing things because you have an entire express app bootstrapping on every incoming request
This will cost more over time with additional ms runtime because of express overhead
This introduces the idea that monoliths can work in lambda functions and this is considered an anti pattern

4. Auth decorators

Similar to auth middleware, decorators wrap the function code and return another function

Some developers prefer this more explicit approach as opposed to middleware

@AuthDecorator // <-- ref to auth wrapper function
function protectedFunction(event, context, callback) {
  // protected logic
}

5. Custom authorizers

Custom authorizers are a feature from AWS API gateway.

They are essentially another function that checks if the user is authorized to access the next function. If the auth checks out, then request then invokes the next lambda function.

Benefits to this approach:

authorization can be cached with a TTL (time to live). This can save on subsequent requests where the cached authentication doesn't need to make the potential slow auth check each time. This saves on compute time, ergo saves $$

Drawbacks to this approach:

you need to be using AWS API gateway to use custom authorizers

6. Proxy level

Similar to custom authorizers, you can verify requests at the proxy level.

This works in Netlify by checking for an http only secure cookie.

If the nf_jwt cookie exists in the request headers, Netlify will deserialize it and pass it into the context object of the lambda function

If the cookie is no valid, you can send the request to a non authorized endpoint (http code X)

# If visitor has 'nf_jwt' with role set, let them see site.
/.netlify/functions/protected-function /.netlify/functions/protected-function 200! Role=*

# Else, redirect them to login portal site.
/.netlify/functions/protected-function /not-allowed 401!

7. Single use access token

Some third party services like AWS, and faunaDB make it possible to use single use tokens in the client to invoke their APIs directly.

This means no function middleman to make the API calls to other services.

Benefits to this approach:

Cheaper (no function runtime to pay for)
Faster (no function latency in the middle)

Cons to this approach:

More complex to setup
Provider must support secure single use access tokens

For more information on this approach see AWS Cognito docs.

Mocking chainable APIs with ES6 JavaScript Proxies

David Wells — Tue, 23 Jul 2019 16:30:44 +0000

Originally posted on davidwells.io 🎉

ES6 proxies are pretty crazy.

Proxies give you the ability to intercept object calls and do pretty much whatever you want with them 🤯.

I highly recommend checking out this video from Michel Weststrate, the creator of immer, for a super deep dive on ES6 proxies and everything you can do with them.

Beware of proxies he warns, they are cool but can potentially lead to some back debugging issues.

Anywho, onto the use case...

How to mock a chainable API

I came across the need for replacing a chainable API inside the netlify-cli for the chalk module.

We needed a global mechanism for disabling terminal colors made with chalk.

There is a setting for this in chalk but that's the easy road out. We are a developer and we must re-invent the wheel. Also, wasn't working for my version of chalk...

So... Let's try some proxies!

First, what is a chainable API?

// Methods can chain together in any order
chalk.blue.bgRed.bold('Hello world!')

There are lots of libraries out there that allow for this type of flexibility.

You can actually use proxies to create chainable APIs

Proxy time

Thanks to safe-object-proxy, I found a proxy implementation that ensures that objects never throw errors if the keys on that object don't exist.

So no more this:

whatever.haha.yolo()
// Uncaught TypeError: Cannot read property 'yolo' of undefined

Instead, the proxy will magically make the function return null.

Pretty cool

There is a similar project out there called nevernull that comes packed with polyfills if you are running proxies in the browser.

With a little tweaking, console logging, & scratching my head on WTF proxies do, I managed to have the chainable API return my values no matter what.

Success 🎉

// safeChalk.js
const chalk = require('chalk')

/**
 * Chalk instance for CLI
 * @param  {boolean} noColors - disable chalk colors
 * @return {object} - chalk instance or proxy noOp
 */
module.exports = function safeChalk(noColors) {
  // if no colors return chainable "noOp" API
  if (noColors) {
    return NeverNull(chalk)
  }
  // else return normal chalk library
  return chalk
}

/* Chalk NoOp proxy */
function NeverNull(obj) {
  function match(some, none = noOp) {
    return obj != null ? some(obj) : none()
  }
  return new Proxy((some, none) => {
    /* Here was my tweak to make this work with chalks chainable API */
    if (some) return some

    if (!some && !none) return obj
    return match(some, none)
  }, {
    get: (target, key) => {
      const obj = target()
      if (obj !== null && typeof obj === 'object') {
        return NeverNull(obj[key])
      } else {
        return NeverNull()
      }
    },
    set: (target, key, val) => {
      const obj = target()
      if (obj !== null && typeof obj === 'object') {
        obj[key] = val
      }
      return true
    }
  })
}

function noOp() {}

And then using it

const safeChalk = require('./safeChalk')
/**
 * Usage
 */
const disableChalkColors = true
const myChalk = safeChalk(disableChalkColors)

console.log(myChalk.blue.bgRed.bold('Hello world!'))
// 'Hello world!' no coloring

const normalChalk = safeChalk()

console.log(normalChalk.blue.bgRed.bold('Hello world!'))
// 'Hello world!' blue text with red BG that is BOLD

We did it! The chainable API works no matter what!

Wrapping up

As you can see, the proxy is tiny and pretty powerful.

I'd recommend heeding Michael's word of warning with proxies. They are a bit "magical" and debugging proxies looks like a bad time.

What other use cases for proxies have you seen? Leave a comment below

What is GraphQL schema stitching & why do I need it?

David Wells — Sun, 14 Jan 2018 19:22:21 +0000

Explain it to me like I'm five!

What's are they and why do you need them when working with graphQL?

DEV Community: David Wells

Promisified node.js file system utilities

Analytics tip: How to trigger custom logic for utm campaign parameters

The campaign event

Using a custom plugin

Using a Listener

Simplify form handling in your app using valid HTML elements

Why you no HTML?

How can we simplify?

1. Using get-form-data

2. Using analytics-util-forms

Listening to form submissions with form-utils

Listening to form field changes with form-utils

Listening to submission & change events with form-utils

Installing

Keepin' things small

Wrapping up

A practical approach for clean analytics

Why Standardize?

How?

Event Naming Conventions

Format/Syntax

Conventions

Implementation

1. Map out existing projects & properties

2. Decide on naming conventions

3. Communicate changes to org

4. Implement & ship

Using analytics plugin

Bring your own convention

Research & links

5 advanced npm package.json config tips

1. Using NPM script pre & post lifecycle

2. Passing an argument into a command

3. Using values from package.json config key

4. Using cross-env for cross platform scripts

5. Passing arguments to other npm commands

Putting it all together

Do you know any other package.json tips?

How to align images in markdown

Image alignment in markdown

Left alignment

Right alignment

Center alignment example

Markdown Formatting on steroids

Regex to match markdown links in Javascript

Parsing all links out of file

7 Serverless Auth Strategies for Protecting Gated Functions

Pick your auth provider

Choose a strategy

1. Inline auth checking

2. Middleware

3. "Legacy" middleware

4. Auth decorators

5. Custom authorizers

6. Proxy level

7. Single use access token

Mocking chainable APIs with ES6 JavaScript Proxies

How to mock a chainable API

Proxy time

Wrapping up

What is GraphQL schema stitching & why do I need it?

The `campaign` event

1. Using NPM script `pre` & `post` lifecycle

3. Using values from package.json `config` key

4. Using `cross-env` for cross platform scripts

Do you know any other `package.json` tips?