DEV Community: Kevin Lewis

Announcing the Directus Panel Quest Hackathon

Kevin Lewis — Mon, 09 Oct 2023 11:54:16 +0000

Following the success of the Directus AI Hackathon in August, we are today announcing a brand new contest for October. Directus Insights provides a fantastic canvas for helping people across your organization understand data. During Directus Panel Quest, we're challenging you to build new extensions to enhance and expand your dashboards.

Key Dates

Hackathon starts: October 4 2023 at 00:00 UTC.
Submissions end: October 31 2023 at 23:59 UTC.
Voting period: November 1 2023 until November 8 2023 at 23:59 UTC.
Winners announced: November 10 2023.

The Challenge

The Directus Insights Module is more powerful than many users realize. Data from your collections can be visualized to glean insights from your Directus project, but panels can do a lot more. You can display and visualize external data via API, or even build forms to interact with your data or external services.

The challenge of Panel Quest is to build new Panel Extensions that interact with or integrate with external services or data. Since Panels only run on the frontend, you should submit a bundle that includes any endpoints required to power your Panel.

Any external API or data will be accepted for this hackathon. Here are a few categories and ideas:

Communication APIs like Twilio, Vonage, Discord, and Slack
Marketing platforms like Salesforce, HubSpot, and Mailchimp.
Developer workflow tools like CircleCI, GitHub, and Vercel.
AI and Machine Learning APIs like OpenAI, Deepgram, and Replicate.
Payment and E-commerce platforms like Stripe, Shopify, and GoCardless.
Device and Browser APIs like sensors, location, and speech.

Submitting Your Project

To submit your project, you must:

Add your extension code to a public repository on GitHub.
Use our README template and replace anything in curly brackets with your submission's value.
Add the directus and directus-panel-quest topics (tags) to your GitHub repository.
Fill in this form to provide your email address so we can get in touch with the results.

We encourage all projects to include a LICENSE file in your repository with the Apache License 2.0, the GNU General Public License v3.0, or the MIT License.

The submission period ends on October 31 2023 at 23:59 UTC. You must complete all of the steps above by this time to be eligible. If you work with others, only one person must complete the steps.

Judging

On November 1 2023, we will curate all valid submissions into a blog post on the Directus Developer Blog with links to the last commit before the submission deadline (further commits will not be assessed).

Core Team Vote

This award is decided by the core team behind Directus. Each team member is given one vote to cast. When judging projects, we will look at the following criteria with no specific weighting:

Creative: Is the idea original and innovative?
Technical: How effectively do you integrate the external AP or dataI? Is the project well-executed? Does it run?
Relevance: Do you solve a real problem faced by people?
Presentation: Have you taken time to add polish to your project?

Judging will take place from November 1 2023 until November 8 at 23:59 UTC.

Community Vote

One of our prizes is decided by the community. From the valid submissions posted on our blog, the repository with the most GitHub Stars before the end of the judging period will win this category. No stars given or removed after the end of the judging period will be accepted.

The Prizes

Both the Directus Core Team and Community Vote prizes are the same:

$1000 USD awarded to the project submitter.
$250 USD donation to a charity of the winner’s choice from the following list:
- codebar - a charity that facilitates the growth of a diverse tech community by running free regular programming workshops for minority groups in tech.
- VetsInTech - who support transitioning military, veterans and spouses of either with reintegration services and by connecting them to the national technology ecosystem.
- Out In Tech - the world’s largest non-profit community of LGBTQ+ tech leaders.
- Carbon180 works toward developing equitable, science-based policies that will scale carbon removal.
- Cool Earth works with rainforest communities, tackles the root causes of deforestation, and protects vital carbon sinks.

Winners will be announced at a live Discord event on November 10 2023, and winners contacted over email on the same day.

Resources

We've created a few guides to get you started:

Create a Panel With Dynamic Form Based On Collection Fields shows you how to build dynamic user forms inside of panels, and create new items in your collections.
Create A Panel To Display External API Data With Vonage shows you how to visualize external data in your dashboards without needing to import data to Directus first.
Create An Interactive Panel To Send SMS With Twilio shows you how to build a polished and flexible panel that submits external POST requests.

We also have a few videos to show you how to create extensions:

This last one has an accompanying blog post: Using External Weather Data In A Custom Panel Extension

Additional Notes

Multiple submissions are allowed, and will be judged separately.
You own your projects - our incentive is to see the cool things people build.
We don’t handle prize-splitting, so if you’re working with others and win, we’ll send the prize to the Team Lead and they’ll be responsible for distributing it.
You can find the Terms & Conditions for this contest here.

Get Started

You can run Directus locally using Docker in just a few minutes

You can find educational resources here on the Directus Developer Blog, in our extension-building guides, and on our YouTube channel.

At any point during the hacking period, feel free to drop by our Discord server and ask for help, or ask questions about the hackathon in the #hackathon channel. We also have fortnightly Office Hours events on Thursdays which we’ll dedicate to the hackathon throughout October.

Using External Weather Data In A Custom Panel Extension

Kevin Lewis — Mon, 09 Oct 2023 11:51:27 +0000

In this post, you will learn how to fetch data from an external data source and display it in a custom panel extension for Directus Insights.

Panels can only talk to internal Directus services, and can't reliably make external web requests because browser security protections prevent these cross-origin requests from being made. To create a panel that can interact with external APIs, you will create bundle including an endpoint (that can make external requests) and a panel (that uses the endpoint).

Add an Extensions Volume

Follow our Directus Self-Hosted Quickstart, adding a volume for extensions:

volumes:
  - ./database:/directus/database
  - ./uploads:/directus/uploads
  - ./extensions:/directus/extensions // [!code ++]

Create a Bundle

Once you have run docker compose up for the first time, local directories for the volumes will be created. Navigate to the extensions directory and use the Directus Extensions CLI to create a bundle:

npx create-directus-extension@latest
├ type: bundle
├ name: directus-extension-bundle-weather
└ language: javascript

Then, navigate to the newly-created extension directory.

Create an Endpoint

Use npm run add to add a new extension to the bundle:

npm run add
├ type: endpoint
├ name: weather-endpoint
└ language: javascript

Navigate to the new weather-endpoint/index.js file and replace it with the following:

export default {
    id: 'weather',
    handler: (router) => {
        router.get('/', async (req, res) => {
            try {
                const response = await fetch(`https://api.open-meteo.com/v1/forecast?current_weather=true&${req._parsedUrl.query}`);

                if (response.ok) {
                    res.json(await response.json());
                } else { 
                    res.status(response.status).send(response.statusText);
                }
            } catch(error) {
                res.status(500).send(error.message);
            }
        })
    }
}

From the directus-extension-bundle-weather directory, run npm run build. Restart your Directus project, and you should now be able to access http://localhost:8055/weather?longitude=0&latitude=0.

The Open-Meteo API requires a longitude and latitude, so they must always be provided.

Create a Panel

Use npm run add to add a new extension to the bundle:

npm run add
├ type: panel
├ name: weather-panel
└ language: javascript

Navigate to the new weather-panel/index.js file, update the id to weather-panel and the name to Weather.

In the options array, remove the text field, and add two new fields for longitude and latitude:

options: [
    {
        field: 'longitude',
        name: 'Longitude',
        type: 'string',
        meta: {
            interface: 'input',
            width: 'half',
        },
    },
    {
        field: 'latitude',
        name: 'Latitude',
        type: 'string',
        meta: {
            interface: 'input',
            width: 'half',
        },
    },
],

Open panel.vue, and at the top of the <script>, import useApi and Vue's ref:

import { useApi } from '@directus/extensions-sdk'; // [!code ++]
import { ref } from 'vue'; // [!code ++]

export default {

Then, add longitude and latitude props:

props: {
    showHeader: {
        type: Boolean,
        default: false,
    },
    longitude: { // [!code ++]
        type: String,  // [!code ++]
        default: '0',  // [!code ++]
    },  // [!code ++]
    latitude: {  // [!code ++]
        type: String,  // [!code ++]
        default: '0',  // [!code ++]
    },  // [!code ++]
},

Create a setup method which will run when the panel is loaded:

setup(props) {
    const api = useApi();
    const weather = ref({});

    async function fetchData() {
        const response = await api.get(`/weather?longitude=${props.longitude}&latitude=${props.latitude}`);
        weather.value = response.data;
    }
    fetchData();

    return { weather };
}

This code will immediately fetch weather data using the new internal endpoint, and then make it available as weather to the template.

Finally, update the template:

<template>
    <div class="text" :class="{ 'has-header': showHeader }">
        {{ text }}  // [!code --]
        {{ weather }}  // [!code ++]
    </div>
</template>

Run npm run build from the bundle directory, and restart Directus.

Use the Panel

Create a new Insights Dashboard and add a Weather panel. Add coordinates, and you should see external data displayed in the panel.

You can now create panels using external data. If you have any questions at all, feel free to join our Discord community.

Detecting High-Risk Phone Numbers with Vonage and Directus Automate

Kevin Lewis — Tue, 19 Sep 2023 09:32:37 +0000

When creating new users for your service, it's important to take steps to prevent fraudulent or malicious activity. In this post, you'll use Vonage's Number Insight V2 API and Directus Automate to determine the likelihood of a number being risky at the time of user registration, and let you act on it.

The Vonage Number Insight V2 API assigns a fraud score to numbers, along with a risk recommendation - allow, flag, or block. You can use this recommendation to allow user creation, allow it with a note to your team to validate or block creation.

Before You Start

You will need a Directus project - check out our quickstart guide if you don't already have one. You will also need a Vonage Developer API account, taking note of your API Key and Secret. You should also have a high-risk number to test with (I used the last spam caller I had).

Finally, in Your Directus project, add an input field called phone_number to the directus_users collection. This is a system collection, so you will need to expand them in the Data Model settings in order to see the directus_users collection.

Set Up Trigger

Create a new Flow from your Directus project settings - call it "Check Phone Fraud Risk". Create an Event Hook trigger that is Blocking - this means the flow will run before the data is entered in the database. Set the Scope to items.create on the directus_users collection - this means the flow will start whenever a new user is created.

Set Up Number Insight Check

When authenticating your Vonage API call, you must provide your API Key and Secret Base64-encoded. Find an online encoder, and encode the following your_key:your_secret (the colon is important). Take note of the encoded string.

Create a Webhook / Request URL operation and set the Key to check_number. Setting the key inserts the data into the Flow's data chain using this property name. Set the Method to POST and the URL to [https://api.nexmo.com/v2/ni](https://api.nexmo.com/v2/ni).

Set one header:

Authorization: Basic BASE_64_ENCODED_AUTH_STRING

In the body, send the following JSON:

{
   "type": "phone",
   "phone": "{{$trigger.payload.phone_number}}",
   "insights": ["fraud_score"]
}

Navigate to the Users Module in the module bar and create a new user. You can leave all information blank apart from the phone number - make sure it's in E.164 format. If you need to format numbers, check out a section in our post on building a wedding invite system - it also uses a Vonage API so you can reuse your credentials.

Come back to your flow and open the logs in the sidebar. You can see the operation outputs an object containing a fraud_score and related information. The risk_score is a scale of 0 to 100. This project will use the risk_recommendation value of allow, flag, or block in future steps.

Set Up Conditionals For Fraud Recommendation Result

From the resolved path of the API request, create a Condition operation with the following rule:

{
    "check_number": {
        "data": {
            "fraud_score": {
                "risk_recommendation": {
                    "_eq": "allow"
                }
            }
        }
    }
}

The resolved path will be followed if the risk_recommendation is allow. If a flow ends on a resolved path, the Blocking Flow will end and the item will be added to the collection.

The reject path will be followed if the value is anything else. From the reject path, create another Condition operation to determine whether the recommendation is flag or block:

{
    "check_number": {
        "data": {
            "fraud_score": {
                "risk_recommendation": {
                    "_eq": "flag"
                }
            }
        }
    }
}

The resolved path will be flag, and the reject path will be block.

The flow now looks like this:

Send Emails If Flagged or Blocked

In this tutorial, a flagged user will still be created, but an email will be sent to an internal team member to perform manual validation. From the resolved path of Is Flagged create a Send Email operation. Add an email address, and inject dynamic values in the Subject and Body:

— Subject —

Flagged Phone Number from {{$trigger.payload.first_name}} {{$trigger.payload.last_name}}

— Body —

The phone number {{$trigger.payload.phone_number}} was flagged by the Vonage Number Insight system with a score of {{check_number.data.fraud_score.risk_score}}. 

Please manually validate this user's profile before approving their jobs.

Reject New User Item If Blocked

From the reject path of Is Flagged create a Send Email operation. Form a rejection message and send it to the user who tried to register using {{$trigger.payload.email}} as the recipient.

When a Blocking Flow concludes, data is entered into the database. Currently, there is no elegant way to stop this, but there is a reliable way to make this happen. After the email operation that sends the rejection, create a Run Script operation from the resolved path:

throw new Error('Phone number failed fraud checks');

This will cause the Flow to fail and not enter the item into the directus_users collection.

Your final flow should look like this:

Summary

In this post, you have learned how to use the Vonage Number Insight V2 API to check a phone number for the likelihood of fraud at the time of new user registration. Based on the outcome, users are created, flagged, or blocked from being created.

Based on the allow, flag, or block recommendation, you can add any operations from those provided by Directus, or through building your own.

If you have any questions, please feel free to join our Discord server and ask them.

Building a Wedding Invite System with Node.js, Vonage, and Directus

Kevin Lewis — Mon, 14 Aug 2023 07:08:07 +0000

My wedding is coming up in just over a week, which is exciting. But you know what isn't exciting? Getting a confirmed guest list with all of the variations in invites and information gathered consistently. I'm going to show you how I built my wedding invite system with various types of invite (all-day/ceremony-only), auto-formatted contact information when RSVPs were submitted, and generated a constantly-updated guest-list.

Before You Start

You will need Node.js and a text editor installed on your computer.

You will also need a Directus project running - either using Directus Cloud or by Self Hosting - and an API token for your admin user.

Finally, sign up for a Vonage Developer account and take note of your API Key and Secret.

In your Directus project, create the following collections (and fields):

people - Primary Key Field: id (Generated UUID)
- first_name (Type: String, Interface: Input)
- last_name (Type: String, Interface: Input)
- rsvp (Type: Boolean, Interface: Toggle)
- requirements (Type: String, Interface: Input)
- is_plus_one (Type: Boolean, Interface: Toggle)
invites - Primary Key Field: slug (Manually entered string)
- message (Type: String, Interface: Input)
- people (Type: Alias, Interface: One to Many)
  - Related Collection: people
  - Foreign Key: invite
  - Display Template First Name Last Name
- plus_one_allowed (Type: Boolean, Interface: Toggle)
- ceremony_invite (Type: Boolean, Interface: Toggle)
- completed (Type: DateTime, Interface: Datetime)
- email (Type: String, Interface: Input)
- phone (Type: String, Interface: Input)

One invite will have multiple people, and optional characteristics (allowed +1, invited to ceremony). When guests respond, they will provide requirements (dietary/accessibility), and the person filling it in will provide an email and phone number. Finally, if given a +1 and it's used, a new person will be created and is_plus_one will be set to true.

Here's an illustration of an invite page, annotated to show where values in Directus will alter it.

The most minimal invite (one guest, reception-only, no +1) looks like this:

Create a new directory for this project and open it in a code editor. Create a .env file and populate it with your keys:

DIRECTUS_URL=your_project_url_here
DIRECTUS_TOKEN=your_admin_token_here

Run npx gitignore node to create a suitable .gitignore file for this project. Create a package.json file with npm init -y and then install our dependencies:

npm install dotenv express hbs @directus/sdk

If you don't want to follow along, you can now skip to the bottom of this tutorial for the complete code.

Create an index.js file and open it in your code editor.

Set Up Application

Create and set up an Express.js application with the handlebars view engine:

import 'dotenv/config';
import express from 'express';
import { engine } from 'express-handlebars';

const app = express();
app.use(express.static('public'));
app.use(express.urlencoded({ extended: true }));

app.engine('handlebars', engine());
app.set('view engine', 'handlebars');
app.set('views', './views');

app.get('/invite/:slug', (req, res) => {
  res.render('invite', { 
    layout: false, 
    name: req.params.slug 
  });
});

app.listen(3000);

Create a views directory and, inside of it, an invite.handlebars file:

<!DOCTYPE html>
<html>
<body>
   {{ name }}
</body>
</html>

Run your application with node index.js in the terminal, and navigate to localhost:3000/invite/hello. You should see a page with 'hello'.

Set Up Directus SDK

Import and initialize a new Directus client with rest features. Import the createItem, readItem, and updateItem functions for later.

import 'dotenv/config';
import express from 'express';
import { engine } from 'express-handlebars';

import { createDirectus, staticToken, rest, createItem, readItem, updateItem } from '@directus/sdk'; // [!code ++]
const directus = createDirectus(process.env.DIRECTUS_URL) // [!code ++]
  .with(rest()) // [!code ++]
  .with(staticToken(process.env.DIRECTUS_TOKEN)); // [!code ++]

Create Invite Pages

Update the invite route handler to utilize the Directus SDK and read the invite from your Directus project.

app.get('/invite/:slug', async (req, res) => {
  const data = await directus.request( // [!code ++]
    readItem('invites', req.params.slug, { // [!code ++]
      fields: ['*', { people: [ 'id', 'first_name', 'last_name' ] }] // [!code ++]
    }) // [!code ++]
  ); // [!code ++]

  res.render('invite', { 
    layout: false, 
    name: req.params.slug // [!code --]
    ...data // [!code ++]
  }); 
});

Update the content of the <body> tag to include values from the invite:

<body>
  <h1>You're invited!</h1>
  <p>{{ message }}</p>
  <div>
    {{#if ceremony_invite}}
      <p>Ceremony-specific information</p>
    {{/if}}
    <p>Reception-specific information</p>
  </div>
</body>

Show everyone the personalized message as part of their invite and the reception details. Only a subset of guests are invited to the ceremony as denoted by cermeony_invite being set to true.

Create five items in the people collection with just a name, and four invites to test:

ceremony_invite = false, plus_one_allowed = false, slug = 'a'
ceremony_invite = true, plus_one_allowed = false, slug = 'b'
ceremony_invite = false, plus_one_allowed = true, slug = 'c'
ceremony_invite = true, plus_one_allowed = true, slug = 'd' (assign two people to this invite)

This should give all key permutations to test against. Restart your server, navigate to localhost:3000/invite/a and you should see only the reception information. When you go to localhost:3000/invite/b you should see ceremony information as well.

Build RSVP Form

Underneath the existing <div>, add a conditional that will only show the form is the invite has not yet been RSVP'd to:

{{#if completed}}
  <p>Thanks for responding</p>
{{else}}
  <!-- Further code goes here -->
{{/if}}

In the {{else}} block, create a form:

<form action="/rsvp" method="POST">

  {{#each people}}
    <div style="border: 1px solid black;">
      <h2>{{ this.first_name }} {{ this.last_name }}</h2>

      <input type="radio" name="rsvp-{{ this.id }}" value="false" id="rsvp-no-{{ this.id }}" required>
      <label for="rsvp-no-{{ this.id }}">Accept</label><br>

      <input type="radio" name="rsvp-{{ this.id }}" value="true" id="rsvp-yes-{{ this.id }}" required>
      <label for="rsvp-yes-{{ this.id }}">Regrets</label><br>

      <input type="text" name="requirements-{{ this.id }}" id="requirements-{{ this.id }}" placeholder="Requirements">
    </div><br>
  {{/each}}

  <p>Only one needed per invite</p>
  <input type="email" placeholder="Email" name="email" required>
  <input type="text" placeholder="Phone" name="phone" required>
  <input type="hidden" value="{{ slug }}" name="slug">
  <input type="submit">
</form>

Let's break this down:

The form will submit a POST request to /rsvp. This will be created later.
For every person assigned to this invite, create a set of three inputs - a pair of radio buttons for RSVP yes/no, and one to optionally collect any accessibility or dietary requirements. Note that all inputs have a dynamic name which includes that person's ID from the database. For example, name="rsvp-{{ this.id }}" would become name="rsvp-ff028423-7111…"
There is one email and phone number collected per RSVP.
A hidden field contains the slug, so it is also submitted with the rest of the form.

Add +1 Form Fields

If the invite has plus_one_allowed set to true, show additional fields to collect their information:

{{/each}}

{{#if plus_one_given}} // [!code ++]
  <div style="border: 1px solid black;"> // [!code ++]
    <h2>Your +1</h2> // [!code ++]
    <input type="text" name="first-name-plus-one" id="first-name-plus-one" placeholder="First Name"> // [!code ++]
    <input type="text" name="last-name-plus-one" id="last-name-plus-one" placeholder="Last Name"> // [!code ++]
    <input type="text" name="requirements-plus-one" id="requirements-plus-one" placeholder="Requirements"> // [!code ++]
  </div> // [!code ++]
{{/if}} // [!code ++]

<p>Only one needed per invite</p>

Restart your server, navigate to localhost:3000/invite/b and you should not see the +1 fields. When you go to localhost:3000/invite/d you should. Here's what the form looks like when a +1 is allowed on the invite:

It isn't winning any design awards, but it works.

Submit RSVP

Create a new route handler in index.js for the POST submission:

app.post('/rsvp', async (req, res) => {
  console.log(req.body);
});

Restart the server, fill and submit the form. The terminal should look something like this:

Update People With Their RSVP

To get a list of only people IDs, filter this object to an array of just the properties starting with rsvp, and then remove rsvp- from the start of the strings. Under the console.log:

const rsvps = Object.keys(req.body)
  .filter(k => k.includes('rsvp'))
  .map(k => k.split('rsvp-')[1]);

Then, loop through these IDs and update each person with their RSVP and requirements:

for(let rsvp of rsvps) {
  await directus.request(
    updateItem('people', rsvp, { 
      rsvp: req.body[`rsvp-${rsvp}`],
      requirements: req.body[`requirements-${rsvp}`]
    })
  );
}

Create Person Item For +1

If the +1 form was filled in, create a new person and add their ID to the RSVPs list. Under the for loop add the following:

if(req.body['first-name-plus-one']) {
  const { id } = await directus.request(
  createItem('people', {
    first_name: req.body['first-name-plus-one'],
    last_name: req.body['last-name-plus-one'],
    requirements: req.body['requirements-plus-one'],
    is_plus_one: true,
    rsvp: true
   })
  );
  rsvps.push(id)
}

Update Invite

Each invite contains a phone number and email address of the lead contact. Under the for loop, update the invite:

await directus.request(
  updateItem('invites', req.body.slug, { 
    email: req.body.email,
    phone: req.body.phone,
    people: rsvps,
    completed: new Date()
  })
);

If there was a +1, the new person will be added to the invite. The value of completed will also be set to the current Datetime. In Directus, you'll easily be able to see when an RSVP was made. In the application, this will replace the form with the message "Thanks for responding".

As the form will not appear when completed has a value, redirect back to the same page and it will appear as if the form has been replaced with the thank you message:

res.redirect(`/invite/${req.body.slug}`);

Standardize Phone Numbers

I can tell you from past experience - no one ever fills in their phone number consistently, and I wanted to build a batch update system so we could send people important messages as the day approaches.

To format numbers, I used the Vonage Number Insight API. In your Directus Project Settings create a new Flow.

Use an non-blocking Event Hook Trigger so the item is added to the collection and then the flow is run. Keep the scope to only the items.update on the Invites collection.

Add a new Request URL operation and set the URL to https://api.nexmo.com/ni/basic/json?api_key=VONAGE_API_KEY&api_secret=VONAGE_API_SECRET&number={{$trigger.payload.phone}}, being sure to change the values for your key and secret.

Once the request has been successful, add an Update Data operation on the Invites collection. Set the payload to the following:

{
    "phone": "{{$last.data.international_format_number}}"
}

Save the Flow and test it by submitting a new invite RSVP. You may need to manually set an invite's Completed value to null to see the form again. The whole flow should look like this:

See Guest List

To see a running guest-list, navigate to the People collection in the content module and add a filter RSVP = true. You can see just the ceremony list by adding the filter Invite -> Ceremony Invite = true.

Summary & Next Steps

In this tutorial, you have created a wedding invite and RSVP system backed by a Directus database, and then standardized provided data using the Vonage Number Insight API.

Now that you have standardized data, you may choose to send batch messages via SMS or email using Flows.

If you have any questions or feedback, feel free to join our Discord server.

Complete Code

index.js

import 'dotenv/config';
import express from 'express';
import { engine } from 'express-handlebars';
import { createDirectus, staticToken, rest, createItem, readItem, updateItem } from '@directus/sdk';
const directus = createDirectus(process.env.DIRECTUS_URL)
  .with(rest())
  .with(staticToken(process.env.DIRECTUS_TOKEN));

const app = express();
app.use(express.static('public'));
app.use(express.urlencoded({ extended: true }));

app.engine('handlebars', engine());
app.set('view engine', 'handlebars');
app.set('views', './views');

app.get('/invite/:slug', async (req, res) => {
  const data = await directus.request(
    readItem('invites', req.params.slug, { 
      fields: ['*', { people: [ 'id', 'first_name', 'last_name' ] }] 
    })
  );
  res.render('invite', { layout: false, ...data });
});

app.post('/rsvp', async (req, res) => {
  console.log(req.body)
  // Get list of RSVP IDs
  const rsvps = Object.keys(req.body)
    .filter(k => k.includes('rsvp'))
    .map(k => k.split('rsvp-')[1]);

  // Update people with RSVPs
  for(let rsvp of rsvps) {
    await directus.request(
      updateItem('people', rsvp, { 
        rsvp: req.body[`rsvp-${rsvp}`],
        requirements: req.body[`requirements-${rsvp}`]
      })
    );
  }

  // Add +1 if exists
  if(req.body['first-name-plus-one']) {
    const { id } = await directus.request(
      createItem('people', {
        first_name: req.body['first-name-plus-one'],
        last_name: req.body['last-name-plus-one'],
        requirements: req.body['requirements-plus-one'],
        is_plus_one: true,
        rsvp: true
      })
    );
    rsvps.push(id);
  }

  // Update invite
  await directus.request(
    updateItem('invites', req.body.slug, { 
      email: req.body.email,
      phone: req.body.phone,
      people: rsvps,
      completed: new Date()
    })
  );

  res.redirect(`/invite/${req.body.slug}`);
});

app.listen(3000);

views/invite.handlebars

<!DOCTYPE html>
<html>
<body>
  <h1>You're invited!</h1>
  <p>{{ message }}</p>
  <div>
    {{#if ceremony_invite}}
      <p>Ceremony-specific information</p>
    {{/if}}
    <p>Reception-specific information</p>
  </div>
  {{#if completed}}
    <p>Thanks for responding</p>
  {{else}}
    <form action="/rsvp" method="POST">
      {{#each people}}
        <div style="border: 1px solid black;">
          <h2>{{ this.first_name }} {{ this.last_name }}</h2>

          <input type="radio" name="rsvp-{{ this.id }}" value="false" id="rsvp-no-{{ this.id }}" required>
          <label for="rsvp-no-{{ this.id }}">Accept</label><br>

          <input type="radio" name="rsvp-{{ this.id }}" value="true" id="rsvp-yes-{{ this.id }}" required>
          <label for="rsvp-yes-{{ this.id }}">Regrets</label><br>

          <input type="text" name="requirements-{{ this.id }}" id="requirements-{{ this.id }}" placeholder="Requirements">
        </div>
        <br>
      {{/each}}
      {{#if plus_one_allowed}}
      <div style="border: 1px solid black;">
        <h2>Your +1</h2>
        <input type="text" name="first-name-plus-one" id="first-name-plus-one" placeholder="First Name">
        <input type="text" name="last-name-plus-one" id="last-name-plus-one" placeholder="Last Name">
        <input type="text" name="requirements-plus-one" id="requirements-plus-one" placeholder="Requirements">
      </div>
      {{/if}}
      <p>Only one needed per invite</p>
      <input type="email" placeholder="Email" name="email" required>
      <input type="text" placeholder="Phone" name="phone" required>
      <input type="hidden" value="{{ slug }}" name="slug">
      <input type="submit">
    </form>
  {{/if}}
</body>
</html>

}

Announcing the Directus AI Hackathon

Kevin Lewis — Wed, 09 Aug 2023 14:40:55 +0000

We're excited to announce our very first Directus Hackathon - this time with the theme of AI. Directus provides an extensive toolkit to work with data and is perfect for huge enterprise and hobby projects alike. We’re challenging you to bring your creativity and technical skills to the table for the chance to win cash prizes and swag.

Key Information

Start: August 3 2023 at 00:00 UTC.
End: August 31 2023 at 23:59 UTC.
One challenge: Build a project that utilizes AI and Directus.
Two prizes: $1000 USD and a $250 USD charity donation from our list of charities. One decided by the Directus Core Team, one by community voting.

The Theme

We want to see how you would combine Directus and AI features by creating new Directus extensions.

You may choose to build a new Insights Panel for our dashboards, a custom Flows Operation for our automations feature, a cool Editor Interface to support content creation or review, or any of the other extensions types.

We’re pretty open to seeing what you build, so we’ve left the scope quite wide for this one.

Submitting Your Project

To submit your project, you must:

Add your extension code to a public repository on GitHub.
Use our README template and replace anything in curly brackets with your submission's value.
Add the directus and directus-ai-hackathon tags to your GitHub repository.
Fill in this form to provide your email address so we can get in touch with results.

We encourage all projects to include a LICENSE file in your repository with the Apache License 2.0, the GNU General Public License v3.0, or the MIT License.

The submission period ends on August 31 2023 at 23:59 UTC. You must complete all of the steps above by this time to be eligible. If you work with others, only one person must complete the steps.

Judging

On September 1 2023, we will curate all valid submissions into a blog post on the Directus Developer Blog with links to the last commit before the submission deadline (further commits will not be assessed).

When judging projects, we will look at the following criteria with no specific weighting:

Creative: Is the idea original and innovative?
Technical: How effectively do you integrate AI? Is the project well-executed? Does it run?
Relevance: Do you solve a real problem faced by people?
Presentation: Have you taken time to add polish to your project?

Judging will take place from September 1 2023 until September 10 at 23:59 UTC.

Community Vote

The Prizes

Both the Directus Core Team and Community Vote prizes are the same:

$1000 USD awarded to the project submitter.
$250 USD donation to a charity of the winner’s choice from the following list:
- codebar - a charity that facilitates the growth of a diverse tech community by running free regular programming workshops for minority groups in tech.
- VetsInTech - who support transitioning military, veterans and spouses of either with reintegration services, and by connecting them to the national technology ecosystem.
- Out In Tech - the world’s largest non-profit community of LGBTQ+ tech leaders.
- Carbon180 works toward developing equitable, science-based policies that will scale carbon removal.
- Cool Earth works with rainforest communities, tackles the root causes of deforestation, and protects vital carbon sinks.

Winners will be announced at a live Discord event on September 11 2023, and winners contacted over email on the same day.

Get Started

You can get started using a hosted Directus Cloud project or by running your own locally, using Docker, in just a few minutes

You can find educational resources here on the Directus Developer Blog, in our docs guides, and on our YouTube channel.

At any point during the hacking period, feel free to drop by our Discord server and ask for help in the #help-general channel, or ask questions about the hackathon in the #hackathon channel. We also have weekly Office Hours events on Thursdays which we’ll dedicate to the hackathon throughout August.

Additional Notes

Multiple submissions are allowed, and will be judged separately.
You own your projects - our incentive is to see the cool things people build.
We don’t handle prize-splitting, so if you’re working with others and win, we’ll send the prize to the Team Lead and they’ll be responsible for distributing it.
You can find the Terms & Conditions for this contest here.

Let’s Go!

This is the first time we’re running our own hackathon, and we’re always open to adapting the format for future editions. Please do provide feedback and what does and doesn’t work for you.

We are so excited to see your projects and the cool ideas you come up with!

Migrating From Nuxt Content To Directus

Kevin Lewis — Wed, 09 Aug 2023 14:39:21 +0000

Away from my day job here at Directus I run a free library of online content focused around core skills.

Up until now, I leveraged Nuxt Content, a file-based CMS, for content management. Nuxt Content served the project well, providing nice utilities for fetching and displaying data from a content directory in the repository. Being file-based, authoring used a git-based workflow, meaning content is stored as files, and changes are tracked using version control.

Recently, I've been lucky enough to bring on a team member for a few hours a week to help run the project, and while I feel at home spinning up the application locally, working in code to add and edit content, it isn't fair or reasonable to expect that of others.

Directus is an API-driven headless CMS. Unlike file-based CMSs like Nuxt Content, Directus separates the content from the presentation layer. This decoupling means that non-developers can handle content updates without needing to understand the underlying code. They can work in a user-friendly interface, updating content, and the website pulls data from the API to construct pages.

Many projects start the same way as mine, and I hope sharing this journey of maturing our tooling and processes helps!

Hacky Fixes With Nuxt Content

I was pretty happy with the latest iteration of the You Got This! website prior to the Directus migration, but there were some challenges with the setup that led to some hacky workarounds…

Co-Locating Assets With Content

Nuxt Content allows you to create, query, and fetch data in the content directory such as markdown. However, assets like blog post images can't be served relative to the data files.

In the past, this meant any given piece of content needed to be split between the content and directly-served static directory. I built a helper which would, on server boot, copy all non-markdown files to a mirrored file structure in the static directory (not checked in to git). This means I could still use relative file paths.

This was quite hacky and had some major drawbacks:

Only run on server boot. I'm sure I could have utilized a hook which could re-copy files when content is updated, but that's not how my utility worked.
If the content file structure is not the same as the routing structure, which it often wasn't, it led to hacky file path construction. For example, an article would be stored in the /content/2023/post-slug directory, but served at /library/post-slug, meaning the directory mirroring did not help.

Grappling With No Relationships

There is a surprising number of relationships between different content types in this project.

Nuxt Content doesn't really have a concept of relationships. Within each of these content types, I manually created lists of 'items' that match the directory names for items I want to reference. These were then used in multiple round-trips to fetch all required data for a page. For example:

// Get single collection based on page URL
const collection = await $content('collections', params.slug).fetch()

// Get all library items, regardless of relevance
const library = await $content('library').fetch()

// Filter library items to those present in collection.items frontmatter
const items = library.filter(libItem => collection.items.find(colItem => libItem.path.includes(colItem)))

This is two distinct round-trips for data, and then manual work to reconcile them. If there are one-character typos, expect things to not act as expected.

Setting Up A Directus Project

I had a loose concept of relationships through manual creation of lists within markdown files, but for the first time, I had to sit down and think properly about the relationships between every data type in my project.

There were some clear low-level entities to start with that don't rely on others - people and sponsors. The library is next - which has a many-to-many (M2M) relationship with people. The content collections link to the library and sponsors. Finally, events link to people and sponsors. It was important to create these in the right order (mostly to make sure I didn't get confused). I also created asset folders for each of these collections.

Moving Data To Directus

I wrote scripts that used the Directus JavaScript SDK to move each type of data over - I chose to do one collection at a time so I was able to spot any issues. Here's an example of the sponsors migration script:

require('dotenv').config()
const fs = require('fs')
const fetch = require('cross-fetch')
const { Directus } = require('@directus/sdk')
const directus = new Directus(process.env.DIRECTUS_URL, { auth: { staticToken: process.env.DIRECTUS_TOKEN } })

const ASSET_FOLDER = 'Sponsors'
const CONTENT_DIR = 'content/sponsors'
const CONTENT_API_BASE = 'http://localhost:3000/_content/sponsors/'

(async () => {
  // Get id for Sponsors assets folder
  const { data: folders } = await directus.items('directus_folders').readByQuery()
  const { id: folder } = folders.find(f => f.name == ASSET_FOLDER)

  // Get all subfolders of content/sponsors
  let dirs = fs.readdirSync(CONTENT_DIR)
  const items = []

  for(let dir of dirs) {
    // Get sponsor from Nuxt Content Development API
    const [ item ] = await fetch(CONTENT_API_BASE+dir).then(r => r.json())

    // Import asset to Directus in the "Sponsors" folder
    const { id: image } = await directus.files.import({
      url: `https://yougotthis.io${item.dir}/${item.file}`,
      data: { title: item.title, folder }
    })

    // Push all frontmatter, Directus asset id, and slug to items array
    items.push({ ...item, image, id: dir })
  }

  // Form payload in correct structure
  const payload = items.map(item => {
    return {
      slug: item.id,
      title: item.title,
      file: item.image,
      url: item.url
    }
  })

  // Create all sponsors items in Directus
  const { data } = await directus.items('sponsors').createMany(payload)
})();

Here's some notable parts of the script:

The first couple of lines get the asset folder id, which is needed later to upload assets to the right folder.
Then each directory inside of content/sponsors is returned in an array.
For each sponsor directory, the data is fetched using the Nuxt Content Development API. This is only available when running the Nuxt dev server.
With this additional information, the main sponsor image is imported from the live web URL, in the correct folder.
The array of items is then formed into the correct payload for Directus, and bulk-created using the SDK items().createMany() method.

This exercise was repeated for each content type in the Nuxt project. Once completed, there was one collection in Directus for each content subdirectory. The only addition was a featured singleton collection to control what collections and sponsors were shown on the home and library pages.

Consuming Directus From A Nuxt Application

Now data exists within Directus, Nuxt Content could be replaced within the Nuxt project. Firstly, I installed the Directus JavaScript SDK:

npm install @directus/sdk

Following the same steps as our guide "Build a Website With Nuxt 3 and the Directus JavaScript SDK", I created a plugin. However, being Nuxt 2, this looks slightly different:

import { Directus } from '@directus/sdk';
const DIRECTUS_URL = 'my-directus-url'
const directus = new Directus(DIRECTUS_URL);

export default({ app }, inject) => {
   inject('directus', directus)
}

Once added to your plugins array in nuxt.config.js, this.$directus is available throughout the application, and allows it to be used inside of asyncData to fetch data at build-time. For example, the library page's asyncData goes from this:

async asyncData({ $content }) {
  // Get each type of content
  let content = await $content('library').sortBy('date', 'desc').fetch()
  const collections = await $content('collections').sortBy('highlight', 'desc').sortBy('date', 'desc').limit(4).fetch()
  const people = await $content('people').fetch()

  // Add people to content items, as there are no relationships
  content = content.map(item => {
      let people = item.people.map(name => people.find(person => person.dir.split('/')[2] === name))
      people = profiles.map(profile => ({ ...profile, avatar: `${profile.dir}/${profile.avatar}` }))
      return { ...item, people }
  })

  return { content, collections }
}

To this:

async asyncData({ $directus }) {
  const { data: content } = await $directus.items('library').readByQuery({ limit: -1, sort: '-date', fields: ['*', '*.*'] })
  let { data: { collections } } = await $directus.items('featured').readByQuery({fields: ['*', '*.*', '*.*.*']})
  return { content, collections }
}

The main difference is that relationships are real, as opposed to something that needs to be hacked together after fetching data.

Working With Assets

Images being stored by Directus and accessed via a URL also means lots of hacky work can be removed from the site, like this line from my head-factory.js utility - which generated meta tags for many platforms by feeding in one object:

const image = meta.image ? meta.absolute ? meta.image : `${config.baseURL}${meta.imageDir || meta.path}/${meta.image}` : `${config.baseURL}${config.image}`

Honestly, I hate myself for that one. No idea what it meant, but it's gone now and replaced with a direct URL or a fallback:

const image = meta.image ? meta.image : `${config.baseURL}${config.image}`

Needing to constantly provide full asset URLs throughout the site (directus-url/assets/asset-id) was getting repetitive, so I decided to create a helper and inject it with my plugin:

export default({ app }, inject) => {
   inject('directus', directus)
   const asset = (id) => `${DIRECTUS_URL}/assets/${id}` // [!code ++]
   inject('asset', asset) // [!code ++]
}

Now, anywhere the full URL is needed, only the ID needs to be provided and it is automatically-expanded:

<img :src="$asset(person.image)">

The $asset helper is available globally, for free. No need to import it. If you want to alter the size of the image or use other supported media transformations, string interpolation can be used:

<img :src="`${$asset(person.image)}?width=50`">

In Summary

This was a super fun project that marked a point of maturity in my personal project where a robust CMS was required for others to be successful in their work. The broad process was:

Set up Directus data model.
Write and run migration scripts.
Replace $content with $directus throughout.

I've got to remove a load of hacky code and feel more confident in this project going forward. In the future, I may allow speakers to access their own profiles (through the Directus Data Studio, or through a custom-built frontend), and build a better authoring workflow that supports non-published states (like 'draft' and 'archived').

If you are considering moving from a file-based CMS to a headless API-based CMS, Directus is a great choice. We're always happy to answer questions over in our Discord server.

Build a Presentation Coaching Application with Recall.ai

Kevin Lewis — Mon, 24 Oct 2022 11:37:09 +0000

Recall provides a developer API to get real-time meeting data from a number of different platforms. It does this by sending Recall bots into meetings to observe what is happening and then provide data on-demand or in real-time. As well as transcripts, they provide metadata including a participant list and linked calendar invite.

In this tutorial, you will build a virtual presentation coaching application. The application will allow you to send a Recall bot into a Zoom call, remove them, and get insights once the call is over. One of the great things about Recall is their support of other platforms like Google Meet, Microsoft Teams, and WebEx with no additional code.

For this project, we'll complete the following steps:

Add a bot to a Zoom call
Get data about speakers in the call
Calculate speaker turn counts (to see if you took up more turns than others)
Create a speaker-separated transcript
Calculate talk-time per speaker

Before You Start

Make sure you have Node.js installed. You will need a Deepgram API Key and a Recall API Key.

Create a new directory for this project and open it in a code editor. Create a .env file and populate it with your keys:

RECALL_API_KEY=your-key-here
DEEPGRAM_API_KEY=your-key-here

Create a package.json file with npm init -y and then install our dependencies:

npm install dotenv express hbs axios

Create an index.js file and open it in your code editor.

Set Up Application

Import your dependencies:

import 'dotenv/config'
import axios from 'axios'
import express from 'express'

Set up your express application:

const app = express()
app.set('view engine', 'hbs')
app.use(express.urlencoded({ extended: false }))

// Further code goes here

const PORT = process.env.PORT || 3000
app.listen(PORT, () => console.log(`Listening on port ${PORT}`))

Create a route handler to load the initial page. Firstly, create a views directory and an index.hbs file inside of it. .hbs files use Handlebars to add conditional and looping logic to HTML files. In the new view file, add:

<h1>Call Coacher</h1>

Inside of index.js, render the view:

app.get('/', (req, res) => res.render('index'))

Start your server with node index.js, visit localhost:3000, and you should see Call Coacher.

Create a Recall.ai Helper Function

Recall's API Reference shows all of the available endpoints to manage bots - your application will use four of them. To make your code more readable, create a reusable recall() helper method at the very bottom of your index.js file:

async function recall(method, path, data) {
  try {
    const payload = {
      method,
      url: `https://api.recall.ai/api/v1${path}`,
      headers: {
          Authorization: `Token ${process.env.RECALL_API_KEY}`
      }
    }
    if(data) payload.data = data
    const response = await axios(payload)
    return response.data
  } catch(error) {
    throw error
  }
}

Now, for example, endpoints can be accessed like so:

const bots = await recall('get', '/bot')
const newBot = await recall('post', '/bot', { meeting_url: '...' })

Use Recall.ai To Add a Bot to a Zoom Call

Add a new form to views/index.hbs:

<h2>Add a bot to a call</h2>
<form action="/join" method="post">
    <label for="meeting_url">Meeting URL</label>
    <input type="text" id="meeting_url" name="meeting_url"><br>

    <label for="bot_name">Bot Name</label>
    <input type="text" id="bot_name" name="bot_name">

    <input type="submit" value="join">
</form>

Providing a bot name is optional, but your application will allow users to specify it. When submitted, this form will send a POST request to /join. Its payload will contain meeting_url and bot_name.

Add the following to index.js underneath the existing route handler for the homepage:

let bots = []
app.post('/join', async (req, res) => {
    try {
        const { meeting_url, bot_name } = req.body
        // Adds bot to call, returned data does not include meeting_url
        const bot = await recall('post', '/bot', { meeting_url, bot_name })
        // Add new bot to bots array
        bots.push({ ...bot, meeting_url })
        // Re-render the homepage, making a message available to the template
        res.render('index', { message: 'The bot has joined your call' })
    } catch(error) {
        console.log(error)
        res.render('index', { message: 'There has been a problem adding the bot' })
    }
})

Being able to send dynamic data into templates is a feature available by including handlebars in our application. At the bottom of index.hbs show the message:

<p>{{ message }}</p>

The message is empty (leaving an empty paragraph) when initially loading the page and will show the message after submitting the form.

Try it out! Restart your server, create a new Zoom call, get the meeting invite URL and submit it in the form. You should have a bot immediately join you with the bot name you specified.

Make a Recall.ai Bot Leave a Zoom Call

Currently, the only way to make the bot leave the call is to end it for everyone (or manually remove it in the Zoom interface). Recall also provide an endpoint to remove a bot. Add a new form below the previous one in index.hbs:

<h2>Leave call</h2>
<form action="/leave" method="post">
    <label for="meeting_url">Meeting URL</label>
    <input type="text" id="meeting_url" name="meeting_url">
    <input type="submit" value="leave">
</form>

In index.js create a new route handler:

app.post('/leave', async (req, res) => {
  try {
    const { meeting_url } = req.body
    // Get the bot from the bots array with matching meeting_url
    const { id } = bots.find(bot => bot.meeting_url == meeting_url)
    // Remove bot form call
    await recall('post', `/bot/${id}/leave_call`)
    // Redirect to /:botId
    res.redirect(`/${id}`)
  } catch(error) {
    console.log(error)
    res.render('index', { message: 'There has been a problem removing the bot' })
  }
})

Restart your server and try to add and remove a bot. The bot should leave the call when the new form is submitted, and you should be redirected to a new page (causing an error because it does not yet exist.)

Show Data From Call

Create a new data.hbs file in the views directory:

<h1>Data for {{ id }}</h1>
{{#if video_url}}
  <a href="{{video_url}}">Watch video until {{ media_retention_end }}</a>
{{/if}}

In index.js add a new route handler:

app.get('/:botId', async (req, res) => {
  try {
    // Get bot data
    const bot = await recall('get', `/bot/${req.params.botId}`)
    // Get transcript (each object is one speaker turn)
    const turns = await recall('get', `/bot/${req.params.botId}/transcript`)

    // Further code here

    // Return all properties in bot object
    res.render('data', bot)
  } catch(error) {
    res.send('There has been a problem loading this bot data')
  }
})

Restart your server, start a new Zoom call (preferably with someone else), speak for a couple of minutes, remove the bot with the form, and you should be redirected to a page.

Get All Speaker Usernames

A full timeline for the call including who spoke and when is made available as part of the bot object. Extract just usernames and de-duplicate the list by adding the following:

const { timeline } = bot.speaker_timeline
let usernames = [...new Set(timeline.map(turn => { username: turn.users[0].username }))]

Update the res.render() method to the following:

res.render('data', { ...bot, usernames })

Finally, add a list of who spoke to the bottom of data.hbs:

<h2>Who spoke:</h2>
<ul>
  {{#each usernames}}
    <li>
      <span>{{ this.username }}</span>
    </li>
  {{/each}}
</ul>

Show Each Speaker's Turn Count

Below where usernames is defined, add the following:

for(let i=0; i<usernames.length; i++) {
  let userTurns = timeline.filter(turn => turn.users[0].username == usernames[i])
  usernames[i] = {
    username: usernames[i],
    turns: userTurns.length
  }
}

Now each username in the usernames array also has a turns property, which is equal to the number of times they spoke in the call. Update the loop to show the new data:

{{#each usernames}}
  <li>
    <span>{{ this.username }}</span>
    <span>{{ this.turns }} turns speaking</span>
  </li>
{{/each}}

Display Call Transcript with Usernames

Recall is a Deepgram customer and provides our accurate AI-powered transcription within their product. The transcript is already available in our application in the turns variable. Add the following below the for loop in index.js:

let transcript = []
for(let i=0; i<turns.length; i++) {
  // Get all words for this turn
  const turnWords = turns[i].words
  // Form a single stream of words
  const words = turnWords.map(w => w.text).join(' ')
  // Add to transcript array along with speaker username
  transcript.push({ speaker: turns[i].speaker, words })
}

Add the transcript to the rendered data:

res.render('data', { ...bot, usernames, transcript })

Finally, in data.hbs, add the following to the bottom:

<h2>Transcript</h2>
{{#each transcript}}
  <p><b>{{ this.speaker }}: </b>{{ this.words }}</p>
{{/each}}

Calculate Each Speaker's Speaking Time

Each word in the transcript is accompanied by a word's start and end time. Using this data, each speaker's 'talking time' can be calculated. Firstly, turns is added to usernames[i], add a new speakTime value:

usernames[i] = {
  username: usernames[i],
  turns: userTurns.length,
  speakTime: 0
}

Calculate the speakTime just after you add transcripts with transcripts.push(), and add it to the speaker's entry in the username array:

const speakTime = +(turnWords[turnWords.length-1].end_timestamp - turnWords[0].start_timestamp).toFixed(2)
const user = usernames.findIndex(u => u.username == turns[i].speaker)
usernames[user].speakTime += speakTime

Finally, update data.hbs to contain this new data just below where each speaker's turns are shown:

<span>{{ this.speakTime }}s total talking time</span>

The World Is Your Oyster

This application only scratches the surface of the analysis you can perform with data returned by Recall and Deepgram. You may choose to detect non-inclusive language, summarize what has been said, and more. Recall provides a developer-friendly way to avoid writing 'glue' into various conferencing platforms, so if you want to use Google Meet, Microsoft Teams, WebEx, or others, there is no more code to write. Fab!

If you have any questions, please don't hesitate to get in touch. We love to help!

Automatically Transcribe, Summarize, and Send Phone Call Summaries

Kevin Lewis — Mon, 24 Oct 2022 11:28:06 +0000

In this tutorial, we'll use Twilio Functions and Deepgram's summarize feature to send call summaries via SMS once a conversation has ended.

Users can call a Twilio phone number, which will be forwarded to your agent while recording. Once the call is completed, we will get a transcript and summary of the call using Deepgram. Finally, the summary will be sent to both the caller and the agent via SMS.

We've previously written a post on transcribing phone calls with Twilio Functions. We'll run through all the steps here, but do take a look for more detail.

Before You Start

You will need a Deepgram API Key - get one here. You will also need a Twilio account and a phone number in your account with SMS and Voice capabilities. Finally, you'll need two phones to test your project - one to make the call and one to receive.

Set Up Twilio Functions

Inside the Twilio Console, head to Developer Tools -> Functions & Assets and create a new service. A service can contain multiple Twilio Functions and assets related to a single project. It’s important that you create a new service here and not a standalone function.

In the Dependencies section, add @deepgram/sdk (you can omit the version for the latest).

In the Environment Variables section, add a key called DEEPGRAM_KEY with the value of your API Key generated in your Deepgram console. Create a second key called FORWARDING_NUMBER with the value of your agent phone number with E.164 formatting like this: +14155552671.

Record and Forward Inbound Calls

Rename the /welcome function to /inbound. Replace the whole file with the following:

exports.handler = function(context, event, callback) {
  let twiml = new Twilio.twiml.VoiceResponse()
  const dial = twiml.dial({
    record: 'record-from-answer-dual',
    recordingStatusCallback: '/recordings'
  })
  dial.number(process.env.FORWARDING_NUMBER)
  return callback(null, twiml)
}

When this function receives incoming call data, it will forward it to your agent while recording it. Once completed, call data will be sent to /recordings (which will be created in the next section).

Save the function, and click Deploy All. Once deployed, this function is ready to be used. Go to your Twilio number settings, and when a call comes in, select Function. Select your service and the /inbound function path.

Transcribe and Summarize Call

Create a new function - /transcribe. Delete the boilerplate and set up the function with the following code:

const { Deepgram } = require('@deepgram/sdk')
const deepgram = new Deepgram(process.env.DEEPGRAM_KEY)

exports.handler = async function(context, event, callback) {
  const { RecordingUrl, CallSid } = event
  const twilioClient = context.getTwilioClient()
  const { from: caller, to: twilioNumber } = await twilioClient.calls(CallSid).fetch()

  // Further code here

  return callback(null, true)
}

This code takes the CallSid and looks up the call for further call information. The caller’s phone number is now available in a variable called caller, and the number they called as twilioNumber. Now generate a transcription with Deepgram’s Node.js SDK:

const transcriptionFeatures = { punctuate: true, tier: 'enhanced', summarize: true }
const { results } = await deepgram.transcription.preRecorded({ url: RecordingUrl }, transcriptionFeatures)
const { summaries } = results.channels[0].alternatives[0]

summaries is an array of objects containing summary text and what time period it is summarizing. Add the following to turn it into one string that can be sent via SMS:

const summary = summaries.map(s => s.summary).join('\n\n')

Send Summary Messages

Now that you have a summary of the call provided by Deepgram's summarization feature, it's time to send it to both the caller and the agent. Just below summary add the following:

for(let number of [process.env.FORWARD_NUMBER, caller]) {
  await twilioClient.messages.create({
    body: summary,
    to: number,
    from: twilioNumber
  })
}

Save both files again and deploy all functions in your service. Call your Twilio number, pick it up on your 'agent device', speak, and hang up. You should receive a summary message a few seconds later.

In Summary

Getting high-quality summaries from your audio is as simple as adding one parameter in your Deepgram request. They're super useful in understanding what was said and what needs to happen next.

If you have questions about anything in this post, we’d love to hear from you. Head over to our forum and create a new discussion with your questions, or send us a tweet @DeepgramAI

The final code is as follows:

// /inbound
exports.handler = function(context, event, callback) {
  let twiml = new Twilio.twiml.VoiceResponse()
  const dial = twiml.dial({
    record: 'record-from-answer-dual',
    recordingStatusCallback: '/transcribe'
  })
  dial.number(process.env.FORWARDING_NUMBER)
  return callback(null, twiml)
}

// /transcribe
const { Deepgram } = require('@deepgram/sdk')
const deepgram = new Deepgram(process.env.DEEPGRAM_KEY, 'api.beta.deepgram.com')

exports.handler = async function(context, event, callback) {
  const { RecordingUrl, CallSid } = event
  const twilioClient = context.getTwilioClient()
  const { from: caller, to: twilioNumber } = await twilioClient.calls(CallSid).fetch()

  const transcriptionFeatures = { punctuate: true, tier: 'enhanced', summarize: true }
  const { results } = await deepgram.transcription.preRecorded({ url: RecordingUrl }, transcriptionFeatures)
  const { summaries } = results.channels[0].alternatives[0]
  const summary = summaries.map(s => s.summary).join('\n\n')

  for(let number of [process.env.FORWARDING_NUMBER, caller]) {
    await twilioClient.messages.create({
      body: summary,
      to: number,
      from: twilioNumber
    })
  }

  return callback(null, true)
}

Improve IVR Prompts with Custom Reporting

Kevin Lewis — Thu, 22 Sep 2022 16:49:25 +0000

One of the biggest challenges for conversational AI is anticipating all the ways in which a user may express a single phrase. Even with decent natural language processing, users often have frustrating experiences with 'retry rates' - the number of times a request is rejected before it succeeds, even more so in Interactive Voice Response (IVR) systems. However, data around failed attempts can be key in improving understanding of how people frame their requests.

In this project, we'll cover an approach to gather failed failed IVR scripts and infer their meaning based on the successful attempt. This data can ultimately be used to improve your intent triggers and improve customer experience.

We'll be using Deepgram with JavaScript and browser live transcription to demonstrate the concept, but it can easily be applied in other settings and programming languages.

Before we start, you will need a Deepgram API Key - get one here and keep it handy.

Create an empty index.html file and open it in your code editor.

Set Up Live Transcription

Add the following code to index.html to set up live transcription in your browser. For a detailed explanation of how this works, check out our blog post on browser live transcription.

<html>
    <body>
        <script>
            navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => {
                const mediaRecorder = new MediaRecorder(stream)

                const DG_KEY = 'replace-with-your-deepgram-api-key'
                const socket = new WebSocket('wss://api.deepgram.com/v1/listen', [ 'token', DG_KEY ])

                socket.onopen = () => {
                    mediaRecorder.addEventListener('dataavailable', event => {
                        if (event.data.size > 0 && socket.readyState == 1) {
                            socket.send(event.data)
                        }
                    })
                    mediaRecorder.start(250)
                }

                socket.onmessage = (message) => {
                    const { transcript } = JSON.parse(message.data).channel.alternatives[0]
                    if(transcript) handleResponse(transcript)
                }
            })

            function handleResponse(transcript) {
                console.log(transcript)
            }
        </script>
    </body>
</html>

Open the file in your browser. You should immediately be prompted for access to your microphone. Once granted, open up your browser console and start speaking to see your words logged.

Set Up Intents

In reality, your conversational AI system will be a lot more complex and robust than what we'll build today, but they mostly have the same characteristics:

A list of request options - 'intents'
Each option has a number of phrases or terms that can be used to trigger it - 'triggers'
An action to happen when an intent occurs - 'response'

Intents normally inform a machine learning model which will match phrases similar but not identical, and responses may execute some logic before continuing. For this project, we'll need a partial match on an intent trigger. The response will be speaking a fixed phrase back to the user.

At the top of your <script> tag, add the following intents:

const intents = [
    {
        intent: 'balance',
        triggers: [
            'balance',
            'balance',
            'how much money'
        ],
        response: 'Your bank balance is over nine thousand'
    },
    {
        intent: 'new_transaction',
        triggers: [
            'transfer',
            'send',
            'set up payee'
        ],
        response: 'Who would you like to send money to?'
    },
    {
        intent: 'help',
        triggers: [
            'help',
            'advice',
            'struggling'
        ],
        response: 'Putting you through to one of our agents now'
    },
]

Match User Speech to Intents

When a user speaks, we need to determine if there was a match or not. Update handleResponse() with the following:

function handleResponse(transcript) {
    const match = intents.find(intent => intent.triggers.some(trigger => transcript.includes(trigger)))
    console.log(match)
}

match will either be the entire intent object for the matching item or undefined.

Save Intent Matching

Just above handleResponse(), create two new variables - current that will contain the current string of requests towards a single intent and report that will contain all failed intents and the final successful phrase.

let current = {}
const report = []

Update handleResponse() with logic if there was no match. Specifically, add the phrase to current.retries, creating it if it doesn't already exist:

function handleResponse(transcript) {
    const match = intents.find(intent => intent.triggers.some(trigger => transcript.includes(trigger)))

    if(!match) {
        console.log(`No match for ${transcript}`)
        if(!current.retries) current.retries = [transcript]
        else current.retries.push(transcript)
    }
}

If there was a match, add it to the current object, and push it into the report array. Each object in report will contain failed attempts and the eventual successful trigger:

if(!match) {
    console.log(`No match for ${transcript}`)
    if(!current.retries) current.retries = [transcript]
    else current.retries.push(transcript)
} else {
    if(current.retries) {
        current.intent = match.intent
        report.push(current)
    }
    current = {}
    console.log(match.response)
}

Try it out. Refresh the browser and start speaking. Try some random phrases, and then one which will trigger a match - "I need help", "What's my overdraft balance?", and "send some money" should all work.

Prompt the User to Speak

To wrap up, let's add spoken prompts and replies for this application using the Web Speech API.

At the bottom of the <script> tag, create a speak() function:

const synth = window.speechSynthesis
function speak(text) {
    if (synth.speaking) synth.cancel()
    const utterThis = new SpeechSynthesisUtterance(text)
    synth.speak(utterThis)
}

Add an initial prompt to speak. Under mediaRecorder.start(250) add:

speak('What can we help you with today?')

At the bottom of the logic in the if statement, when there is no match, add a retry prompt:

speak('I didn\'t understand that, sorry. Can you try again?')

When there is a match, respond to the user:

speak(match.response)

At any point, the report variable contains an array of potential improvements you can make to your conversational AI intents.

In Practice

This tutorial shows an overall approach for inferring the meaning of failed intent triggers, assuming that a user does not change their intent. To build this system out further, you should consider the common change in intent from interfacing with a bot to "speak to a human."

You may also choose to do this after an interaction has ended rather than live, but you'll need to determine when a retry occurs.

If you have questions about anything in this post, we’d love to hear from you. Head over to our forum and create a new discussion with your questions, or send us a tweet @DeepgramAI

Finding All Mentions Of Your Brand From A Podcast

Kevin Lewis — Wed, 14 Sep 2022 09:54:05 +0000

In this post, we'll cover how to check podcast episodes for mentions of your brand. This can be particularly useful for ensuring sponsorship obligations are met, or to check when competitors are spoken about.

Given an input of a podcast feed URL, start/end dates, and a brand name, the script will generate a report of all mentions as detected by Deepgram's fast and accurate speech recognition API.

Before You Start

You must have Python installed on your machine - I'm using Python 3.10 at the time of writing. You will also need a Deepgram API Key - get one here.

Create a new directory and navigate to it in your terminal. Create a virtual environment with python3 -m venv virtual_env and activate it with source virtual_env/bin/activate. Install dependencies with pip install deepgram_sdk asyncio python-dotenv feedparser.

Open the directory in a code editor, and create an empty .env file. Take your Deepgram API Key, and add the following line to .env:

DEEPGRAM_API_KEY="replace-this-bit-with-your-key"

Dependency and File Setup

Create an empty script.py file and import the dependencies:

import os
import json
from datetime import datetime
from time import mktime
import asyncio
from dotenv import load_dotenv
from deepgram import Deepgram
import feedparser

Load values from the .env file and initialize the Deepgram Python SDK:

load_dotenv()
deepgram = Deepgram(os.getenv('DEEPGRAM_API_KEY'))

Finally, set up a main() function that is executed automatically when the script is run:

async def main():
    print('Hello world')

if __name__ == '__main__':
    asyncio.run(main())

Define Parameters

Above the main() function, create a set of variables with settings for your report:

podcast_feed = 'http://feeds.codenewbie.org/cnpodcast.xml' # CodeNewbie Podcast
brand_name = 'stellar'
required_confidence = 0.9
start_date = '2022-05-01' # Start of season 20
end_date = '2022-06-27' # End of season 20

Each time Deepgram returns a search result, it will come with a confidence between 0 and 1. The required_confidence value will only report results above the specified confidence level.

Fetch Episodes with Feedparser

Remove the print() statement in the main() function, fetch the podcast, and take a look at the returned data by pretty-printing it:

async def main():
    rss = feedparser.parse(podcast_feed)
    print(json.dumps(rss.entries, indent=2))

Try it out! In your terminal, run the file with python3 script.py and you should see a bunch of data for each episode.

Filter Episodes Within Date Range

Feedparser will take in many different date formats for when a RSS entry is published/updated and normalize them to a standard format. Using the standardized output, create a helper function just below the main() function:

def check_if_in_date_range(episode):
    date_with_time = datetime.fromtimestamp(mktime(episode.published_parsed))
    date = date_with_time.replace(hour=0, minute=0, second=0)
    is_not_before_start = date >= datetime.fromisoformat(start_date)
    is_not_after_end = date <= datetime.fromisoformat(end_date)
    return is_not_before_start and is_not_after_end

This function takes in an episode, gets the date (without time), and returns True if it is within the range between and including start_date and end_date.

Remove print(json.dumps(rss.entries, indent=2)) and replace it with the following:

episodes = list(filter(check_if_in_date_range, rss.entries))

The episodes array now contains only episodes within the date range.

Transcribe Episodes with Keyword Boosting and Search

Inside of the main() function, extract the podcast media URL, set transcription options, and request a Deepgram transcription. Finally, extract search results from the result:

for episode in episodes:
    # Get podcast episode URL
    source = { 'url': episode.enclosures[0].href }
    # Increase chance of hearing brand_name
    # Return search results for brand_name
    transcription_options = { 'keywords': f'{brand_name}:2', 'search': brand_name }
    # Request transcription
    response = await deepgram.transcription.prerecorded(source, transcription_options)
    # Extract search results
    search_results = response['results']['channels'][0]['search'][0]['hits']

Filter Only High Confidence Results

Add the following line below search_results to filter out any values which are below the required confidence:

strong_results = list(filter(lambda x: x['confidence'] > required_confidence, search_results))

Save Mentions Report

Below strong_results, take each episode (and each result within the episode), and add it as a new line in a report file:

# Define file name
filename = f'{brand_name}-mentions-{start_date}-to-{end_date}.txt'
with open(filename, 'a+') as f:
    # Format publish date
    pub = datetime.fromtimestamp(mktime(episode.published_parsed))
    # Create line per episode
    f.write(f'{pub}: "{episode.title}" ({len(strong_results)} mentions)\n')
    # Create line per result (indented two spaces)
    for result in strong_results:
        f.write(f'  Mention at {result["start"]}s of \"{result["snippet"]}\"\n')

That's it! Rerun the script with python3 script.py and, once completed, you should see a new file called stellar-mentions-2022-05-01-to-2022-06-27.txt - perfect if you want to run several reports.

Extending This Project

This project should equip you with the information you need to understand if and how often brand mentions occur throughout several podcast episodes. You should extend this further by creating more complex or graphical reports, allowing several brands to be searched for in one request, or by building a UI around this logic.

As ever, if you have any questions please feel free to get in touch or post in our community discussions.

The final code is as follows:

import asyncio
import os
from datetime import datetime
from time import mktime
from dotenv import load_dotenv
from deepgram import Deepgram
import feedparser

load_dotenv()
deepgram = Deepgram(os.getenv('DEEPGRAM_API_KEY'))

podcast_feed = 'http://feeds.codenewbie.org/cnpodcast.xml'
brand_name = 'stellar'
required_confidence = 0.9
start_date = '2022-05-01'
end_date = '2022-06-27'

async def main():
    rss = feedparser.parse(podcast_feed)
    episodes = list(filter(check_if_in_date_range, rss.entries))
    print(len(episodes))

    for episode in episodes:
        source = { 'url': episode.enclosures[0].href }
        transcription_options = { 'keywords': f'{brand_name}:2', 'search': brand_name }
        response = await deepgram.transcription.prerecorded(source, transcription_options)
        search_results = response['results']['channels'][0]['search'][0]['hits']
        strong_results = list(filter(lambda x: x['confidence'] > required_confidence, search_results))

        filename = f'{brand_name}-mentions-{start_date}-to-{end_date}.txt'
        with open(filename, 'a+') as f:
            pub = datetime.fromtimestamp(mktime(episode.published_parsed))
            f.write(f'{pub}: "{episode.title}" ({len(strong_results)} mentions)\n')
            for result in strong_results:
                f.write(f'  Mention at {result["start"]}s of \"{result["snippet"]}\"\n')

def check_if_in_date_range(episode):
    date_with_time = datetime.fromtimestamp(mktime(episode.published_parsed))
    date = date_with_time.replace(hour=0, minute=0, second=0)
    is_not_before_start = date >= datetime.fromisoformat(start_date)
    is_not_after_end = date <= datetime.fromisoformat(end_date)
    return is_not_before_start and is_not_after_end

if __name__ == '__main__':
    asyncio.run(main())

Automatically Transcribing Podcast Episodes with Pipedream and Python

Kevin Lewis — Mon, 12 Sep 2022 13:30:38 +0000

I love podcasts, but I rarely find the time to listen to all the new episodes of the ones I love. However, I often find time to read an article or newsletter so I thought it would be fun to automatically transcribe new episodes and send myself the output via email.

We'll be using Pipedream - an online workflow builder that integrates with many services while allowing us to write code in Python, JavaScript, Go, and more. Each workflow has one trigger that starts it and any number of actions that can happen as a result.

Before you start, you will need a Pipedream account, a Google account, and a free Deepgram API Key.

Create a new empty Pipedream workflow for this project.

Trigger Your Workflow When New Episodes Are Released

All Podcasts publish via a public RSS feed, which is updated whenever there is a new episode. Pipedream has a built-in trigger which will check every 15 minutes if a new item has been added.

Use the 'New Item in RSS Feed' trigger with any podcast RSS feed - you can use https://feeds.buzzsprout.com/1976304.rss, which is for the new Deepgram podcast Voice of the Future.

Click Create Source and select the first event. This is representative of one podcast episode.

Once selected, you can see all of the data contained in that RSS feed entry - including the episode's metadata and direct media link. All of this data can be used in future steps within the workflow.

Transcribe Podcast With Python

Create a new step (now an 'action') that will be run whenever the workflow is triggered. Pick Python -> Run Python Code. In the Configure section, click Add an App and select Deepgram. Insert your API Key and save it before continuing.

Delete all of the code except def handler(pd: "pipedream"):, which is the required function signature that will be executed when this step is reached in the workflow. Make sure you have indented your code underneath this line. Then, get the URL from the trigger and your Deepgram API Key from the configured app:

url = pd.steps["trigger"]["event"]["enclosures"][0]["url"]
token = pd.inputs["deepgram"]["$auth"]['api_key']

As mentioned above, Pipedream requires the def handler(pd: "pipedream"): signature for the main function in a Python step. Because of this, the asynchronous Deepgram Python SDK won't be usable in this context. Instead, we'll directly make an API request with the requests library.

At the very top of your code, add the following line:

import requests

Then, at the bottom of your handler function, prepare your Deepgram API request:

listen = "https://api.deepgram.com/v1/listen?tier=enhanced&punctuate=true&diarize=true&paragraphs=true"
headers = { "Authorization": f"Token {token}" }
json = { "url": url }

This request will use Deepgram's enhanced tier, diarization (speaker detection), and format the output using punctuation and paragraphs.

Now that you are set up, make the request, extract the formatted response, and return the value:

r = requests.post(listen, json=json, headers=headers)
response = r.json()
transcript = response['results']['channels'][0]['alternatives'][0]['paragraphs']['transcript']
return transcript

Final code:

import requests

def handler(pd: "pipedream"):
  url = pd.steps["trigger"]["event"]["enclosures"][0]["url"]
  token = pd.inputs["deepgram"]["$auth"]['api_key']

  listen = "https://api.deepgram.com/v1/listen?tier=enhanced&punctuate=true&diarize=true&paragraphs=true"
  headers = { "Authorization": f"Token {token}" }
  json = { "url": url }

  r = requests.post(listen, json=json, headers=headers)
  response = r.json()

  transcript = response['results']['channels'][0]['alternatives'][0]['paragraphs']['transcript']
  return transcript

Click Test and the URL from the trigger will be sent to Deepgram, and the returned value will be shown in Pipedream:

Send Email With Transcript

Now a transcript has been automatically generated, you can do anything with it - either through Pipedream's integrations or by adding another Python step. As mentioned at the start of this post, the outcome of this project is to send yourself an email with the content of the podcast. You can also include variables in the subject line.

Add a Send Email To Self step, and set the subject line to:

New episode of {{steps.trigger.event.meta.title}}: {{steps.trigger.event.title}}

Set the text to:

Episode description: {{steps.trigger.event.description}}\n\n{{steps.python.$return_value}}

It should look like the following:

Test the step, and you should receive an email in just a few seconds. Deploy the workflow, and enjoy reading new podcast episodes. If you have any questions about this project or anything else to do with Deepgram, please get in touch!

Detect Non-Inclusive Language with Retext and Node.js

Kevin Lewis — Fri, 02 Sep 2022 11:54:30 +0000

Personal language usage is a journey of learning and adapting, which certainly extends to terms you may not yet realize are non-inclusive or potentially profane to others. By detecting and pointing out some potentially-problematic language, you can work towards being more considerate and kind to others.

alex is a lovely command-line tool that takes in text or markdown files and, using retext-equality and retext-profanities, highlights suggestions for improvement. alex checks for gendered work titles, gendered proverbs, ableist language, condescending or intolerant language, profanities, and much more.

In this short tutorial, we'll cover how to use the retext libraries alex depends on to check your Deepgram transcript utterances for suggestions.

Before You Start

Before we start, you will need a Deepgram API Key - get one here.

Create a new directory and open it in your code editor, navigate to it in your terminal, create a new package.json file by running npm init -y, and install dependencies:

npm install retext retext-equality retext-profanities vfile-reporter-json @deepgram/sdk

The retext packages require an ES6 project. The easiest way to do this without needing to compile code with a tool like Babel is to add the following property to your package.json file:

"type": "module"

Create and open an index.js file in your code editor.

Generating a Transcript With Deepgram

The Deepgram Node.js is a CommonJS module, but can be imported via the default export. Because of this, our import will go from this in CommonJS:

const { Deepgram } = require('@deepgram/sdk')

To this in ES6 (DG can be anything as long as it's the same in both uses):

import DG from '@deepgram/sdk'
const { Deepgram } = DG

Then, generate a transcript. Here I am using a recording of my voice reading out the alex sample phrase for demonstration.

const deepgram = new Deepgram('YOUR_DEEPGRAM_API_KEY')
const url = 'http://lws.io/static/inconsiderate.mp3'
const { results } = await deepgram.transcription.preRecorded({ url }, { utterances: true })
console.log(results)

As the utterances feature is being used, an array will be provided with each utterance (spoken phrase) along with when it was spoken.

Test it out! Run the file with node index.js, and you should see a payload in your terminal. Once you know it works, remove the console.log().

Setting Up the Language Checker

At the very top of index.js, include the dependencies required to set up retext and then report issues found from it:

import { reporterJson } from 'vfile-reporter-json'
import { retext } from 'retext'
import retextProfanities from 'retext-profanities'
import retextEquality from 'retext-equality'

Then, create the following reusable function:

async function checkText(text) {
    const file = await retext()
        .use(retextProfanities)
        .use(retextEquality)
        .process(text)
    const outcome = JSON.parse(reporterJson(file))
    const warnings = outcome[0].messages.map(r => r.reason)
    return warnings
}

This function processes the provided text through the specified plugins (here, retextProfanities and retextEquality). The outcome is actually quite a large amount of data:

{
    reason: '`man` may be insensitive, use `people`, `persons`, `folks` instead',
    line: 1,
    column: 9,
    position: {
        start: { line: 1, column: 9, offset: 8 },
        end: { line: 1, column: 12, offset: 11 }
    },
    ruleId: 'gals-man',
    source: 'retext-equality',
    fatal: false,
    stack: null
},

The warnings map in the reusable checkText function extracts only the reason and returns an array of these strings. Try it out by temporarily adding this line:

const testSuggestions = await checkText('He is a butthead.')
console.log(testSuggestions)

The result should be:

[
    'Don’t use `butthead`, it’s profane',
    '`He` may be insensitive, use `They`, `It` instead'
]

Once you know it works, remove the console.log().

Checking Each Utterance's Language

Add the following to your index.js file below where you generate Deepgram transcripts:

let suggestions = []

for(let utterance of results.utterances) {
    const { transcript, start } = utterance

    // Get array of warning strings
    let warnings = await checkText(transcript)

    // Alter strings to be objects including the utterance transcript and start time
    warnings = warnings.map(warning => ({ warning, transcript, start }))

    // Append to end of array
    suggestions = [...suggestions, ...warnings]
}

console.log(suggestions)

Your terminal should show all of the suggestions presented by the two retext plugins:

Wrapping Up

This full snippet (below) is a great place to start identifying and changing usage and non-inclusive language patterns. You may quickly realize that the retext plugins lack nuance and sometimes make suggestions on false-positive matches. Don't consider the suggestions as "must-dos", but rather points for consideration and thought.

There's a whole host of other retext plugins which you can process text with, including those that handle assumptions, cliches, passive voice, repetition, overly-complex words, and more. Enjoy!

import { reporterJson } from 'vfile-reporter-json'
import { retext } from 'retext'
import retextProfanities from 'retext-profanities'
import retextEquality from 'retext-equality'
import DG from '@deepgram/sdk'
const { Deepgram } = DG
const deepgram = new Deepgram(process.env.DG_KEY)

const url = 'http://lws.io/static/inconsiderate.mp3'
const { results } = await deepgram.transcription.preRecorded({ url }, { utterances: true })

async function checkText(text) {
    const file = await retext()
        .use(retextProfanities)
        .use(retextEquality)
        .process(text)
    const outcome = JSON.parse(reporterJson(file))
    const warnings = outcome[0].messages.map(r => r.reason)
    return warnings
}

let suggestions = []
for(let utterance of results.utterances) {
    const { transcript, start } = utterance
    let warnings = await checkText(transcript)
    warnings = warnings.map(warning => ({ warning, transcript, start }))
    suggestions = [...suggestions, ...warnings]
}

console.log(suggestions)