DEV Community: Ivan Porollo

Custom Emails with Supertokens, Resend, and React Email

Ivan Porollo — Sun, 23 Jun 2024 21:05:59 +0000

At Cerebral Valley we use Supertokens for authentication into our platform.

Supertokens comes with a default email template / design that is sent to users upon account creation, email confirmation, and other actions.

We wanted to customize emails sent out from Supertokens to our users to keep the brand aesthetic, so we used Resend and React Email to do so.

I wrote up this post to give step by step instructions of how to combine the three technologies and showcase their simplicity.

Pre-requisites

Your project will need to be using Supertokens as the method of authentication.

In this walkthrough, I am working out of a Typescript project with an Express backend.

Resend setup

First, you will need to create a Resend account. Resend is a new platform to send emails programmatically. Think Sendgrid but modern and easier to use.

Sign up and go through the onboarding flow to get an API key. By default, the first key that is created is called "Onboarding"

Add your API key to your .env file, something like

RESEND_API_KEY="<your_api_key>"

Then, add the domain you want to send your emails from.

You will have to add a few records to your domain which Resend will walk you through.

Next, create a file in your project called smpt.ts with the following contents

const smtpSettings = {
  host: 'smtp.resend.com',
  authUsername: 'resend',
  password: process.env.RESEND_API_KEY,
  port: 465,
  from: {
    name: '<your_email_sender_name>',
    email: '<your_email_account>',
  },
  secure: true,
};

export { smtpSettings };

React Email setup

React Email is a library built by the founder of Resend. Since emails support HTML, the library allows you to customize emails with React components and then compiles them down to HTML before sending.

To install React Email, run

yarn add react @react-email/components @react-email/render

Create your a file named Email.tsx for your React Email components. The file contents would be something like this

import * as React from 'react';
import { render } from '@react-email/render';
import {
  Body,
  Button,
  Container,
  Head,
  Heading,
  Hr,
  Html,
  Img,
  Link,
  Preview,
  Section,
} from '@react-email/components';

const emailSubject = 'Email Subject';

const EmailHtml = (content: string): string => {
  return render(<Email content={content} />);
};

// Component

interface EmailProps {
  content: string;
}

function Email(props: EmailProps): JSX.Element {
  const { content } = props;
  return (
    <Html>
      <Head />
      <Preview>{'What the user sees in the preview'}</Preview>
      <Body style={main}>
        <Container style={container}>
          <Img
            src={`https://yourlogo.com/logo.png`}
            width="42"
            height="42"
            alt="Logo"
            style={logo}
          />
          <Heading style={heading}>Click the button below</Heading>
          <Section style={buttonContainer}>
            <Button style={button} href={content}>
              My button
            </Button>
          </Section>
        </Container>
      </Body>
    </Html>
  );
}

// Styling

const logo = {
  borderRadius: 21,
  width: 42,
  height: 42,
};

const main = {
  backgroundColor: '#090909',
  fontFamily:
    '-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif',
  // ui-sans-serif, system-ui, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"
};

const container = {
  margin: '0 auto',
  padding: '20px 0 48px',
  maxWidth: '560px',
};

const heading = {
  fontSize: '24px',
  letterSpacing: '-0.5px',
  lineHeight: '1.3',
  fontWeight: '400',
  color: '#fff',
  padding: '17px 0 0',
};

const paragraph = {
  margin: '0 0 15px',
  fontSize: '15px',
  lineHeight: '1.4',
  color: '#3c4149',
};

const buttonContainer = {
  padding: '27px 0 27px',
};

const button = {
  backgroundColor: '#fff',
  borderRadius: '3px',
  fontWeight: '600',
  color: '#000',
  fontSize: '15px',
  textDecoration: 'none',
  textAlign: 'center' as const,
  display: 'block',
  padding: '11px 23px',
};

const reportLink = {
  fontSize: '14px',
  color: '#b4becc',
};

const hr = {
  borderColor: '#dfe1e4',
  margin: '42px 0 26px',
};

export { EmailHtml, emailSubject };

Note that the above code is for a showing a link in the email. Your content may be different.

Supertokens Config

Supertokens allows you to use your own domain / SMTP server (link to docs). We take advantage of this method to plug in Resend and the created React component.

In your Supertokens config, override the smtp settings as described in their docs here.

Don't forget to import your smtpSettings and Email component.

Your Supertokens config would look something like this

import supertokens from "supertokens-node";
import Passwordless from "supertokens-node/recipe/passwordless";
import Session from "supertokens-node/recipe/session";
import { SMTPService } from "supertokens-node/recipe/passwordless/emaildelivery";
import EmailVerification from "supertokens-node/recipe/emailverification"
import { SMTPService as EmailVerificationSMTPService } from "supertokens-node/recipe/emailverification/emaildelivery";
import { smtpSettings } from './smtp';
import { EmailHtml, emailSubject } from './Email';

supertokens.init({
    appInfo: {
        apiDomain: "...",
        appName: "...",
        websiteDomain: "..."
    },
    recipeList: [
       Passwordless.init({
         emailDelivery: {
          service: new SMTPService({
            smtpSettings,
            override: (originalImplementation): any => {
              return {
                ...originalImplementation,
                getContent: async function (input): Promise<any> {
                  const {
                    isFirstFactor,
                    codeLifetime, // amount of time the code is alive for (in MS)
                    email,
                    urlWithLinkCode, // magic link
                  } = input;

                  if (isFirstFactor) {
                    return {
                      body: EmailHtml(urlWithLinkCode),
                      isHtml: true,
                      subject: emailSubject,
                      toEmail: email,
                    };
                  } else {
                    return {
                      body: EmailHtml(urlWithLinkCode),
                      isHtml: true,
                      subject: emailSubject,
                      toEmail: email,
                    };
                  }
                },
              };
            },
          }),
        },
      }),

      Session.init()
    ]
});

In the config above, we override the email delivery service with our own definition of the SMPTService, where we set the previously defined smptSettings and a new getContent function definition. In the new getContent, we get the defined React Component as the body of the email. The component is compiled into HTML, so we can use it in the email.

With those changes, you can start the server and run through the authentication flow. When you receive the email from Supertokens, you should see your new component design as the body of the email.

Wrapping up

If you're using Supertokens for your authentication, I highly recommend using Resend and React Email to customize your emails. It's very simple to setup and a pleasant developer experience.

If you have any questions, send me a DM in our community Slack.

Check out what we're building at Cerebral Valley.

Reference Data Stack for Data-Driven Startups

Ivan Porollo — Thu, 03 Mar 2022 21:15:20 +0000

At Monosi, we recently set up our internal data stack to fully enable data collection and metrics tracking. I figured a post like this may be useful for earlier stage companies that are in the process of setting up their own pipelines as they start tracking metrics and becoming more data driven.

This post is a high level overview of the technologies we are using and why we chose them. If you’re interested in a more technical overview or tutorial with actual Terraform files to spin up a replica stack, comment below and I can write up a more detailed post.

Technologies Used

Monosi is an open source data observability platform, therefore our stack is composed mostly of open source technologies. Our current stack looks like:

Technologies include Snowplow, Airbyte, PostgreSQL, dbt, Snowflake, Metabase, and Monosi itself. All of it is hosted on AWS running in a VPC.

Data Collection / Extraction / Ingestion

There are two ways we collect data and ingest it into our pipeline.

First, we have data coming in from various third party tools such as Google Analytics, MailChimp, Github, Slack, etc. For extraction we use Airbyte.

We chose Airbyte because of their open source self-hosted offering and their connector flexibility. There are other open source tools in the market like StitchData or Meltano but have limitations as discussed in this post by the Airbyte team. If you don’t want to self host extraction, consider Fivetran.

We also have telemetry set up on our Monosi product which is collected through Snowplow,. As with Airbyte, we chose Snowplow because of its open source offering and because of their scalable event ingestion framework. There are other open source options to consider including Jitsu and RudderStack or closed source options like Segment. Since we started building our product with just a CLI offering, we didn’t need a full CDP solution so we chose Snowplow.

Data Storage

To store our collected data, we use a Snowflake data warehouse for all final data collection and a PostgreSQL database as an event destination.

All data from third party tools extracted with Airbyte is piped into Snowflake. We chose Snowflake because it’s easy to get started with, has a simple UX, and we use it internally for testing. Snowflake offers a free trial to quickly get started with and provides an intuitive UI. Since Monosi supports a Snowflake integration, it also makes development and testing easier.

All events collected with Snowplow are piped into a PostgreSQL database. This is separate because we wanted an individual database to act as storage for product event collection and it was quick to setup with Snowplow.

The setup also enables us to sync all of our event data to the data warehouse with Airbyte and perform necessary transformations on the data with dbt cloud.

Data Visualization / Analysis

To analyze the data stored in Snowflake and Postgres, we use Metabase. We chose Metabase because of it’s open source offering and easy to use interface. Other open source tools like Lightdash and Superset exist which we may add to the stack as our data team grows.

Data Reliability

As a data driven company, we need to make sure that our data is flowing reliably from extraction to visualization. It is important to have our metrics up to date and accurately defined at all times. Hence, when a certain part of the data stack stops working or changes (e.g. event ingestion, data extraction, schema updates) we need to be aware of what happened and why.

This is where our own tool, Monosi, comes in. To ensure data reliability within our stack, we’ve deployed our own internal instance of Monosi. With Monosi, we get alerted if anomalies occur in our data and can perform root cause analysis.

Other Tooling

There are other tools that we will have to adopt in the future but haven’t yet due to lack of necessity. Specifically, one category that is popular in modern data stacks is Reverse ETL (Hightouch, Census, or Grouparoo). We currently don’t have a usecase for piping data back into 3rd party tools but it will definitely come up in the future.

Wrapping up

Thanks for reading the overview of what our startup data stack looks like! If you have any questions about the setup or would like to see a more in depth post with actual Terraform files that handle both service creation and networking, leave a comment below!

How to monitor Segment event data with Monosi

Ivan Porollo — Fri, 18 Feb 2022 17:03:59 +0000

Segment is a popular CDP (Customer Data Platform) used by over 20,000 companies. One of the primary use cases for Segment is to collect event or clickstream data and pipe it into a destination of choice.

With any event collection system, anomalies are bound to occur in the ingestion pipeline and the data itself. General observability of such pipelines is important. It is possible with tools like Datadog, which collects metadata around event delivery such as latency and delivery success vs. failure.

Unfortunately, these tools fall short when it comes to monitoring the actual data (and the related metadata) collected in the events.

Segment events generally hold vital information standardized by a set schema and instrumented in code. Schemas defined in code are occasionally incorrect or not set to the underlying standards. This quickly becomes a data quality issue which cannot be detected by standard observability tools.

This is where data observability tooling comes to the rescue. Monosi is an open source data observability and monitoring platform for data teams (see Monosi Github). It is used to quickly set up monitors on a data store to run checks for data quality issues and alert on detected anomalies.

This post will walk you through getting started with Segment event data monitoring through the Monosi platform, ensuring the data quality of your events in less than 10 minutes.

Prerequisites

For this tutorial, you will need a Segment business account with an event source set up. If you don't have a business plan, Segment offers a 14 day free trial. If you don't have an event source, learn more on how to start collecting events with Segment here.

You will also need one of your Segment destinations to be Postgres or Snowflake to work with Monosi. If you don't have either, I recommend creating a free Snowflake account and creating a Snowflake destination in Segment. For the purposes of this tutorial, we will be using Snowflake, but Postgres works as well.

Finally, you will need Docker to install and run Monosi. If you're not familiar with Docker, follow the tutorial here. Make sure you have docker installed and ready to use.

Installing Monosi

Monosi provides a Docker image to run the web interface and simplify deployment. To install and run Monosi through Docker, run the following command:

docker run -p 3000:3000 monosi/monosi

Note: Monosi also has a CLI version distributed through pypi. More information on how to run Monosi through the CLI can be found here.

Connecting to the Segment Event Data Store

Since Segment is writing data to a data store destination (in this case Snowflake), we need to give Monosi the appropriate connection details.

Set up a connection to the data store in the UI by navigating to http://localhost:3000/settings/sources. Click the Create a Data Source button and fill out the connection details.

Integrating with Slack

Monosi sends alerts on detected anomalies to Slack. Set up a Slack integration by navigating to http://localhost:3000/settings/integrations and creating a new integration with a Slack channel webhook. More information can be found here.

Creating a Monitor

With the Segment event data source and the Slack integration connected, we can now create a monitor. Navigate to the Monosi page http://localhost:3000/monitors and select the table health monitor.

Fill out the form with a monitor name, description, and an interval for the monitor to run on (in minutes).

Then, select your created data source. For the purposes of this tutorial we are using Snowflake, therefore the inputs for a Segment Page event monitor would be:

Table name: SEGMENT_WAREHOUSE.SEGMENT_EVENTS.PAGES
Timestamp column: RECIEVED_AT

Save the monitor and it will appear in the monitors index view. This monitor will run until deletion. If any anomalies in the data are detected, it will send them to the connected Slack channel.

Wrapping up

🎉 Congratulations, you've just set up and scheduled a data monitor on your Segment event data. You can now add more monitors to other event tables in your database. Find more information on how to use Monosi here.

If you have any questions, join our Slack community or open an issue in our repository on Github. If you want to see more posts like this, subscribe to our newsletter!

How to Monitor Supabase with Monosi

Ivan Porollo — Tue, 15 Feb 2022 22:01:08 +0000

Monosi is an open source data observability and monitoring platform for data teams (see Monosi Github). It is used to quickly set up monitors on a data store to run checks for data quality issues and alert on detected anomalies.

Supabase is an open source Firebase alternative. It lets you easily create a backend with a Postgres database, storage, APIs and more.

In this article, we will show you how to set up data observability for Supabase using Monosi. With this setup, you will be able to detect data anomailes that occur in your Supabase instance.

Prerequisites

For this tutorial, you will need a Supabase account with a populated database. If you don't have an account, it's quick to get started with their platform by navigating to the Supabase app and following their getting started documentation.

You will also need Docker to install and run Monosi. If you're not familiar with Docker, follow the tutorial here. Make sure you have docker installed and ready to use.

Installing Monosi

Monosi provides a Docker image to run the web interface and simplify deployment. To install and run Monosi through Docker, run the following command:

docker run -p 3000:3000 monosi/monosi

Note: Monosi also has a CLI version distributed through pypi. More information on how to run Monosi through the CLI can be found here.

Connecting to Supabase

Since Monosi supports PostgreSQL, it can connect and monitor any Supabase instance.

To get your database credentials from Supabase, navigate to the Settings > Database page and find the Connection info section.

With the Supabase connection details in hand, navigate to the Monosi datasources page at http://localhost:3000/settings/sources. Fill out the Postgres connection details with your respective Supabase instance.

Integrating with Slack

Creating a Monitor

With the Supabase data source and Slack integration connected, we can now create a monitor. Navigate to the Monosi page http://localhost:3000/monitors and select the table health monitor.

Fill out the form with a monitor name, description, and an interval for the monitor to run on (in minutes).

Then, select the Supabase data source and enter the table name that you want monitored. Ensure that this table has a timestamp column. The timestamp column field must be put into the timestamp field.

Save the monitor and it will appear in the monitors index view. This monitor will run until deletion. If any alerts are detected, it will send them to the connected Slack channel.

Wrapping up

🎉 Congratulations, you've just set up and scheduled a data monitor on your Supabase instance. You can now add more monitors to other tables in your database. Find more information on how to use Monosi here.

If you have any questions, join our Slack community or open an issue in our repository on Github. If you want to see more posts like this, subscribe to our newsletter!

Setting up data monitoring for PostgreSQL

Ivan Porollo — Tue, 08 Feb 2022 23:26:30 +0000

Data quality and reliability are still a source of headaches for data organizations today. Monosi exists to resolve the issues that teams face.

This article will walk you through how to get started monitoring a PostgreSQL database in less than 10 minutes with Monosi.

Pre-requisites

For this tutorial, we are going to use PostgreSQL as our database to monitor. We are going to use a public Postgres instance that is available online from RNAcentral.

Installing Monosi (CLI)

There are two ways of using Monosi - a CLI and a web interface. We will first run through the CLI workflow, then showcase the web UI.

Monosi's CLI is a Python package that is distributed through pypi.

Note: Monosi CLI is only compatible with Python 3.6 and higher.

To install it, open your terminal and run:

pip install monosi

Check that it's been installed by running:

monosi --version

Configuring a data source

With Monosi installed, we need to configure a data source to monitor. Monosi reads connection details from the ~/.monosi/workspaces.yml file, so let's create it:

mkdir ~/.monosi
touch ~/.monosi/workspaces.yml

Edit the workspaces.yml file in your editor of choice and fill it out with the public database information. This is what the file will look like:

default:
    sources:
        default:
            type: postgres
            user: reader
            password: NWDMCE5xdipIjRrp
            host: hh-pgsql-public.ebi.ac.uk
            port: 5432
            database: pfmegrnargs

You can find more information on setting up the workspaces.yml file here.

Creating a Monosi project

Navigate to the directory where you want your Monosi project to live and create a project repository by running:

monosi init

In the directory you should now see a folder called monosi-repo (you can rename this if you want). Navigate into the folder by running:

cd monosi-repo

In the monosi-repo directory, you should see a monosi_project.yml file.

This file configures which connection to use and your monitor paths, as well as other metadata. More information on setting up the monosi_project.yml file can be found here.

For the purposes of this tutorial, we don't need to edit the file.

Creating a monitor

Monosi automatically creates a folder called monitors in the monosi-repo directory. In that folder, an example custom SQL monitor is defined to show how the syntax works.

The example public database is populated with 40+ tables. To take a look at how the schema is structured, click here. For the purposes of this example, we will create a monitor on one of the tables, specifically auth_user. To do so, run:

touch ./monitors/auth_user.yml

Edit the auth_user.yml file in your editor of choice and paste the following:

monosi:
  monitors:
  - name: pfmegrnargs.rnacen.auth_user - Table Health
    description: Monitoring the health of the auth_user table
    type: table
    table: pfmegrnargs.rnacen.auth_user
    timestamp_field: last_login

This creates a monitor as code on the provided auth_user table. It monitors for table health metrics.

Run the monitors

Start the monitor by running the following command in the monosi-repo directory:

monosi run

It takes a few seconds to run. The resulting output should be similar to:

🎉 Congratulations, you just ran your first Monosi monitor! From the output of the run, you should see that the public data on the auth_user table has no anomalies that the Monosi monitor has detected.

Installing Monosi (Web Interface)

Note: At the moment, the web interface is decoupled from the CLI, so any monitors that are created in the CLI will not be synced.

With the release of v0.0.3, Monosi has a web user interface. For a full walkthrough of the UI, watch the video here.

Monosi provides a docker image to run the web application. If you're not familiar with docker, follow the tutorial here. Make sure you have docker installed and ready to use by running:

docker ps

To get the Monosi UI up and running with docker, run the following command:

docker run -p 3000:3000 monosi/monosi

Navigate to http://localhost:3000 and you will see the Monosi UI.

Set up a connection to the public PostgreSQL instance in the UI by navigating to http://localhost:3000/settings/sources.

Fill out the database connection form with the following values:

Name for Data Source: RNAcentral Public Database
User: reader
Password: NWDMCE5xdipIjRrp
Host: hh-pgsql-public.ebi.ac.uk
Port: 5432
Database: pfmegrnargs

After setting up the connection, create a monitor in the UI by navigating to http://localhost:3000/monitors and clicking the Create Monitor button. Fill out the form with the following information:

Name: auth_user table health
Description: auth_user table health monitor
Check every: 720 minutes
Monitor type: Table Health
Data Source: RNAcentral Public Database
Table: pfmegrnargs.rnacen.auth_user
Timestamp Field: last_login

Hit save and your new monitor will appear in the monitors table.

🎉 Congratulations, you've just set up and scheduled a data monitor! This process will run indefinitely until you delete the monitor. To get alerts on detected anomalies, set up a Slack connection in http://localhost:3000/settings/integrations

Wrapping up

Now that you’ve worked through an example using a public PostgreSQL instance, you can further extend this to your own data store. For more information, get started here.

If you have any questions, join our Slack community or open an issue in our repository on Github.

Setting up data monitoring for Snowflake

Ivan Porollo — Mon, 31 Jan 2022 08:51:46 +0000

Updated Feb. 4th, 2022

Data quality and reliability are still a source of headaches for data organizations today. Monosi exists to resolve the issues that teams face.

This article will walk you through how to get started monitoring a data warehouse in less than 10 minutes with Monosi.

Pre-requisites

For this tutorial, we are going to use Snowflake as our data warehouse.

If you don't have a Snowflake account, it's easy to create one. Go ahead and create an account - select the standard edition and your preferred cloud provider (for this example we will be using AWS - US West).

After signing up, you should receive an email with account details to login. With the account details in place, you are ready to start with Monosi.

Installing Monosi

Monosi is a Python package that is distributed through pypi.

Note: Monosi is only compatible with Python 3.6 and higher.

To install it, open your terminal and run:

pip install monosi

Check that it's been installed by running:

monosi --version

Configuring a data source

With Monosi installed, we need to configure a data source to monitor. Monosi reads connection details from the ~/.monosi/workspaces.yml file, so let's create it:

mkdir ~/.monosi
touch ~/.monosi/workspaces.yml

Edit the workspaces.yml file in your editor of choice and fill it out with your information (specifically <user-name>, <password>, <account-name>). This is what the file will look like:

default:
    sources:
        default:
            type: snowflake
            user: <user-name>
            password: <password>
            account: <account-name>
            warehouse: COMPUTE_WH
            database: SNOWFLAKE_SAMPLE_DATA

For the purposes of this tutorial, we will be using the SNOWFLAKE_SAMPLE_DATA database. This database is provided out of the box by Snowflake.

You can find more information on setting up the workspaces.yml file here.

Creating a Monosi project

Navigate to the directory where you want your Monosi project to live and create a project repository by running:

monosi init

In the directory you should now see a folder called monosi-repo (you can rename this if you want). Navigate into the folder by running:

cd monosi-repo

In the monosi-repo directory, you should see a monosi_project.yml file.

This file configures which connection to use and your monitor paths, as well as other metadata. More information on setting up the monosi_project.yml file can be found here.

For the purposes of this tutorial, we don't need to edit the file.

Creating a monitor

Monosi automatically creates a folder called monitors in the monosi-repo directory. In that folder, an example custom SQL monitor is defined to show how the syntax works.

As mentioned, Snowflake provides some example data in their SNOWFLAKE_SAMPLE_DATA database. We can create a monitor for the orders table:

touch ./monitors/orders.yml

Edit the orders.yml file in your editor of choice and paste the following:

monosi:
  monitors:
  - table: SNOWFLAKE_SAMPLE_DATA.TPCH_SF100.orders
    timestamp_field: o_orderdate
    type: table
    days_ago: -10000

This creates a monitor as code on the provided orders table. It monitors for table health metrics.

Run the monitors

Start the monitor by running the following command in the monosi-repo directory:

monosi run

It takes a few seconds to run. The resulting output should be similar to:

🎉 Congratulations, you just ran your first Monosi monitor! From the output of the run, you should see that the example data has several anomalies that the Monosi monitor has detected.

Scheduling monitors

With the release of v0.0.3, we have added a user interface that supports the scheduling of monitors. We've also created a video walkthrough of the UI here.

Monosi provides a docker image to run the application. Make sure you have docker installed and ready to use by running:

docker ps

To get the Monosi UI up and running with docker, run the following command:

docker run -p 3000:3000 monosi/monosi

Navigate to http://localhost:3000 and you will see the Monosi UI.

Set up a connection to your Snowflake account in the UI by navigating to http://localhost:3000/settings/sources.

After setting up the connection, create a monitor in the UI by navigating to http://localhost:3000/monitors and clicking the Create Monitor button. Fill out the form with the following information:

Name: Orders Monitor
Check every: 720 minutes
Monitor type: Table Health
Data Source: <Your Snowflake Datasource Name>
Table: SNOWFLAKE_SAMPLE_DATA.TPCH_SF100.orders
Timestamp Field: o_orderdate

Hit save and your new monitor will appear in the monitors table.

🎉 Congratulations, you just scheduled a data monitor! This process will run indefinitely until you delete the monitor. To get alerts on detected anomalies, set up a Slack connection in http://localhost:3000/settings/integrations

Wrapping up

Now that you’ve worked through an example using Snowflake's provided data, you can further extend this to your own data with the Monosi profiler and custom SQL monitors.

If you have any questions, join our Slack community or open an issue in our repository on Github.