DEV Community: Mailmeteor

How to schedule tasks in more than 30 days in Google Cloud Tasks API?

Corentin — Mon, 28 Mar 2022 13:40:55 +0000

At Mailmeteor, we rely heavily on Google Cloud Tasks to send emails. In fact, each time we send an email, there's one (or more) Cloud Tasks associated to it. That's a lot of tasks in the end.

While Google's product is really robust, one thing that has always being tricky is that you can't schedule a task that will run in more than 30 days.

Resource	Value	Description
Maximum schedule time for a task	30 days from current date and time	The maximum amount of time in the future that a task can be scheduled.

Extract from Google Cloud Tasks documentation on quotas & limits

It is still way more than what AWS proposes (AWS SQS - Simple Queue Service - lets you queue messages for up to 15 minutes). Nevertheless, there are so many use cases when having a very-long tasks scheduler is needed.

While I wasn't sure why Google has limited the execution delay to a month, one of their employee has explained on StackOverflow that such limit "is a design decision. Google does not charge for the storage space of tasks, so extending that would detrimental to our costs." (source).

Though, Google Cloud Tasks is already a paid product. So extending the date, whether you need to pay for it or not, wouldn't be that much of an issue for them. In fact, according to this StackOverflow thread, more than 1,000 people have been interested in extending the task delay. And there's already a feature request, back from 2020, which I urge you to star to make sure Google prioritizes this.

Too much talking. Let's see how we can keep using Google Cloud Tasks and extend the execution delay "to infinity and beyond".

Solution

The trick is in adding an ETA header to your tasks. This way, before executing the task, you can check if the ETA is now (and thus execute the task) or in the future (and thus re-schedule the task). This way you can recursively keep creating tasks and eventually execute your task at your desired time.

Let's take an example:

I have a task to run in 45 days
I create a new task with the max execution time (30 days)
Then:
- 30 days later, the task executes, but it's too early, so I reschedule it in 45-30 = 14 days
- 14 days later (45 days in total), the task executes normally.

In fact, doing so let's you create tasks in 1 year (or more) from now.

Implementation (JS)

In Express.js, all you need is a middleware that will check if the execution time is in the future, and if so will reschedule the tasks:

// Middleware to reschedule Google Cloud Tasks
export const googleTasksScheduleMiddleware = async (req, res, next) => {
  const taskETAHeader = req.headers['google-cloud-tasks-eta'];

  // If no header, skip middleware
  if(taskETAHeader == null || taskETAHeader == ""){
    next()
    return
  }

  const now = Date.now();
  const intHeader = parseInt(taskETAHeader);

  // Time has passed, let's process the task now
  if(intHeader < now) {
    next()
    return
  }

  // It's too early! Reschedule the task
  else {
    // Construct the task.
    createTask(req.method, req.url, req.headers, req.body)

    res.send('Re-scheduled')    
    return
  }
}

Then, add your middle before the first routes of your application:

app.use(googleTasksScheduleMiddleware)

Conclusion

As you can see, it's pretty easy to implement and doesn't require refactoring your application. If you are interested in more engineering articles from Mailmeteor, make sure to follow my account.

Cannot find name 'Bugsnag'

Corentin — Wed, 03 Nov 2021 13:28:07 +0000

When you get started with Bugsnag, they have a handy guide to help you make sure errors are well reported to their monitoring tool. In Mailmeteor, we are using Bugsnag to make sure our tools are robust and bug-free.

But recently, when initializing a new TypeScript project on Bugsnag, I end up having this error: Cannot find name 'Bugsnag' (which might also be seen as Bugsnag is not defined).

Quick fix

Just add import Bugsnag from '@bugsnag/js'. The type error is gone :-)

How to add a sparkline to your Vue.js app

Corentin — Tue, 13 Jul 2021 14:02:31 +0000

Very recently, I was looking to add a neat sparkline to a Vue.js application of my own.

As always, I googled just that, looking for sparkline vue.js or sparkline npm. But I couldn't find something that was easy, with a small footprint and yet customizable.

After playing a bit with Chart.js, I just stopped and considered how I could build a decent, yet very simple, sparkline component (i.e. without any dependency).

If you look at how npm's sparkline works as well as the ones from Stripe's dashboard, you will quickly realize that it's just a SVG element that you customize with JavaScript.

So bear with me, I'll show you how to do just that.

Demo

Behind the scene

The sparkline is just a Vue.js component where you provide the coordinates of the sparkline as an array. Here's how I've rendered the sparkline in the example above:

<sparkline v-bind:data="[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31]"></sparkline>

The source code of the component is the following:

As you might have noticed, the code renders an <svg> HTML element by computing the coordinates of the different <path>.

There are two <path>. One for the blue line. And another one for the blue background. I've used the color #1f8ceb but of course this is totally customizable, just like the width/height of the sparkline.

That component is pretty basic and contrary to NPM or Stripe, it doesn't handle when a mouse hovers the <svg>. I didn't need that for my use case, but if ever you implement it, feel free to edit the gist and share it with the community.

Basic mail merge script for Gmail (using Google Apps Script)

Corentin — Mon, 12 Apr 2021 15:04:32 +0000

Sending mass emails from Gmail is sometimes seen as a challenge. I can tell you it's not! In this article we'll look at the basics of building a mail merge script for Gmail and how it can fit your email marketing needs.

First, here's what you need to send emails in bulk with Gmail:

a Gmail (or Google Workspace) account
a list of contacts
a template of the emails you want to send
basic development skills

Now let's see the 6 steps to reproduce to create your Google script to send emails.

Tutorial: Building a mail merge script for Gmail

Open Google Apps Script
Click on the "New project" button to create a new project.
You should now see the script editor.
From the editor, copy and paste the following script:

Update the recipients variable with your list of recipients. Also update the template subject and content to make it fit your needs
Click on "Save" to save your changes
Then click on "▶️ Run" to send your emails.

Note: the first time you use this script it will ask for the permission to send emails on your behalf. Then it won't ask you again.

Emails are usually immediately delivered, but sometimes it takes a few seconds. Check your "Sent" folder in Gmail to confirm that all emails have been sent! As you will see, all your recipients receive a unique email sent from your email address. You don't need to use bcc or any other techniques.

Conclusion

Some say that it's impossible to send emails in bulk with Gmail. It's totally untrue. Of course, this is a very basic macro for Google Sheets to send lots of emails with Gmail. You might need to adapt it a little bit more, but thanks to Google Apps Script you can do lot of things.

Before you go, here are two things to keep in mind when using Gmail as an email marketing tool:

Be sure to abide by Gmail bulk senders guidelines. Especially, note that you are limited to send a reasonable amount of emails per day. If you have a @gmail.com email address, you can send at most 500 emails/day while with a Google Workspace account, you sending limit hits 2000 emails per day. That's probably well enough for 99% of Gmail users, but it's good to have in mind those limits.
If you are looking for more a advanced script, for example that let you send personalized emails (e.g. Hi {{ firstname }}), I'd recommend you to use a Gmail mail merge tool such a Mailmeteor that does it for you. It already handles personalization, as well as dozens of features from attachments to aliases. Learn more about the features and benefits of Mailmeteor here.

This article is part of an extended guide on Mail merge in Gmail (2021). If you want to learn much more, go check it out!

How to send a mail merge with Gmail?

Corentin — Wed, 03 Mar 2021 10:59:05 +0000

There are 2 ways to mail merge in Gmail. You can either use a Google add-on that will do the job for you or build your own mail merge script in Gmail.

We’ll cover both methods in this guide. Even though we recommend using software built for that purpose which cover most issues and will probably save you time.

Method 1: building a merge in Gmail using Google Apps Scripts

As a developer, it's a good challenge to try to build your own Gmail mail merge without an add-on. And thankfully we’re going to use Google Apps Script, which makes it really easy to create Google add-ons.

Google Developer Advocates have already released a great script to help us move forward with the code.

Here is the latest version of the open-source code hosted on GitHub written by Martin Hawksey - @mhawksey. We gonna look at it step-by-step just after.

// Copyright Martin Hawksey 2020
//
// Licensed under the Apache License, Version 2.0 (the "License"); you may not
// use this file except in compliance with the License.  You may obtain a copy
// of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
// WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
// License for the specific language governing permissions and limitations under
// the License.

/**
 * @OnlyCurrentDoc
*/

/**
 * Change these to match the column names you are using for email 
 * recipient addresses and email sent column.
*/
const RECIPIENT_COL  = "Recipient";
const EMAIL_SENT_COL = "Email Sent";

/** 
 * Creates the menu item "Mail Merge" for user to run scripts on drop-down.
 */
function onOpen() {
  const ui = SpreadsheetApp.getUi();
  ui.createMenu('Mail Merge')
      .addItem('Send Emails', 'sendEmails')
      .addToUi();
}

/**
 * Send emails from sheet data.
 * @param {string} subjectLine (optional) for the email draft message
 * @param {Sheet} sheet to read data from
*/
function sendEmails(subjectLine, sheet=SpreadsheetApp.getActiveSheet()) {
  // option to skip browser prompt if you want to use this code in other projects
  if (!subjectLine){
    subjectLine = Browser.inputBox("Mail Merge", 
                                      "Type or copy/paste the subject line of the Gmail " +
                                      "draft message you would like to mail merge with:",
                                      Browser.Buttons.OK_CANCEL);

    if (subjectLine === "cancel" || subjectLine == ""){ 
    // if no subject line finish up
    return;
    }
  }

  // get the draft Gmail message to use as a template
  const emailTemplate = getGmailTemplateFromDrafts_(subjectLine);

  // get the data from the passed sheet
  const dataRange = sheet.getDataRange();
  // Fetch displayed values for each row in the Range HT Andrew Roberts 
  // https://mashe.hawksey.info/2020/04/a-bulk-email-mail-merge-with-gmail-and-google-sheets-solution-evolution-using-v8/#comment-187490
  // @see https://developers.google.com/apps-script/reference/spreadsheet/range#getdisplayvalues
  const data = dataRange.getDisplayValues();

  // assuming row 1 contains our column headings
  const heads = data.shift(); 

  // get the index of column named 'Email Status' (Assume header names are unique)
  // @see http://ramblings.mcpher.com/Home/excelquirks/gooscript/arrayfunctions
  const emailSentColIdx = heads.indexOf(EMAIL_SENT_COL);

  // convert 2d array into object array
  // @see https://stackoverflow.com/a/22917499/1027723
  // for pretty version see https://mashe.hawksey.info/?p=17869/#comment-184945
  const obj = data.map(r => (heads.reduce((o, k, i) => (o[k] = r[i] || '', o), {})));

  // used to record sent emails
  const out = [];

  // loop through all the rows of data
  obj.forEach(function(row, rowIdx){
    // only send emails is email_sent cell is blank and not hidden by filter
    if (row[EMAIL_SENT_COL] == ''){
      try {
        const msgObj = fillInTemplateFromObject_(emailTemplate.message, row);

        // @see https://developers.google.com/apps-script/reference/gmail/gmail-app#sendEmail(String,String,String,Object)
        // if you need to send emails with unicode/emoji characters change GmailApp for MailApp
        // Uncomment advanced parameters as needed (see docs for limitations)
        GmailApp.sendEmail(row[RECIPIENT_COL], msgObj.subject, msgObj.text, {
          htmlBody: msgObj.html,
          // bcc: 'a.bbc@email.com',
          // cc: 'a.cc@email.com',
          // from: 'an.alias@email.com',
          // name: 'name of the sender',
          // replyTo: 'a.reply@email.com',
          // noReply: true, // if the email should be sent from a generic no-reply email address (not available to gmail.com users)
          attachments: emailTemplate.attachments,
          inlineImages: emailTemplate.inlineImages
        });
        // modify cell to record email sent date
        out.push([new Date()]);
      } catch(e) {
        // modify cell to record error
        out.push([e.message]);
      }
    } else {
      out.push([row[EMAIL_SENT_COL]]);
    }
  });

  // updating the sheet with new data
  sheet.getRange(2, emailSentColIdx+1, out.length).setValues(out);

  /**
   * Get a Gmail draft message by matching the subject line.
   * @param {string} subject_line to search for draft message
   * @return {object} containing the subject, plain and html message body and attachments
  */
  function getGmailTemplateFromDrafts_(subject_line){
    try {
      // get drafts
      const drafts = GmailApp.getDrafts();
      // filter the drafts that match subject line
      const draft = drafts.filter(subjectFilter_(subject_line))[0];
      // get the message object
      const msg = draft.getMessage();

      // Handling inline images and attachments so they can be included in the merge
      // Based on https://stackoverflow.com/a/65813881/1027723
      // Get all attachments and inline image attachments
      const allInlineImages = draft.getMessage().getAttachments({includeInlineImages: true,includeAttachments:false});
      const attachments = draft.getMessage().getAttachments({includeInlineImages: false});
      const htmlBody = msg.getBody(); 

      // Create an inline image object with the image name as key 
      // (can't rely on image index as array based on insert order)
      const img_obj = allInlineImages.reduce((obj, i) => (obj[i.getName()] = i, obj) ,{});

      //Regexp to search for all img string positions with cid
      const imgexp = RegExp('<img.*?src="cid:(.*?)".*?alt="(.*?)"[^\>]+>', 'g');
      const matches = [...htmlBody.matchAll(imgexp)];

      //Initiate the allInlineImages object
      const inlineImagesObj = {};
      // built an inlineImagesObj from inline image matches
      matches.forEach(match => inlineImagesObj[match[1]] = img_obj[match[2]]);

      return {message: {subject: subject_line, text: msg.getPlainBody(), html:htmlBody}, 
              attachments: attachments, inlineImages: inlineImagesObj };
    } catch(e) {
      throw new Error("Oops - can't find Gmail draft");
    }

    /**
     * Filter draft objects with the matching subject linemessage by matching the subject line.
     * @param {string} subject_line to search for draft message
     * @return {object} GmailDraft object
    */
    function subjectFilter_(subject_line){
      return function(element) {
        if (element.getMessage().getSubject() === subject_line) {
          return element;
        }
      }
    }
  }

  /**
   * Fill template string with data object
   * @see https://stackoverflow.com/a/378000/1027723
   * @param {string} template string containing {{}} markers which are replaced with data
   * @param {object} data object used to replace {{}} markers
   * @return {object} message replaced with data
  */
  function fillInTemplateFromObject_(template, data) {
    // we have two templates one for plain text and the html body
    // stringifing the object means we can do a global replace
    let template_string = JSON.stringify(template);

    // token replacement
    template_string = template_string.replace(/{{[^{}]+}}/g, key => {
      return escapeData_(data[key.replace(/[{}]+/g, "")] || "");
    });
    return  JSON.parse(template_string);
  }

  /**
   * Escape cell data to make JSON safe
   * @see https://stackoverflow.com/a/9204218/1027723
   * @param {string} str to escape JSON special characters from
   * @return {string} escaped string
  */
  function escapeData_(str) {
    return str
      .replace(/[\\]/g, '\\\\')
      .replace(/[\"]/g, '\\\"')
      .replace(/[\/]/g, '\\/')
      .replace(/[\b]/g, '\\b')
      .replace(/[\f]/g, '\\f')
      .replace(/[\n]/g, '\\n')
      .replace(/[\r]/g, '\\r')
      .replace(/[\t]/g, '\\t');
  };
}

Follow the guidelines below to understand how to mail merge in Gmail using Apps Script:

1. Create a copy of the sample mail merge spreadsheet

Open this demonstration spreadsheet and click on “Make a copy” to get your own copy.

2. In your new spreadsheet

Click on Tools > Script editor to open Google Apps Script. From there, you will see that a script is already tied to your spreadsheet. That's because you've made a copy of the previous spreadsheet!

In the script editor, you can update the code as you wish. Changes will be reflected immediately.

Back to copied spreadsheet, update the “Recipients” column with email addresses you would like to use in the mail merge. Replace the cells’ value under the “Recipients” column with your own email address for example.

3. Now, open Gmail to create a new draft email

You can use personalized variables, like {{ firstname }} which correspond to column names of the spreadsheet you just copied. This indicates text you’d like to be replaced with data from the copied spreadsheet.

4. Back in the copied spreadsheet, click on the custom menu item called “Mail Merge” and then click on “Send Emails”

This item menu was created by the Apps Script project and will start the mail merge process.

5. A dialog box appears for authorization. Read the authorization notice and continue

Important note: The script we are using has been created and proofread by Google Apps Script teams. Always be super careful when authorizing scripts and third-party apps in general.

When prompted enter or copy-paste the subject line used in your draft Gmail message. Then click OK.

6. Sending your emails

You will see that the “Email Sent” column will update with the message status. Back in Gmail, check your Sent folder and review the emails the program just sent for you!

Keep in mind that using a script is at your own risk. Check Gmail’s sending limit before sending large volumes of emails. Especailly, be aware that your account can get blocked by Gmail if your emailing activity seems unusual in the eyes of anti-spam filters.

For these reasons, we would recommend using a mail merge solution such as Mailmeteor. Mailmeteor deals with all these aspects for you and ensures that your privacy remains protected.

Method 2: using a mail merge add-on like Mailmeteor

We’ll start with a real-life example to show you how to do a mail merge from Gmail using a Google Sheets add-on. In this example, we’re using Mailmeteor, the best rated Google mail merge add-on.

1. Get Mailmeteor

All you have to do is to install Mailmeteor from the Google Workspace Marketplace. The Worskpace Marketplace is a place where you can find all the apps compatible with your Google Suite. Mailmeteor is a tool that integrates with Gmail and Google Sheets to merge emails with Gmail.

2. Add contact in Google Sheets

Once Mailmeteor is installed, open a Google Sheets spreadsheet.

First of all, you will need to add recipients to a Google Sheets spreadsheet. This spreadsheet will be the place where you store your contact list. You will also be able to track your campaign metrics from there.

To create a mailing list you can either add your recipients manually or import contacts. To import contacts in Google Sheets, go to Menu > File > Import and select your Excel or .csv file.

Here’s a mail merge demo spreadsheet we’re going to use:

Link to mail merge demo spreadsheet

Note: when opening Mailmeteor for the first time, you will be guided through a quick onboarding tutorial. A demo spreadsheet like this one will be created for you.

Let’s breakdown how your spreadsheet should look like:

➤ Add column headers on the 1st row

Mailmeteor will pull the information from your spreadsheet to personalize your emails. Each column represents a personalized field. This field will be replaced in your email template.

In our example, we have 4 columns named: firstname, email, company, postscriptum.

Add as many columns as you want and pick any column header name you want. Make sure you have a column named “email”.

➤ Fill the columns with your recipients’ information

Fill your spreadsheet with your recipients’ info. Ensure all email cells are filled with valid email addresses. Apart from the emails, you can leave some cells blank, that is fine! In the example below, some recipients will get a Post Scriptum whereas others won't.

3. Open Mailmeteor from the Add-ons menu in Google Sheets

Once your contact list is ready, open Mailmeteor. To open Mailmeteor go to the menu and select Add-ons> Mailmeteor > Open Mailmeteor.

This is the Mailmeteor interface. It tells you how many emails you can send per day and details related to your campaign. Next, we are going to compose the template that will be used for the mail merge.

4. Compose a new email template

Click on the “Create new template” button. This will open an editor in which you can compose your email. The Mailmeteor editor is the exact same as Gmail, you will find all the actions you need to customize your email.

Now we’re going to personalize your email. Personalizing emails is important as it helps make your recipients feel unique when they receive your emails. Using personalization will also dramatically improve your opening rates - and thus the replies you will get.

5. Personalize your emails

A mail merge transforms a standard email template into a personalized email copy. It’s done by replacing variables fields within the template with the content from your spreadsheet.

To insert a variable it’s easy: add variables using double brackets like this {{ firstname }}

Here is a template you can copy-paste:

When adding a variable, always make sure that it matches a header in your spreadsheet.

Once you are satisfied with your template, click the “Save” button.

6. Preview emails before sending

Mailmeteor offers a preview feature that is super helpful to review emails before sending. The preview mode gives you a glimpse of the actual output of your email once personalized for each recipient.

You can also send a test email to yourself. Testing your emails on several devices is a best practice. This will ensure your emails will display correctly in most situations.

7. Send your mail merge

Ready for take-off? It’s time to send your mailmerge campaign.

We know that sending your mail merge can be a bit daunting at first. No worries though, if you follow these steps, everything will be alright!

✨ That's it! You are now ready to mail merge emails with Gmail using an add-on such as Mailmeteor ✨

Here's a real-life example. Watch this teacher use Mailmeteor to mail merge emails to his students:

Now it’s your turn

That’s it for this guide to mail merge in Gmail. We hope you now better understand this simple yet incredibly powerful tool called mail merge.

This guide is part of an extended guide on Mail merge in Gmail (2021). If you want to learn much more, go check it out!

⚡️ How to call an OAuth based API in React?

Corentin — Wed, 02 Sep 2020 15:43:19 +0000

Do you know what Facebook, Google, GitHub, and thousands more APIs have in common? They use OAuth to authenticate requests.

OAuth, especially OAuth 2.0, is now everywhere. It's a very powerful authentication framework that powers up developers to have granularity over the data that it needs.

React + OAuth = 🤔

When it comes to OAuth-based API, your React app is not well-suited to send requests. You just can't hide your API keys deep into your codebase. Someone will easily find it.

What you need to do is to set up some backend service, that proxies requests to the third-party API. It can be fun to do, but it's a long process for a quick API call.

For example, you will need a backend to protect your API keys. Declare routes, understand packages (like Passport.js), proxying requests, and dozens of more actions. It doesn't have to be so hard.

Today, I'll showcase an open-source project that I'm actively contributing to. It's called Pizzly and it helps a lot with using API from a single page application.

Let's see how it looks like with a simple demo:

Curious about how you can do it on your application? Here's a full guide.

The React skeleton 🦴

To learn how to make API calls to an API, we first need a React skeleton. And the least that we need is an app that consumes an API endpoint using OAuth2.

As you probably have a GitHub account, we will use that API, but you can easily switch to any other API that uses OAuth2 (Slack, Salesforce, ...) or OAuth1 (Twitter, Trello, ...).

Here's how the application will look like:

import React, { useState } from 'react'
import Pizzly from 'pizzly-js'
import Profile from './Profile'

const App = () => {
  const [profile, setProfile] = useState()

  return (
    <div className="App">
      <h1>Hello!</h1>
      <p>
        Click the button bellow to retrieve your GitHub profile using{' '}
        <a target="_blank" rel="noopener noreferrer" href="https://github.com/Bearer/Pizzly">
          Pizzly
        </a>
        .
      </p>
      <button onClick={connect}>Retrieve your GitHub profile</button>
      {profile && <Profile profile={profile} />}
    </div>
  )
};

export default App;

It's a very basic React application that renders the user's profile as a plain JSON when it has been fetched. Otherwise, it asks the user to connect to GitHub.

The authentication 🔐

Here, we gonna use Pizzly, the open-source project I told you about a few lines above.

It provides a .connect() method that triggers an authentication flow from your frontend, which you can handle with callbacks. No need to create a redirect URL, deal with backend, etc.

Let's see how to update the skeleton above to use with Pizzly.

First, we need to initialize Pizzly:

// Initialize Pizzly
const pizzly = new Pizzly({
  host: PIZZLY_HOSTNAME,
  publishableKey: PIZZLY_PUBLISHABLE_KEY
})

const github = pizzly.integration('github', {
  setupId: PIZZLY_SETUP_ID_GITHUB_DEMO_APP
})

Then, we add a new connect() method to trigger the authentication flow:

const App = () => {

  // ...

  /**
   * The connect method lets us authenticate a user
   * to our GitHub application (i.e. the OAuth dance)
   */

  const connect = () => {
    github
      .connect()
      .then(({ authId }) => {
        console.log('Sucessfully connected!', authId)
        fetchProfile(authId)
      })
      .catch(console.error)
  }

  // ...
};

export default App;

That's it. A few lines of code in your application and you are ready to handle any OAuth based API 💪.

The configuration 🤖

Pizzly is a self-hosted OAuth manager. This means that you need to host it somewhere, for example on Heroku (it takes 30 seconds). Once hosted, you can access Pizzly's dashboard, which is where you configure your GitHub integration.

To deploy your own Pizzly instance right now, click on any of the following button:

Heroku	Platform.sh

Then, select the GitHub API. And configure it by saving your application's client ID, client credentials and scopes. You will get them from GitHub by following this guide.

Once your Pizzly instance is created and you have configured a GitHub application, edit your React application with the following values:

// Pizzly environment variables, make sure to replace
// these with those of your own Pizzly instance
const PIZZLY_HOSTNAME = "";
const PIZZLY_PUBLISHABLE_KEY = "";
const PIZZLY_SETUP_ID_GITHUB_DEMO_APP = "";

The least that you need is PIZZLY_HOSTNAME. The two others are optional.

An authenticated API request 🎉

Alright, we have already spend a few minutes on the configuration. Let's move back to funny things.

The GitHub API provides a handy endpoint (/user) to retrieve the profile of the authenticated user. This endpoint uses OAuth authentication, so it looks like a good use case.

Let's add a new method to our application to do that:

const App = () => {
  // ...

  /**
   * The fetchProfile method retrieves the authenticated
   * user profile on GitHub (the request is proxied through Pizzly)
   */

  const fetchProfile = async (authId) => {
    await github
      .auth(authId)
      .get("/user")
      .then((response) => response.json())
      .then((json) => setProfile(json));
  };

  // ...
};

export default App;

And voilà!

NB: To authenticate the request, we have provided an authId. It's like a user identity for that particular API. It was generated (and saved) with the connect() method.

What's next? 💡

You now know how to authenticate a user towards any OAuth based API using React. If you prefer Vue.js, the same tutorial is available for Vue.js.

It's easily adaptable to all the most famous APIs. No need to create backend routes or understand every single concept of OAuth. Pizzly takes care of that for you (and for the experts, Pizzly automatically refreshes the access_token).

Again, have a look at the CodeSandbox to have a full understanding of the code and share your thoughts/questions in the comments below.

And if you've liked this tutorial, please add a star to Pizzly on GitHub. Here's the link: https://github.com/Bearer/Pizzly.

Request for Node.js has been deprecated

Corentin — Wed, 19 Feb 2020 15:05:33 +0000

The breaking news landed a few days ago, with a simple commit entitled "doc: note full deprecation".

That's right! request, one of the top and oldest npm packages, has been deprecated by @mikeal its creator.

As of writing, over 40K packages rely on request. In fact, it's the 4th most depended upon package on npm. And it has a whopping 18.5 million weekly downloads.

For the most familiar with the package's development, there's no news here. request was in maintenance mode since March 2019 (see issue #3142). And deprecating the package is just one step further.

Still, it is, I believe, a truly courageous and clever step toward the future of the Node.js (and more broadly the JavaScript) ecosystem.

Let's review what led to this situation.

Without paraphrasing too much @mikeal statement back from March 2019, here are the key points that led to this deprecation:

Back in 2009, request was one of the first modules ever created for the Node.js ecosystem.
For a few years, request and Node.js evolved together, each learning from the other.
And as one of the very first modules listed on npm, request quickly became one of the most downloaded.

But:

"The place request has in the Node.js ecosystem is no longer one of an innovator but of an incumbent." @mikeal

That's right that most tutorials, even here on DEV, use request as the per-default HTTP client - while there are dozens of very good alternatives.

"The best thing for these new modules is for request to slowly fade away, eventually becoming just another memory of that legacy stack." @mikeal

That's the very first time that I see such a humble position. It's truly clever. And while it's going to require 40K+ packages to be upgraded, that's totally worth it.

What do you think?

How to send data to a Google spreadsheet, in 2 seconds?

Corentin — Fri, 13 Sep 2019 09:00:26 +0000

Note: The "Push to GSheet" service used in this article is no longer active, but you can use Pizzly to host your own version.

I'm a huge fan of Google APIs. I use a lot of them on a daily basis. Whether in my code or through the infinite number of services that rely on it.

Recently, I was running a scraping script and quickly found myself doing some horrible copy/pasting into a Google Sheet. I thought that there should be a better way to dynamically push data into a spreadsheet:

Having a quick look at the Google Sheets API reference, I found an /append endpoint that would do the work nicely. 30 minutes later, I was sipping lemonade while my script was running in the background 😎

If I wasn't in love with Google APIs, I'd certainly have moved on to something else. But wait, 20 minutes to send data to a spreadsheet? Come on! Let's try to make it as simple as searching on Google.

What do we need?

First, understanding the API. As said just before, Google Sheets provides a handy endpoint /append to push any type of data. All that it asks for, are values formatted as a two-dimensional array:



const values = [
  // First row
  ["A1","A2","A3"],
  // Second row
  ["B1","B2","B3"]
  // Third row... 
]

Data will always be appended to the sheet. So if the sheet has some values on row #1, new values will be added on row #2 (and so-on).

Next, handling the authentication. Like most APIs that give access to users' data, the Google Sheets API uses OAuth2. It's an authentication schema that is both very powerful and... complicated to setup.

What's interesting for us, is that each request to the Google Sheets API requires an access_token - a long string, unguessable, that certifies the request is made on behalf of a user.

Google provides a playground to retrieve one easily. But I'm going to use a service called Pizzly that handles the OAuth-dance securely for me and will handle refresh token, something that the playground doesn't (disclaimer: I helped built Pizzly).

Anything else? We are just missing on which spreadsheet the data shall be appended. That's the spreadsheetID as per the documentation. Each spreadsheet has a unique ID that we can find by looking into the URL:



https://docs.google.com/spreadsheets/d/{{spreadsheetId}}/edit#gid=0

Show me the code!

Wrapping it up, it took 10 lines of code to actually send data to any spreadsheet.



// 1. Run npm install node-fetch

// 2. Import an HTTP client

const fetch = require("node-fetch");

// 3. Declare spreadsheet and values to append

const spreadsheetId = 'SPREADSHEET_ID'

const data = [["firstname", "lastname"], ["John", "Doe"]]

// 4. Send data with a POST request

const baseUrl = "https://pushtogsheet.herokuapp.com";

const query = valueInputOption=RAW&amp;pizzly_pkey=pope8Qy8qfYyppnHRMgLMpQ8MuEUKDGeyhfGCj;

const url = new URL(/proxy/google-sheets/spreadsheets/</span><span class="p">${</span><span class="nx">spreadsheetId</span><span class="p">}</span><span class="s2">/values/A1:append?</span><span class="p">${</span><span class="nx">query</span><span class="p">}</span><span class="s2">, baseUrl);

fetch(url.href, {

    method: "POST",

    body: JSON.stringify({ values: data }),

    headers: { 'Pizzly-Auth-Id': 'CONNECT_FIRST' }

  })

  .then((res) => res.text())

  .then(console.log)

  .catch(console.error);

How can I use it?

~~I did my best to make pushing data to gsheet as easy as googling, and crafted a dedicated website for that:~~

The "Push to GSheet" service used in this article is no longer active, but you can use Pizzly to host your own version.

PS: it's 100% free and open source ✌️

How to create a bunch of Google Alerts, in 3 minutes?

Corentin — Tue, 27 Aug 2019 16:47:14 +0000

Google Alerts is a very handy and powerful service to stay informed of what's happening in a particular field.

I'm a huge fan of this service and I create dozens of new alerts every week. Recently, I wanted to add 100+ Google Alerts to being informed about APIs that we're working with at Bearer.

Unfortunately, Google Alerts doesn't provide an API. And I didn't felt ok to share my Google credentials (email + password) to the available libraries.

What do we need?

First, a list of keywords that we want to add alerts on:

// Here's my list of keywords to add Google Alerts on;
// Change it with whatever you want to be informed of.
const keywords = ["GitHub API", "Google Alerts API", "Dev.to API"]

Then, head to Google Alerts so we will learn how it works behind the scene:

I'm using Google Chrome, but that should work just fine with Safari or Firefox.

Create a Google Alert with JS

On Google Alerts homepage, open the developer tools Alt+Command+J (on Mac) or Ctrl+Shit+J (on Windows), then open the Network tab. You should see something like:

Now create a sample alert using dev.to as the keyword. The network tab will show a request to the /create endpoint. Use Copy as fetch to see what's inside that request:

We are almost done 🙌 If you paste that into the console, you will have something like:

// Code has been prettified
fetch(
  "https://www.google.com/alerts/create?x=ABJHsmWAbcU-xxxxxxxxxxxxxxxxxxxxx&hl=us",
  {
    "credentials": "include",
    "headers": {
    "accept": "*/*",
    "accept-language": "fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7",
    "cache-control": "no-cache",
    "content-type": "application/x-www-form-urlencoded;charset=UTF-8",
    "pragma": "no-cache",
    "sec-fetch-mode": "cors",
    "sec-fetch-site": "same-origin",
    "x-client-data": "xxxxxxxxxxxxxxxxxxxxx"
    },
    "referrer": "https://www.google.com/alerts?hl=us",
    "referrerPolicy": "no-referrer-when-downgrade",
    // The dev.to keyword is passed ==================== right here ∨∨∨
    "body": "params=%5Bnull%2C%5Bnull%2Cnull%2Cnull%2C%5Bnull%2C%22dev.to%22%2C%22com%22%2C%5Bnull%2C%22en%22%2C%22US%22%5D%2Cnull%2Cnull%2Cnull%2C0%2C1%5D%2Cnull%2C3%2C%5B%5Bnull%2C1%2C%22corentin%40bearer.sh%22%2C%5Bnull%2Cnull%2C10%5D%2C2%2C%22en-US%22%2Cnull%2Cnull%2Cnull%2Cnull%2Cnull%2C%220%22%2Cnull%2Cnull%2C%22AB2xxxxxxxxxxx%22%5D%5D%5D%5D",
    "method": "POST",
    "mode": "cors"
  }
);

As you might see, the dev.to keyword is passed into the body. Changing it to something else, will let us automatically add a new Google Alert 🥳

A script that creates Google Alerts in bulk

// Replace with your keywords list
var keywords = ["GitHub API", "Google Alerts API", "Dev.to API"]

function addAlert(i) {
  // Retrieve the keyword to work with
  const keyword = encodeURIComponent(keywords[i])

  // Stop the script if there's no keyword
  if (!keywords[i] || !keyword) { return; }

  console.log(`Adding ${keyword}`)

  // 1. Replace the line below with your own fetch (see Copy as fetch above)
  // 2. Replace `dev.to` with `${keyword}`
  fetch(/*...*/)

  // Exponentially delay the next request,
  // to avoid rate limit on Google.
  .then(() => { setTimeout(function() {addAlert(i+1)}, (Math.min(i || 2, 30) * 1000)) })
}

addAlert(0)

I recommend adding it as a snippet on your Google Chrome (learn how to do it right here).