Workdrop — Setting Up Serverless

This continues my entry for #twiliohackathon!

GitHub Repository (MIT Licensed)
Stackshare

---

Before we get going with the frontend, I'll describe a summary of some of the serverless setup with MongoDB Stitch, AWS, and Twilio SendGrid.

MongoDB

I already discussed setting up a MongoDB cluster in the second entry in this series, but now it's time to get cooking with serverless.

Functions

MongoDB Stitch, of course, has serverless functions built-in, and makes Atlas database interactions a breeze.

Requests

The architecture is pretty simple: users enter their assignment name, student emails, a personalized message, and their email. Then, it sends all that data to a MongoDB function.

There are two collections in the database: requests and submissions.

Before the user's data is put in the requests collection, two UUID tokens are generated by a separate function:

// generateToken()

const uuid = require("uuid");

const token = uuid.v4('workdrop.app', uuid.v5.DNS);
const accessor = uuid.v4('workdrop.app', uuid.v5.DNS);

return { token, accessor };

The first token will be sent out to the requester and each student. The second accessor token will only be sent out to the requester, and will allow them to access a page where they can download the submissions.

Those tokens are inserted along with the user data in a new document.

// createRequest
const { token, accessor } = context.functions.execute("generateToken");

const requestsCollection = context.services.get("my-cluster").db("workdrop").collection("requests");

const newRequest = {
  token, accessor, name, email, request_emails, message
};

requestsCollection.insertOne(newRequest)
  .then(result => console.log(`Successfully inserted request with _id: ${result.insertedId}`))
  .catch(err => console.error(`Failed to insert request: ${err}`));

After this document is inserted, a trigger calls another function which uses the Twilio SendGrid API to send out the emails.

Submissions

For submissions, the client first connects to MongoDB and then, through the MongoDB API, connects to the AWS S3 bucket (details below). The submitted file is uploaded to AWS, and the returned URL is sent to MongoDB through a function virtually identical to the request function.

AWS

Setting up AWS can be pretty complicated sometimes, especially involving authentication. This was my first time setting up an S3 bucket and permissions.

First, you need to create your account. This will create a root user, which has access to everything in AWS. You should quickly enable multi-factor authentication and have a strong and secure password for your root user.

Your serverless app should never (and I'm not even sure if it can) authenticate with AWS via the root user. Instead, you create a special IAM user where you specify your exact permissions. API keys and secrets are then generated specifically for this user.

For myself, I simply created a new policy called workdrop-stitch and added the S3 GetObject and PutObject permissions.

After setting up AWS via the management console, I went back over to MongoDB and setup a new Third-Party Service, selecting AWS.

Then, you specify your API key and you're off to go!

Twilio SendGrid

Here's the somewhat critical part of my entry: Twilio API integration. For my project, the Twilio service I'm using is Twilio SendGrid.

Even if using a Twilio API wasn't a part of the hackathon requirements, I'm certain I would be still using SendGrid because of how fantastic its API is.

Sender authentication

In order to properly ensure that emails sent out with SendGrid are verified with all email clients and services, you must set up some DNS records to validate your domain with SendGrid. Then, you can have the emails sent out from your app look like [e-mail] instead of using SendGrid's default [e-mail].

Setup is straightforward and simple, especially if you're using CloudFlare DNS as a proxy or Netlify's DNS. All that's required is creating a few specific CNAME records that point to sendgrid.net. Instructions specific to your user are in your SendGrid dashboard.

A note on SendGrid with SSL

If you have to have SSL security on all traffic through your domain, keep in mind that if you're using Click Tracking with SendGrid (which I do have disabled for my app at the moment), you'll have to direct your traffic through a seperate proxy secured with a certificate for your domain. This is because the CNAME record used to point the traffic from your domain to SendGrid's tracking servers is not HTTPS secured, so most browsers will simply refuse to connect to the link if HTTPS is enforced on your domain. And keep in mind, .app and .dev domains are SSL-enforced.

Using the API

For JavaScript, you can either use the SendGrid Node.JS API or manually call the API via REST.

Here's an example of my app's REST API call using JavaScript and MongoDB Stitch:

// sendRequestEmails()

// context.http.post is provided by MongoDB Stitch functions

context.http.post({ 
    "url": "https://api.sendgrid.com/v3/mail/send",
    "headers": { Authorization: [`Bearer [API_KEY]`] },
    "body": {
        "personalizations": [{
          "to": [{
                "email": "[e-mail]"
      }],
    }],
        "from": {
          "email": "[e-mail]"
        }
    },
    "encodeBodyAsJSON": "true"
)
  .then(res => {
      console.log(`EMAIL SUCCESS [${res.statusCode}]`);
  })
  .catch(res => {
      console.log(`EMAIL ERROR [${res.statusCode}]`);
  });

AWS S3 Policy Setup

Here's the part of the project that was the most frustrating for me.

AWS policies can be very difficult to set up, especially for someone like me who never has done it before. And any information on specific policies can be difficult to find on forums and other sites.

User management

First, after you create your AWS root account, you must create an IAM user that your app will use to connect to your services.

When creating your IAM user, you must enable programmatic access since this will only be accessed through an API and not the AWS console.

S3 Policy

You can attach an arbitrary number of policies to any IAM user to enable it to perform specific actions across AWS services.

For S3, there are two primary policies that we need to enable: GetObject and PutObject.

In addition, I found that I had to enable GetObjectAcl and PutObjectAcl.

After enabling these policies, you simply select the bucket resource from your account in the Resources field pictured above.

Bucket setup

After creating these policies, you have to manage a couple permissions for your S3 bucket.

In the Permissions tab of your bucket, you need to enable all block public accecss settings except Block public access to buckets and objects granted through new access control lists (ACLs). This involves the GetObjectAcl and PutObjectAcl permissions we had to set up earlier so authenticated APIs can access our bucket properly.

Next, in the Bucket Policy tab, you need to insert an ARN policy to control access from users. Anything within brackets [] below will need to be replaced with your information. AWS has a policy generator that can help.

{
    "Version": "2012-10-17",
    "Id": "[policy ID]",
    "Statement": [
        {
            "Sid": "[Sid]",
            "Effect": "Allow",
            "Principal": {
                "AWS": "[Your AWS user ARN]"
            },
            "Action": [
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "[Your S3 bucket resource ID]"
        }
    ]
}

Lastly, in the CORS configuration tab, you need to put the following XML. This allows any cross-origin request to be authenticated with the S3 API.

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>HEAD</AllowedMethod>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

---

Now onto the frontend. Thank you for reading! God bless!