DEV Community: Victoria Slocum

Synsong: A prompt-based playlist generator

Victoria Slocum — Thu, 16 Dec 2021 23:57:30 +0000

Super excited to announce the beginnings of Synsong!

TLDR:

I just somewhat finished the beta of a project (synsong.app) to practice NLP, Python, and API skills. Right now, you can input genre, popularity, and a natural language prompt (such as a sentence, quote, or paragraph), and it will create a playlist based on matches with the lyrics, song title, and artist name with the prompt. Eventual goals are to practice model training to auto-detect genres, do some topic modeling with larger prompts, and maybe even have an option for mood.

The Beginnings

The beginnings of synsong have been super fun. It started with the idea, to make a playlist from a sentence, and ended with a whole bunch of other playlist customizations, such as genre and popularity. I made the backend first, and then also created the website.

The Backend

The backend is written in Python, and uses spaCy for all the NLP. I started out by getting the title, spaCy gets of the noun phrases through looking at dependencies (nsubj or pobj) or heads (lemma_ == "be"). The constituents that are searched in the lyrics are found using spaCy's .noun_chunks function. For both the title and the constituents, a random sample is taken (1 for the title and 5 for the constituents).

The constituents are then put in a series of lists with differing amounts of items to ensure that relavant but enough results come up.

# prompt: Potatoes are great food for worms and bugs in the ground.

[['potato', 'great food', 'worm', 'bug', 'ground'], ['great food', 'bug', 'ground'], ['worm', 'bug', 'ground'], ['potato', 'great food', 'worm', 'bug'], ['potato', 'worm', 'bug', 'ground'], ['great food', 'worm', 'bug', 'ground'], ['potato'], ['great food'], ['worm'], ['bug'], ['ground']]

The lists are then formatted and searched through the Musixmatch database with the inputted genre and popularity filters. If there's more than 30 songs, the code will just take the first 30, which are probably the most relevant ones due to the ordering of the search.

I then used Spotipy to link the user's Spotify account and make a playlist on their behalf.

That's basically it! You can find all the code in the GitHub.

The Frontend

The website is powered by Flask, a Python web framework, and Bulma, a CSS framework, and then deployed with Railway.

Demo

Synsong starts on the home page, where it will prompt you to “get started” to log in with your Spotify or “go to app” if you’ve already logged in.

You’ll then go to the generator page...

And then you can input your prompt, genres, and other settings!

You’ll get a playlist (it takes awhile to load though) on the page after, and you can also see it in your Spotify account.

And that’s that!!

Future plans and known issues

I would love to have a whole other aspect of machine learning so I can work on those skills, and one way I’ve thought about doing that is through genre detection. I’d also love to try my hand at topic modeling for larger prompts.

Right now, the popularity filter isn’t too tuned and I’m at a bit of a risk for making too many API requests with Musixmatch’s free plan. Also the footer isn’t stuck to the bottom. Besides that, let me know if you find any other issues!

Links

Thanks for reading! I’m Victoria (victoriaslocum.com), and you can find this projects website at synsong.app and the GitHub here.

JavaScript: A Notion/Slack Integration for Standups

Victoria Slocum — Tue, 14 Sep 2021 18:06:40 +0000

How I made a Notion/Slack integration for Standups

Part 2: JavaScript, because Zapier is expensive 😢

Background:

One of our favorite channels in Slack is our #standup channel, where we post short updates when we finish a task, have a good meeting, or just have something to share about work. Its great to see what people are up to across departments and get updates in a central place.

We originally started doing standups in Notion through a database, but staying up to date with the page was difficult when the majority of our short-term communication happened through Slack. Eventually, our Notion page retired, and we moved to a purely Slack standup.

In part one of this post, I made a Notion and Slack integration for this standups channel using Zapier. Unfortunately, Zapier is expensive and the integration we made wasn't worth paying the money for. Fortunately, I am learning code and figured it would be the perfect project to take on.

I'm extremely happy with the way this turned out. I was able to create a cleaner, smoother interaction than the one I made with Zapier. It did take me awhile to code, but only due to minor complications and lack of experience. As always, I learned a ton, and am excited to share the process with you.

You can find the GitHub repository here!

The Process

Step 1: Setting up

There are three main things to set up the app:

set up a Slack app with your workspace and initialize Bolt
create a Notion integration using their APIs
set up files
get a list of Slack user IDs and Notion user IDs
get the Slack to Notion translator

1. Setting up the Slack Bolt App

I would recommend following this tutorial if you get lost, but I'll also walk you through to help you get started with a Slack Bolt app.

Tokens and installing apps:

After you create an app, you'll need bot and app-level tokens with the following scopes. App-level tokens are found under the "Basic Information" tab in the side menu and bot tokens can be found under "OAuth & Permissions".

You'll also need to enable Socket mode and subscribe to the message.channels event.

2. Setting up the Notion API

Go ahead and follow this guide to set up a new Notion API integration with your standups page (steps 1 and 2). If you don't already have a Notion page, you can make one with our template. If you do have one, make sure it has the following properties with the correct type: Person (person), created (date created), tags (multi-select), link to Slack (text), TS (text).

Feel free to change the names, but just make sure you change it in the code too.

3. Setting up the files

You can go ahead and initialize a folder for package.json and your app. I also put all my tokens into a .env folder and then added .env and node-modules to .gitignore so it wouldn't be published to my public GitHub repository.

mkdir my-standup-integration
cd my-standup-integration
npm init

// add these to .env
NOTION_KEY=secret_
NOTION_DATABASE_ID=

SLACK_BOT_TOKEN=xoxb-
SLACK_SIGNING_SECRET=
SLACK_APP_TOKEN=xapp-

// add this to .gitignore
.env
node_modules
node_modules

In package.json:

{
    "name": "notion-slack-integration",
    "type": "module",
    "version": "1.0.0",
    "description": "",
    "main": "app.js",
    "scripts": {
        "start": "node app.js",
        "dev": "nodemon -r dotenv/config app.js"
    },
    "dependencies": {
        "@notionhq/client": "^0.1.9",
        "@slack/bolt": "^3.6.0",
        "dotenv": "^10.0.0",
        "he": "^1.2.0"
    }
}

Once you have all of those dependencies in your package.json , you can run npm install in the terminal to download the necessary packages.

In app.js:

// Require the Bolt package (github.com/slackapi/bolt)
import pkg from "@slack/bolt";
const { App } = pkg;

// create variables for Slack Bot, App, and User tokens
const token = process.env.SLACK_BOT_TOKEN;
const appToken = process.env.SLACK_APP_TOKEN;

// create Slack app
const app = new App({
  token: token,
  appToken: appToken,
  signingSecret: process.env.SLACK_SIGNING_SECRET,
  socketMode: true,
});

// create Notion client
import { Client } from "@notionhq/client";
const notion = new Client({ auth: process.env.NOTION_KEY });

// create variable for Notion database ID
const databaseId = process.env.NOTION_DATABASE_ID;

4. Getting a dictionary for Slack IDs to Notion IDs

You can find the tutorial for doing this here and the end result should look like this. Go ahead and add it to your app.js.

// Slack user ID to Notion user ID dictionary
const slackNotionId = {
  UT9G67J1Z: "f2ca3fc5-9ca1-46ed-be8b-fb618c56558a",
  U0185FAF1T5: "6718f0c7-f6e3-4c3a-9f65-e8344806b5b6",
  U025P5K0S0Z: "6f7ce62c-fa2e-4440-8805-72af5f937666",
  U021UR4DW5C: "8fd7689c-d795-4ae9-aa53-5846ac1569b7",
  U0224KFNYRW: "7c02e0ba-2aec-4696-a91d-ecaa01b616ce",
  U025J9SLXV3: "94f6b8b7-e8b0-4790-8265-f08e6b1d550c",
  UT9G67YFM: "6c3a6ec1-4b99-4e5c-8214-cea14fd9b142",
};

5. Set up the Slack to Notion translator

You can find the GitHub here and the blog post here for the code.

Great! Now we're set up and we can move onto the functions.

Step 2: The Functions

There are 10 different functions that all play a role in making this app happen. Lets go through them.

1. Finding the Slack channel

This function allows us to filter out messages from any other channel by getting the conversation ID. Its an async function, and the Slack request uses the appToken. We check to see if the channel name matches the inputted name, and from that we can filter out the ID.

Outside of the function, we can make a variable for the ID to our channel, which we will use many times in other functions.

// find Slack channel
async function findConversation(name) {
  try {
    var conversationId = "";

    // get a list of conversations
    const result = await app.client.conversations.list({
      // app token
      appToken: appToken,
    });

    // check if channel name == input name
    for (const channel of result.channels) {
      if (channel.name === name) {
        conversationId = channel.id;
        break;
      }
    }

    // return found ID
    return conversationId;
  } catch (error) {
    console.error(error);
  }
}

// variable for slack channel
const standupId = await findConversation("standup");

2. Adding a page to a Notion database

This function will allow us to add a page to the Notion database. The function takes in a title, body text, Slack user ID (which is then converted using the table defined above), a timestamp, tags, and a link to the Slack message. These inputs are properly formatted and then pushed as a page when the function is called. The function returns the URL of the notion page to be used later.

// add item to Notion database
async function addItem(title, text, userId, ts, tags, link) {
  try {
    // add tags with proper format
    const tagArray = [];
    for (const tag of tags) {
      tagArray.push({ name: tag });
    }

    // create page with correct properties and child using initialNotionItem function
    const response = await notion.pages.create({
      parent: { database_id: databaseId },
      properties: {
        Name: {
          type: "title",
          title: [
            {
              type: "text",
              text: {
                content: title,
              },
            },
          ],
        },
        Person: {
          type: "people",
          people: [
            {
              object: "user",
              id: slackNotionId[userId],
            },
          ],
        },
        TS: {
          type: "rich_text",
          rich_text: [
            {
              type: "text",
              text: {
                content: ts,
              },
            },
          ],
        },
        Tags: {
          type: "multi_select",
          multi_select: tagArray,
        },
        "Link to Slack": {
          type: "rich_text",
          rich_text: [
            {
              type: "text",
              text: {
                content: link,
              },
            },
          ],
        },
      },

      children: newNotionItem(text),
    });

    console.log(response);

    // return the url to be put in thread
    return response.url;
  } catch (error) {
    console.error(error);
  }
}

3. Finding a database item (based on a Slack message)

Remember that weird TS property in the Notion pages? This is how we identify what pages match the Slack message sent so we can append a thread message to the body of the Notion page. The function takes in the Slack message's thread_ts value so it can match it to a Notion property using a filter.

The function will return an ID of the page.

// find database item based on the threadts value from Slack and property from Notion
async function findDatabaseItem(threadTs) {
  try {
    // find Notion items with the correct threadts property
    const response = await notion.databases.query({
      database_id: databaseId,
      filter: {
        property: "TS",
        text: {
          contains: threadTs,
        },
      },
    });

    // return the ID of the page
    return response.results[0].id;
  } catch (error) {
    console.error(error);
  }
}

4. Append text to an existing Notion page

The newNotionItem() function given by the Slack-Notion translator allows us to have a properly formatted body by just inputting some text and the Slack user ID of the author. The block_id is actually just the Notion page ID, which we found using the last function.

// append a body to a Notion page
async function addBody(id, text, userId) {
  try {
    // use ID of page and newNotionItem function for formatting
    const response = await notion.blocks.children.append({
      block_id: id,
      children: newNotionItem(text, userId),
    });
  } catch (error) {
    console.error(error);
  }
}

5. Setting the channel topic with the existing list of tags

We found it helpful to be able to easily access the current list of tags in the database through the channel topic. This function will make an easy-to-read list of tags and only update the channel topic when a new tag has been added.

// make the list of tags for the channel topic
async function setChannelTopic(currentTag) {
  try {
    // get database and then list of tags in database
    const response = await notion.databases.retrieve({
      database_id: databaseId,
    });
    const tags = response.properties.Tags.multi_select.options;

    // make a list of the current tags in the database
    var topic = "Current tags are: ";
    tags.forEach((tag) => {
      topic += tag.name + ", ";
    });

    // set variable for reset channel topic
    var restart = false;

    // for each tag in list of tags presented in the Slack message
    currentTag.forEach((tag) => {
      // if the tag is not found add to list and set restart to true
      if (topic.search(tag) == -1) {
        topic += tag + ", ";
        restart = true;
      }
    });

    // get rid of last ", "
    topic = topic.slice(0, -2);

    // if it should be restarted, set the channel topic again
    if (restart == true) {
      const setTopic = await app.client.conversations.setTopic({
        token: token,
        channel: standupId,
        topic: topic,
      });
    }
  } catch (error) {
    console.error(error);
  }
}

6. Reply to the Slack message with the Notion link in thread

We also found it helpful for the Bot to reply to the Slack message with a link to the created Notion page in the thread. This function takes in the channel ID, thread TS of the message, and the link to the Notion page and then replies to the message when called.

// reply to the Slack message with the Notion link
async function replyMessage(id, ts, link) {
  try {
    const result = await app.client.chat.postMessage({
      // bot token
      token: token,
      channel: id,
      thread_ts: ts,
      text: link,
    });
    return;
  } catch (error) {
    console.error(error);
  }
}

7. Find the name of a user (instead of their ID)

For titles, it's necessary to find the name of a user, because you can't tag in a title and you don't want a weird ID to show up. This function takes in a user ID and outputs their display name.

// find the Slack username of the user using the Slack ID
async function findUserName(user) {
  try {
    const result = await app.client.users.profile.get({
      // bot token and Slack user ID
      token: token,
      user: user,
    });
    return result.profile.display_name;
  } catch (error) {
    console.error(error);
  }
}

8. Get the tags from the message

This was definitely one of the most difficult parts of this whole process. This function takes in text, looks for "tags: " in the text, and then returns an array of tags from that.

The first thing the function is doing is retrieving the current list of tags in the database. Then, it creates an array of the tags within the Notion database. Next, the function looks for a tag line in the item and splits that into individual items in an array.

For each of the tags it found in the Slack message, it compares them to the tags already found in the database. If there is that tag in the database, it sends the database tag to a new array in order to match capitalization. If the function doesn't find the new tag in the already existing database, it will create a new tag and put that into the array.

This function returns an array of tags.

// find the tags in the Slack message
async function findTags(text) {
  try {
    // get database and then list of tags in database
    const response = await notion.databases.retrieve({
      database_id: databaseId,
    });
    const databaseTags = response.properties.Tags.multi_select.options;

    // make a list of the current tags in the database
    var dbTagArray = [];
    databaseTags.forEach((dbtag) => {
      dbTagArray.push(dbtag.name);
    });

    var tags = [];
    // search for Tags indicator
    var index = text.toLowerCase().search("tags: ");

    // if found
    if (index != -1) {
      // bypass "tags: "
      index += 6;
      // make a list by slicing from index to end and split on first line
      const tagList = text.slice(index, text.length).split("\n")[0];

      // make array of tags based on the split value
      var slackTagArray = tagList.split(", ");

      // for each found Slack tag
      slackTagArray.forEach((stag) => {
        // set counter
        var index = 0;
        // for each Notion database tag
        dbTagArray.forEach((dbtag) => {
          if (stag.toLowerCase() == dbtag.toLowerCase()) {
            // if the tags match, push the database tag
            tags.push(dbtag);
          } else {
            // if they don't, count
            index += 1;
          }

          // if it went through all of the database items, push the Slack tag
          if (index == dbTagArray.length) {
            tags.push(stag);
          }
        });
      });
    }

    // return array of tags
    return tags;
  } catch (error) {
    console.error(error);
  }
}

9. Make the title!

Another difficult function, it takes in the text and splits in in various ways, eliminating links and users along the way.

First, we see if there's a line split for the title and replace the emojis. Then, we'll search to see if there are any links. If there are, we will split them out of their Slack formatting and just keep the text portion. Then, if there are any users and it finds it in the user dictionary we made, it will replace that tagged user with their name. Finally, it will replace tagged channel or here with a better-formatted version.

With whatever is left, it will split based on any punctuation marks and limit the character count, and return the completed title.

// create the title for the Notion page
async function makeTitle(text) {
  // split based off of line break or emphasis punctuation
  var title = text.split(/[\n]/)[0];

  // replace the emojis
  title = replaceEmojis(title);

  // search for links
  if (title.search("http") != -1 || title.search("mailto") != -1) {
    // split title based on link indicators <link>
    var regex = new RegExp(/[\<\>]/);
    var split = title.split(regex);

    // initialize title
    title = "";

    // for each line in the split text
    split.forEach((line) => {
      if (line.search("http") != -1 || line.search("mailto") != -1) {
        // if it is the link item, split the first half off and only push the text to title
        let lineSplit = line.split("|");
        title += lineSplit[1];
      } else {
        // if it isn't, push the text to title
        title += line;
      }
    });
  }

  if (title.search("@") != -1) {
    console.log(title)
    var split = title.split(" ");

    console.log(split)
    // find all instances of users and then replace in title with their Slack user name
    // wait til this promise is completed before moving on
    await Promise.all(
      split.map(async (word) => {
        if (word.search("@") != -1) {
          const userId = word.replace("@", "");
          if (userId in slackNotionId) {
            var userName = await findUserName(userId);
            title = title.replace(word, userName);
          }
        }
      })
    );
  }

  // replace weird slack formatting with more understandable stuff
  if (title.search("!channel") != -1 || title.search("!here") != -1) {
    title = title.replace("<!channel>", "@channel");
    title = title.replace("<!here>", "@here");
  }

  // split the title based on "." and "!"
  // (can't do above because links have "." and "?" and @channel has "!")
  // and return the first item
  title = title.split(/[\.\!\?]/)[0];
  // make sure its not too long
  title = title.slice(0, 100);
  return title;
}

10. Add tags to an already established page

If you reply in thread with tags in the proper format, it will update the Notion item with the new tags you have provided without getting rid of the old tags that were already there.

The function takes in an array of tags (created by the findTags() function) and properly formats them. Then, it combines an array of the tags that already exist and the new tags and updates the Notion item with that.

// append more tags to an already existing page
async function addTags(pageId, tags) {
  try {
    // add tags with proper format
    const tagArray = [];
    for (const tag of tags) {
      tagArray.push({ name: tag });
    }

    // get already existing tags
    const page = await notion.pages.retrieve({ page_id: pageId });
    var oldTags = page.properties.Tags.multi_select;

    // create conjoined array
    var newTags = oldTags.concat(tagArray);

    // update the Notion page with the tags
    const response = await notion.pages.update({
      page_id: pageId,
      properties: {
        Tags: {
          name: "Tags",
          type: "multi_select",
          multi_select: newTags,
        },
      },
    });
  } catch (error) {
    console.error(error);
  }
}

Step 3: In event of a message...

Yay! We've set up our functions. Now its time to tell the app what happens when someone sends a message, and make sure its picking up on the right channel.

// if a message is posted
app.event("message", async ({ event, client }) => {
  console.log(event);

  // make sure its the right channel
  if (event.channel == standupId) {

        // more stuff to come here
    }
}

Next we have to get the tags, title, and link to Slack message. Tags and title are functions, and then we can just use the .getPermalink call and get the link.

// get the tags
var tags = await findTags(event.text);

// get the title
const title = await makeTitle(event.text);

// get the link to the Slack message
const slackLink = await app.client.chat.getPermalink({
  token: token,
  channel: event.channel,
  message_ts: event.ts,
});

Next we're going to see if its a thread message or a parent message. Thread messages will have the property thread_ts that matches the parent ts .

1) If its a thread message:

First, we have to find the database item and get the Notion page ID. Then, we can append a body to that Notion page. If there are tags in the tag array, then we can add those tags too.

2) If its a parent message:

We'll first set the channel topic if there are any new tags, and then create a Notion item and take that returned link as the variable notionUrl. Finally, we'll reply in thread with the Notion page link.

try {
  if ("thread_ts" in event) {
    // if its a thread message, find the original Notion page and then append the Slack message
    const pageId = await findDatabaseItem(event.thread_ts);
    addBody(pageId, event.text, event.user);
    if (tags.length != 0) {
      addTags(pageId, tags);
    }
  } else {
    // if its a parent message
    // make the list of tags for the channel topic and push it if applicable
    await setChannelTopic(tags);
    // make the Notion page and push to database
    const notionUrl = await addItem(
      title,
      event.text,
      event.user,
      event.ts,
      tags,
      slackLink.permalink
    );

    // reply with the link returned by addItem
    await replyMessage(standupId, event.ts, notionUrl);
  }
} catch (error) {
  console.error(error);
}

Step 4: Start

All that's left is to start our app! Now it will detect a message and add the proper Notion item.

(async () => {
  // Start your app
  await app.start(process.env.PORT || 3000);
  console.log("⚡️ Bolt app is running!");
})();

Results

Here's the resulting flow:

New message posted in Slack, bot replies with link

Channel topic is set with the new tag

The Notion page is created!!

Conclusion

I loved doing this project and working with Slack and Notion's APIs. This turned out so much better than Zapier, which was super rewarding.

Links:

GitHub: https://github.com/victoriaslocum752/standup-integration

Website: https://victoriaslocum.com

Twitter: https://twitter.com/VictoriaSlocum3

Hope to see you around again soon! 👋

Creating a Slack to Notion translator

Victoria Slocum — Fri, 10 Sep 2021 18:20:03 +0000

How I created a Slack to Notion translator

TLDR:

Notion has a very interesting way of setting up their pages, formatting their text, and creating items, and that happens to be very different than Slack's approach.

By taking examples of text from both APIs, I was able to set up a translator between Slack and Notion. This blog post walks you through how I did it, but you can also just check out the project on GitHub.

(Note: this post doesn't walk you through how the APIs work, but you can find out how I did that in this project (coming soon))

Translating Slack to Notion

So far, the code only translates Slack to Notion, but hopefully sometime soon it will be able to translate Notion to Slack messages. For now, I'll walk you through how I set it up.

For testing, we're going to be using this example message from Slack. There is various text formatting in the item, like line breaks, links, tagged users, emojis 🐿️, code, bold, italic, and bullet points. The only thing Notion does inherently is the bullet points and numbered lists.

// example message from Slack
const slackExample =
  'Hi this is a message with:\n' +
  '\n' +
  '• *bold*, _italic_, and `code` , along with <http://endless.horse/|links> and emojis :potato: :shrimp: :wave: \n' +
  '• and tagged users like HEY <@U0185FAF1T5> ';

Notion items work in blocks, so here's that same message in Notion with the json object. The main blocks are split by line breaks and within that the arrays are based on the text type.

[ { type: 'text', text: { content: 'Hi this is a message with:' } } ]
[
  { type: 'text', text: { content: '• ' } },
  {
    type: 'text',
    text: { content: 'bold' },
    annotations: { bold: true }
  },
  { type: 'text', text: { content: ', ' } },
  {
    type: 'text',
    text: { content: 'italic' },
    annotations: { italic: true }
  },
  { type: 'text', text: { content: ', and ' } },
  {
    type: 'text',
    text: { content: 'code' },
    annotations: { code: true }
  },
  { type: 'text', text: { content: ' , along with ' } },
  { type: 'text', text: { content: 'links', link: [Object] } },
  { type: 'text', text: { content: ' and emojis 🥔 🦐 👋 ' } }
]
[
  { type: 'text', text: { content: '• and tagged users like HEY ' } },
  { type: 'mention', mention: { type: 'user', user: [Object] } },
  { type: 'text', text: { content: ' ' } }
]

Step 1: Set up

In your main folder, initialize a package.json with the following dependencies and an main.js . Then go ahead and npm install in the terminal.

{
    "name": "slack-notion-translation",
    "type": "module",
    "version": "1.0.0",
    "description": "",
    "main": "main.js",
    "scripts": {
        "start": "node main.js",
        "dev": "nodemon main.js"
    },
    "dependencies": {
        "he": "^1.2.0"
    }
}

If you want tags for people to work in Notion, the first thing you're going to need is a Slack ID to Notion ID dictionary. To figure out how to do that, you can go to this post. Your table should look like this, with the Slack ID as the key and the Notion ID as the value.

// Slack user ID to Notion user ID dictionary
const slackNotionId = {
  UT9G67J1Z: "f2ca3fc5-9ca1-46ed-be8b-fb618c56558a",
  U0185FAF1T5: "6718f0c7-f6e3-4c3a-9f65-e8344806b5b6",
  U025P5K0S0Z: "6f7ce62c-fa2e-4440-8805-72af5f937666",
  U021UR4DW5C: "8fd7689c-d795-4ae9-aa53-5846ac1569b7",
  U0224KFNYRW: "7c02e0ba-2aec-4696-a91d-ecaa01b616ce",
  U025J9SLXV3: "94f6b8b7-e8b0-4790-8265-f08e6b1d550c",
  UT9G67YFM: "6c3a6ec1-4b99-4e5c-8214-cea14fd9b142",
};

The next thing we need to do is import he in order for us to change HTML emoji codes into the actual emoji item and import fs so we can read other files.

import he from "he";
import fs from "fs";

Next we need to set up the files for the emoji dictionary. You can find the dictionary I used here, and I downloaded that file into my main directory. This will allow us to translate the Slack emojis to HTML.

// import slack to html emoji dictionary
let rawdata = fs.readFileSync("./slack_emoticons_to_html_unicode.json");
let emojis = JSON.parse(rawdata);

Great! Now we're set up, and we can move on to the translation functions.

Step 2: Convert a parsed Slack item to Notion

These functions will allow for text of a singular type to be translated into a Notion item. For example, Notion recognizes code as a separate string than regular text, so code has to be extracted and made into its own array. These functions properly format the text type so then we can make a larger Notion item.

Here's the function for translating emojis. By splitting the string by the spaces, we can isolate the emojis, and then detect them through the ":". Once we find an emoji, we can find the HTML value from the Slack key, and he.decode() allows us to decode the translated HTML into the emoji.

// replace the emojis codes (from Slack) in the text with actual emojis
const replaceEmojis = (string) => {
  // split string based on words
  var splitString = string.split(" ");

  // for each word in the string:
  // see if the word has the emoji marker ":"
  // search keys in the emoji for the word
  // replace the word with the decoded html value
  splitString.forEach((word) => {
    if (word.search(":") != -1) {
      for (var key in emojis) {
        if (word.search(":" + key + ":") != -1) {
          string = string.replace(key, he.decode(emojis[key]));

          // replace all the ":" in the string and return
          string = string.replace(/:/gi, "");
        }
      }
    }
  });
  return string;
};

The following items are for the various other types of formatting. In all instances, the function returns the created Notion array.

// create a new Notion block item for links
const newLinkItem = (plainText, link) => {
  var array = {
    type: "text",
    text: {
      content: plainText,
      link: {
        type: "url",
        url: link,
      },
    },
  };
  return array;
};

// create a new Notion block item for text
const newTextItem = (text) => {
  var array = {
    type: "text",
    text: {
      content: text,
    },
  };
  return array;
};

// create a new Notion block item for users
const newUserItem = (slackUserID) => {
  var array = {
    type: "mention",
    mention: {
      // find the user's Notion ID from the Slack ID and the dictionary 
      type: "user",
      user: { id: slackNotionId[slackUserID] },
    },
  };
  return array;
};

// create a new Notion block item for code
const newCodeItem = (codeText) => {
  var array = {
    type: "text",
    text: {
      content: codeText,
    },
    annotations: {
      code: true,
    },
  };
  return array;
};

// create a new Notion block item for bold text
const newBoldItem = (boldText) => {
  var array = {
    type: "text",
    text: {
      content: boldText,
    },
    annotations: {
      bold: true,
    },
  };
  return array;
};

// create a new Notion block item for code text
const newItalicItem = (italicText) => {
  var array = {
    type: "text",
    text: {
      content: italicText,
    },
    annotations: {
      italic: true,
    },
  };
  return array;
};

// create a new Notion block item for strikethrough text
const newStrikeItem = (strikeText) => {
  var array = {
    type: "text",
    text: {
      content: strikeText,
    },
    annotations: {
      strikethrough: true,
    },
  };
  return array;
};

Ok, now that we've gotten that out of the way, the real fun starts.

Step 3: Creating the block child

Notion sets up their line breaks through creating new child blocks. So for each line in the text, we'll have to parse it accordingly to fit in each of the functions described above.

Lets start by creating the function and setting up the main variable. The function takes in a split array based on the regex expression /[\<\>]/, which splits the item in every instance of '<' and '>'. This is to capture the links and tagged user items, which are formatted like <http://endless.horse/|links> and <@UT9G67YFM> respectively.

// create a new child of a page with different blocks
const newChild = (splitItem) => {
    // create the Item
  var notionItem = [];

    // more code to come
}

Next, we'll create a .forEach() for each line in the inputted split array. In this .forEach(), we'll have a few if statements to capture all the different types.

splitItem.forEach((item) => {
    // if statements here
}

Lets start with the links. First, we'll search for the link markers, both email links and webpage links. Then, we'll split based off of the "|" separating the text from the link. This will create an array with the link in the first item and the text in the second item, which then we can create an item with and push that item to the Notion item array.

if ((item.search(/https?/) != -1) | (item.search(/mailto/) != -1)) {
  // see if its a link item by searching for link text indicators

  // split link into text and link
  let linkSplit = item.split("|");

  // create link item and push to notionItem
  const linkItem = newLinkItem(linkSplit[1], linkSplit[0]);
  notionItem.push(linkItem);
}

Our next search will be for users. We can find them through "@", which we'll get rid of. If that item is somewhere in the dictionary of Slack IDs, then we'll continue with the user item. If its not, we'll just make it a text item with the original item text.

else if (item.search("@") != -1) {
  // see if it is a user by searching for the @ symbol

  // replace indicator symbol
  var string = item.replace("@", "");

  // check if the string is in the table, if not just push the string as a text item
  if (string in slackNotionId) {
    // create a new user item and push to notionItem
    const userItem = newUserItem(string, slackNotionId);
    notionItem.push(userItem);
  } else {
    const textItem = newTextItem(item);
    notionItem.push(textItem);
  }
}

This part is a little bit trickier. We've got to search to see if there's any indication of all the other text formatting options and then if there is, split that text and give the correct functions the correct items.

Lets set up the if statement first then go from there.

else if (item.search(/[\`\_\*\~]/) != -1) {
    // if a string contains any special annotations (bold, italic, code, strikethrough)

    // replace any emojis in string
    item = replaceEmojis(item);

    // more stuff to come here

}

Next, regex. The way I did this is kind of weird, but basically I didn't want to get rid of the markers but still wanted to split the text. My solution was to add an "=" before and after the word, so *bold* would turn into =*bold*=. Then, we can split based off all the "=" and not lose the original formatting. If there's a better solution to this, please let me know 😆.

// kinda wack, but replace all the symbols with = on either end
// so it can break without getting rid of the original symbol
item = item.replace(/[\*](?=[a-zA-Z0-9])/, "=*");
item = item.replace(/(?<=[a-zA-Z0-9,])[\*]/, "*=");
item = item.replace(/[\`](?=[a-zA-Z0-9])/, "=`");
item = item.replace(/(?<=[a-zA-Z0-9,])[\``]/, "`=");
item = item.replace(/[\_](?=[a-zA-Z0-9])/, "=_");
item = item.replace(/(?<=[a-zA-Z0-9,])[\_]/, "_=");
item = item.replace(/[\~](?=[a-zA-Z0-9])/, "=~");
item = item.replace(/(?<=[a-zA-Z0-9,])[\~]/, "~=");

// split item based off of =
var split = item.split(/\=/gi);

This will give us an array that is split based on all of the types of text! Next we'll use a series of if statements to see what type it is, and then translate the type and push it to the Notion item.

// for each item, check to see what type it is, replace the indicator, and push to notionItem
split.forEach((split) => {
  if (split.search("`") != -1) {
    split = split.replace(/\`/gi, "");
    const item = newCodeItem(split);
    notionItem.push(item);
  } else if (split.search("_") != -1) {
    split = split.replace(/\_/gi, "");
    const item = newItalicItem(split);
    notionItem.push(item);
  } else if (split.search(/[\*]/) != -1) {
    split = split.replace(/\*/gi, "");
    const item = newBoldItem(split);
    notionItem.push(item);
  } else if (split.search("~") != -1) {
    split = split.replace(/\~/gi, "");
    const item = newStrikeItem(split);
    notionItem.push(item);
  } else {
    const textItem = newTextItem(split);
    notionItem.push(textItem);
  }
});

Ok, that's done, now we can move back to the original if statement with a final else to capturing any remaining text.

else {
  // if the string is normal, then replace emojis and push text item
  var string = replaceEmojis(item);
  const textItem = newTextItem(string);
  notionItem.push(textItem);
}

Then we can just return the Notion item at the end, and tada 🎉! Here's the complete function.

// create a new child of a page with different blocks
const newChild = (splitItem) => {
  // create the Item
  var notionItem = [];

  // the input is a split item based on (/[\<\>]/), and then for each item
  // both links and users are indicated by <text>
  splitItem.forEach((item) => {
    if ((item.search(/https?/) != -1) | (item.search(/mailto/) != -1)) {
      // see if its a link item by searching for link text indicators

      // split link into text and link
      let linkSplit = item.split("|");

      // create link item and push to notionItem
      const linkItem = newLinkItem(linkSplit[1], linkSplit[0]);
      notionItem.push(linkItem);
    } else if (item.search("@") != -1) {
      // see if it is a user by searching for the @ symbol

      // replace indicator symbol
      var string = item.replace("@", "");

      // create a new user item and push to notionItem
      const userItem = newUserItem(string);
      notionItem.push(userItem);
    } else if (item.search(/[\`\_\*\~]/) != -1) {
      // if a string contains any special annotations (bold, italic, code, strikethrough)

      // replace any emojis in string
      item = replaceEmojis(item);

      // kinda wack, but replace all the symbols with = on either end
      // so it can break without getting rid of the original symbol
      item = item.replace(/[\*](?=[a-zA-Z0-9])/, "=*");
      item = item.replace(/(?<=[a-zA-Z0-9,])[\*]/, "*=");
      item = item.replace(/[\`](?=[a-zA-Z0-9])/, "=`");
      item = item.replace(/(?<=[a-zA-Z0-9,])[\``]/, "`=");
      item = item.replace(/[\_](?=[a-zA-Z0-9])/, "=_");
      item = item.replace(/(?<=[a-zA-Z0-9,])[\_]/, "_=");
      item = item.replace(/[\~](?=[a-zA-Z0-9])/, "=~");
      item = item.replace(/(?<=[a-zA-Z0-9,])[\~]/, "~=");

      // split item based off of =
      var split = item.split(/\=/gi);

      // for each item, check to see what type it is, replace the indicator, and push to notionItem
      split.forEach((split) => {
        if (split.search("`") != -1) {
          split = split.replace(/\`/gi, "");
          const item = newCodeItem(split);
          notionItem.push(item);
        } else if (split.search("_") != -1) {
          split = split.replace(/\_/gi, "");
          const item = newItalicItem(split);
          notionItem.push(item);
        } else if (split.search(/[\*]/) != -1) {
          split = split.replace(/\*/gi, "");
          const item = newBoldItem(split);
          notionItem.push(item);
        } else if (split.search("~") != -1) {
          split = split.replace(/\~/gi, "");
          const item = newStrikeItem(split);
          notionItem.push(item);
        } else {
          const textItem = newTextItem(split);
          notionItem.push(textItem);
        }
      });
    } else {
      // if the string is normal, then replace emojis and push text item
      var string = replaceEmojis(item);
      const textItem = newTextItem(string);
      notionItem.push(textItem);
    }
  });
  console.log(notionItem);
  return notionItem;
};

The final function will be creating a Notion item! This will take in a Slack message and convert it to Notion.

const newNotionItem = (slackMessage) => {
    // stuff goes here
}

First, we'll make an empty block if you wanted to include spacing.

// empty block for spacing
  const emptyBlock = {
    object: "block",
    type: "paragraph",
    paragraph: {
      text: [
        {
          type: "text",
          text: {
            content: "",
          },
        },
      ],
    },
  };

Next, we'll make the item before hand, just like the newChild() function, and split the message based on line breaks. The .filter(Boolean) is just to get rid of the empty items in the array.

// notion Item
const notionItem = [];

// split message on line breaks and filter empty lines
var newLineSplit = slackMessage.split("\n");
newLineSplit = newLineSplit.filter(Boolean);

Then, for each line in the Slack message, we will split it based on the link and user indicators of "<>" and make a new child with that split item. We'll make a block from that child text, and push that to the Notion item.

// for each line in Slack message
newLineSplit.forEach((line) => {
  // split line based on link/user indicators
  var regex = new RegExp(/[\<\>]/);
  var split = line.split(regex);

  // create new child item content
  var item = newChild(split);
  // add child item content to formatted block
  const childBlock = {
    object: "block",
    type: "paragraph",
    paragraph: { text: item },
  };

  // push child to notionItem
  notionItem.push(childBlock);
});

Finally, we'll push an empty block and return the Notion item. This is the whole function.

// create a new Notion item
const newNotionItem = (slackMessage) => {
  // empty block for spacing
  const emptyBlock = {
    object: "block",
    type: "paragraph",
    paragraph: {
      text: [
        {
          type: "text",
          text: {
            content: "",
          },
        },
      ],
    },
  };

  // notion Item
  const notionItem = [];

  // split message on line breaks and filter empty lines
  var newLineSplit = slackMessage.split("\n");
  newLineSplit = newLineSplit.filter(Boolean);

  // for each line in Slack message
  newLineSplit.forEach((line) => {
    // split line based on link/user indicators
    var regex = new RegExp(/[\<\>]/);
    var split = line.split(regex);

    // create new child item content
    var item = newChild(split);
    // add child item content to formatted block
    const childBlock = {
      object: "block",
      type: "paragraph",
      paragraph: { text: item },
    };

    // push child to notionItem
    notionItem.push(childBlock);
  });

  // add an empty block for spacing and return
  notionItem.push(emptyBlock);
  console.log(notionItem);
  return notionItem;
};

And that's it! The newNotionItem function will return something that looks like this:

[
  { object: 'block', type: 'paragraph', paragraph: { text: [Array] } },
  { object: 'block', type: 'paragraph', paragraph: { text: [Array] } },
  { object: 'block', type: 'paragraph', paragraph: { text: [Array] } },
  { object: 'block', type: 'paragraph', paragraph: { text: [Array] } }
]

This is all the arrays in the text field:

[ { type: 'text', text: { content: 'Hi this is a message with:' } } ]
[
  { type: 'text', text: { content: '• ' } },
  {
    type: 'text',
    text: { content: 'bold' },
    annotations: { bold: true }
  },
  { type: 'text', text: { content: ', ' } },
  {
    type: 'text',
    text: { content: 'italic' },
    annotations: { italic: true }
  },
  { type: 'text', text: { content: ', and ' } },
  {
    type: 'text',
    text: { content: 'code' },
    annotations: { code: true }
  },
  { type: 'text', text: { content: ' , along with ' } },
  { type: 'text', text: { content: 'links', link: [Object] } },
  { type: 'text', text: { content: ' and emojis 🥔 🦐 👋 ' } }
]
[
  { type: 'text', text: { content: '• and tagged users like HEY ' } },
  { type: 'mention', mention: { type: 'user', user: [Object] } },
  { type: 'text', text: { content: ' ' } }
]

This project was a bit of a whirlwind, but overall very helpful to me.

Known Issues 🐄

if you do multiple annotations to the same text, like bold and italic at the same time, it will pretty much completely break. This can be solved by adding new functions and parsing with the proper format
if you have some sort of file or image, it won't add it to Notion (Notion doesn't support inputting files at this time)
different block types, like code blocks or quote blocks, won't work (Notion doesn't support yet)
tagging @channel or @here won't work with this because Slack has different formatting, but that can be fixed by adding replacement values. The formatting for those is <!channel> or <!here>.

As always, had so much fun learning with this project. This was part of a larger project, which you can find on GitHub and the blog (coming soon).

The GitHub for this project is here.

Hope to see you around here again soon! ✌️

How to find non-admin Notion user IDs

Victoria Slocum — Thu, 09 Sep 2021 21:57:27 +0000

For our Standup Integration project, we needed both the Slack and Notion user IDs to be able to tag users on both platforms. However, you can only see the User ID of the admin members of Notion through Notion software. So we had to get creative, and this is what we came up with.

The process

Step 1)

Go to your main Notion page and go to the developer tools using inspect. Navigate into the "Network" tab on the top of the console, and then refresh the page. It should look like this when you're done.

Step 2)

On the left hand side near the blue funnel thingy, there's a search icon. Search a team member username. In our experience, the team members that showed up depended on who's name you searched, so in order to find everyone you might have to search a few different names.

Step 3)

Click on the syncRecordValues request and go into the response tab. Copy all text.

Step 4)

Cleaning up the JSON with a JSON Beautifier makes it easier to understand what we're looking at (we used https://codebeautify.org/jsonviewer).

Step 5)

Gather all your teammates ID's within the "notion_user" brackets. Again, if all your teammates aren't there, you may have to search a couple different names.

Step 6)

In order to match the Notion ID to Slack user, I found it easier to use the Slack user ID too. You can find this by going into the teammates profile page, clicking on "More" (three dots), and "Copy member ID".

Finished result:

// Slack user ID to Notion user ID dictionary
const slackNotionId = {
  UT9G67J1Z: "f2ca3fc5-9ca1-46ed-be8b-fb618c56558a",
  U0185FAF1T5: "6718f0c7-f6e3-4c3a-9f65-e8344806b5b6",
  U025P5K0S0Z: "6f7ce62c-fa2e-4440-8805-72af5f937666",
  U021UR4DW5C: "8fd7689c-d795-4ae9-aa53-5846ac1569b7",
  U0224KFNYRW: "7c02e0ba-2aec-4696-a91d-ecaa01b616ce",
  U025J9SLXV3: "94f6b8b7-e8b0-4790-8265-f08e6b1d550c",
  UT9G67YFM: "6c3a6ec1-4b99-4e5c-8214-cea14fd9b142",
};

Zapier: A Notion/Slack Integration for Standups

Victoria Slocum — Fri, 27 Aug 2021 17:55:01 +0000

Part 1: Trying with Zapier

I currently am working on a project called Amici that aims at making it easier to schedule with your friends through an app and web platform. We're a group of seven college students and young professionals, so we're learning lots every day.

We originally started doing standups in Notion through a database, but staying up to date with the page was difficult so we decided to move to a Slack channel. Our Notion page retired, and we moved to a purely Slack standup.

I recently had the idea of sending the slack messages to a Notion database automatically, which I thought would be super helpful. We only have the free Slack plan, so that means a lot of our past messages have been deleted. Also, there's no way to really go back and sort messages through tags, and Notion is also very helpful to visually see topics.

I had planned on doing this through code, working with the Slack API and the new Notion API. However, after doing some preliminary research, I found Zapier, and figured I'd give it a go.

Creating a workflow in Zapier was super easy. Just connect a couple apps and then you can start creating your integration. It took a little bit (and some help from Ben Potter) to figure out the actions and triggers, but eventually we got it.

Our goal was to take a message from Slack, extract a title, person, date, link to the parent message, and tags and put that into a card in a Notion database. We then wanted a bot to send the link to the Notion page into the thread in Slack, and append any additional threads into the Notion page body. It took a lot of figuring out to extract the right information and put it in the right place, but it wasn't necessarily hard. Zapier made it really easy to do things with zero coding experience.

We were able to figure out almost everything. We did not get to appending additional text to a Notion body, it seems like this is possible in the Notion API using blocks, but Zapier doesn't support blocks in their finite list of notion actions. Additionally, we only set up adding a single tag, so you wouldn't be able to add multiple tags to an item.

The steps:

Our first thing was to get any new message posted to a certain channel in Slack. We previously had a channel named "standup", so it was as easy as selecting from a dropdown. We also chose for it to not trigger with Bot messages.

Our next two actions were to extract the title. Firstly, we took the first line of the incoming message using the string.split function. We separated on a newline (using [:newline:], and took the first segment (0).

We also decided to limit the character count in the next action in case someone forgot it was the title. For this, we used string.truncate and set the max length to 100 characters. The input for this is Step 2, the first line we extracted.

This was definitely the hardest part of the entire process. In order to add a person into the Notion page, Notion wants a specific User ID. However, you can only see the User ID of the admin members of Notion through Notion software. So we had to get creative, and this is what we came up with.

Step 1:

Step 2:

Step 3:

Click on the syncRecordValues request and go into the response tab. Copy all text.

Step 4:

Cleaning up the JSON with a JSON Beautifier makes it easier to understand what we're looking at (we used this site).

Step 5:

Gather all your teammates ID's within the "notion_user" brackets. Again, if all your teammates aren't there, you may have to search a couple different names.

Step 6:

We can now populate our table! In order to create this action, choose "Utilities" for the event, and then "Lookup Table". Your lookup key will be the Slack user ID, and you can populate the table with the Slack IDs in the "key" columns and the Notion ID in the "values" columns.

The following part is extracting the tag. Unfortunately, I was only able to get it to find a single tag, but I think having more than one tag would be possible using the string.split function with spaces as the split criteria.

The format for this is going to be "Tags: Text", where Notion will only pick up the "Text" part. I thought it would be helpful to have some sort of indicator in the Slack message for the tags, but feel free to leave that out.

We're going to do a similar thing for the title, and split it on a new line, but this time we want to get the second line. I also chose to do a character limit of 25 characters for the tag, and the input value is the output from step 5. In order to have it only pick up on the text after "Tags: ", I skipped 6 characters in action 6.

This next part was a little bit of a fail. The idea was to have it create the database item if it was the parent message or append the message to the item's body if it was in a thread. We were not able to get it to append the thread message, but I'll explain what we went through anyways.

First, we set up a path to be able to conditionally run different steps. We'll start with the original (parent) message.

The first thing we do is check if the message is the parent message. In Slack, the Thread TS (timestamp) does not exist on parent messages, so we'll use that as our criteria.

(note: We had trouble with this because there seems to be different ideas about this on different internet pages. However, this seemed to work, so we stuck with it.)

We can finally create our database item in Notion. For this, you want to chose the Notion app when you make a new action. You'll have to choose an account and database, but you should already have this set up when you set up Zapier.

The name of the event would be the output from step 3, our limit of 100 characters step of the first line. The person will be the output from step 4, the table that we made. The link to Slack will be the Permalink that you can find by searching the 1st step's items. Tags will be the output from step 6, our limit of 25 characters of the second line step. You'll want to add a property item in your Notion (you can hide it) for the TS (there's a type in the title of the Notion property item, it should be "TS" not "Thread TS"). Finally, the content will be the text from step 1.

Finally for the parent message steps, we want to send a channel message in Slack with the link for the Notion page. For this, we can choose the Slack app from the new action page and set up the account and choose our "standup" channel. You can customize the text, and then insert the link made in step 9 for the Notion page. Send as a bot, and set up a name and a custom icon. You can add the TS in the Thread section to have it be messaged in the thread.

For the thread message part, we want to set up the opposite of the parent message, so we will put "if the Thread TS exists".

Next, we need to find the database item in Notion. Choose your account and database, and then use the "Thread Ts" from step 1 to match the thread Ts already inputted in the Notion database item from the parent message.

(note: there's an error here, don't include the permalink in the link section, just use the thread Ts)

Next, we want to update our Notion database item. Again, this didn't actually end up working, but this is what we tried. The item is the thread message from step 2, and you can customize the content added.

This seemed to work on Zapier's end, but Notion wouldn't add it to the body. Seems like its possible with the API using blocks, but Zapier may not yet support that.

This is the final product:

And so when you send a message in Slack, it will create an item in the Notion database!

Only... not so yay. Zapier's free plan only allows you do do one step integrations, which I didn't quite realize before setting this up. Amici isn't in the pace to spend $20 a month right now for this feature, so we wouldn't be able to afford the paid plan. All is good though, because now I get to do a part 2 of coding it myself.

These are some things I would have done differently or added if I were to do it again:

added the beginning actions to the original action to save on action slots
figured out why it wouldn't append a text body to the end of the notion page
tried to get it to be able to add multiple tags, separated by spaces or commas
solely had the body of the message be added to the body of the page using another action, so the title and tags wasn't in the body of the Notion item too

I can't share the Zapier template, but here's the notion template.

This was super fun to do, and I look forward to working with Notion and Slack more. Sorry Zapier 😭