Transform Notion into RSS reader with AWS Lambda and AWS CDK

Dmitry Kankalovich — Wed, 19 May 2021 14:25:54 +0000

TL;DR: we are going to build a customized RSS reader on the basis of Notion using Notion API beta and AWS Lambda. See the source code here.

As of May 13th, the Notion had released a public beta preview of their API. This is a long-awaited feature that enables you to further consolidate your activities in one single tool. One particular thing which I always wanted Notion to provide is the integration for my daily RSS feed.

While Notion itself doesn’t have anything out-of-the-box for pulling in your RSS feed, the release of Notion API is something that provides us with the opportunity to leverage a myriad of automation platforms as well as serverless solutions from Cloud providers to implement whatever integration we want.

Before we start

You should know that there are several decent RSS readers like Feedly which do a fine job of aggregating your RSS feeds, so before jumping the gun I recommend checking them out.

Choosing the automation platform

Another thing which you might want to check out is the automate.io. This is one of many automation platforms which promises you to do what you need in just a few clicks.

There is indeed an advertised RSS integration with Notion which I went to test out — only to realize that it’s just not good enough for me. There seems to be a lack of configurability, like the ability to set up multiple RSS feeds, as well as be able to tune trigger logic.

However, we still can build what we need using a bit lower level abstraction — serverless function platforms like AWS Lambda or Azure Functions.

I personally was working with AWS for years and often use it as a platform for my personal projects, so here I’d go with that option. However, there are no AWS-exclusive specifics in the problem we’re about to solve, so any serverless function platform would work for you just fine.

Prerequisite

Here is the list of required stuff before moving forward:

NodeJS of version 14+ (comes with its package manager NPM which we’ll need later on)
An active AWS account.
AWS IAM user with enough permissions to deploy via CDK and use AWS Lambda — AdministratorAccess policy will be enough. Check out this guide for more details.
AWS CLI configured with that user credentials — Access Key ID and Access Key Secret. See this guide for examples on how to do it.

Make sure you’ve explicitly exported the AWS profile with these credentials:

% export AWS_PROFILE=your-profile

AWS CDK will check this variable by default to figure out which account it works with and what IAM user permissions it should leverage.

The AWS Part

While you can do all of the needed AWS changes and configurations by hand clicking through AWS Console, I usually go with the infra-as-a-code (IaaC) approach, so I am going to stick with that here.

AWS CDK is a relatively new way to provision your infrastructure following IaaC even more literally than Cloudformation / Terraform did before — using a programming language like Typescript to create AWS resources.

I will use AWS CDK here, but you can again achieve the same results with other IaaC options like Cloudformation, Terraform, and others.

| You will be able to find all source code here

Deploy your AWS CDK app

If you never worked with AWS CDK before — no worries, it’s all fairly simple. I recommend checking out this document. Assuming that you have installed node of at least version 14+ run the following snippet:

% npm i -g aws-cdk

Next, create your project directory and init the CDK project:

~ % mkdir notion-rss-feed && cd $_
notion-rss-feed % cdk init sample-app --language=typescript

As you can see — we explicitly specify the language as typescript. There are many other languages supported for the CDK, however, with Typescript, you'd be able to find most examples for as well as it being the first-class citizen language for the CDK, since all others are just wrappers.

Assuming cdk init ran without problems you should get the following structure:

notion-rss-feed % tree -L 1
.
├── README.md
├── bin
├── cdk.json
├── jest.config.js
├── lib
├── node_modules
├── package-lock.json
├── package.json
├── test
└── tsconfig.json

There is quite a number of files, but don’t worry — you don’t need to deal with most of them. The part you’d be looking into is in the lib folder - all the resource stack definitions will go there. By convention, CDK will create a single file there named notion-rss-feed-stack.ts.

So let’s go ahead and modify our Stack at lib/notion-rss-feed-stack.ts:

There are no resources defined just yet. It’s fine, we’ll add them later and for now, let’s just deploy this stack to verify we are able to perform such operation for your AWS Account.

First, bootstrap the environment (you need to do it only once per AWS account, and might not need to do it if you’ve been using CDK before):

notion-rss-feed % cdk bootstrap --profile your-aws-profile

Then — deploy our resource stack:

notion-rss-feed % cdk deploy

The output should look like this:

NotionRssFeedStack: deploying...
NotionRssFeedStack: creating CloudFormation changeset...
[██████████████████████████████████████████████████████████] (2/2)
 ✅  NotionRssFeedStack
Stack ARN:
arn:aws:cloudformation:us-east-1:69924*******:stack/NotionRssFeedStack/f6b61890-b4a3-11eb-a371-1266a0dd9289

You’re now all set for the actual development!

| If you see Unable to resolve AWS account to use. then check out this doc for the guidance on how to configure deployment target account and associated Access Key ID / Access Key Secret

Create Lambda function

Let’s create a Lambda function that will pull the specified RSS feed and push it to the Notion. We’re going to put lambda code in a separate folder:

notion-rss-feed % mkdir -p resources/lambda
notion-rss-feed % cd resources/lambda

Initialize the NPM project:

lambda % npm init -y

And then add all the needed dependencies:

lambda % npm install --save rss-parser @notionhq/client dotenv

Finally, add a Lambda handler code. Create index.js with the following contents:

Obviously, this is not the final version of the Lambda function, but something we can use for now to implement a proof of concept solution and then gradually improve upon.

Let’s get back to our resource Stack and create an actual Lambda function resource in AWS.

Edit lib/notion-rss-feed-stack.ts and add Lambda provision code:

If @aws-cdk/aws-lambda cannot be resolved - explicitly add the respective NPM package:

notion-rss-feed % npm install --save @aws-cdk/aws-lambda

And now we should be good to deploy our Lambda:

notion-rss-feed % cdk deploy

The CDK would ask you to confirm the creation of the IAM Role needed for the Lambda execution. Proceed with y as your answer and see the stack update progress. Finally, once deployment completed, you should be able to see the ARN of our Lambda function:

Outputs:
NotionRssFeedStack.NotionRssFeedHandlerName = NotionRssFeedStack-NotionRssFeedHandler4C3C1ED2-WD0IieIRVroG

Your exact Lambda name is always account-specific and will differ a little bit from mine, so be sure to copy the one that you actually see in your CDK deploy output.

Let’s use provided function name to test that Lambda function:

notion-rss-feed % aws lambda invoke \
--function-name NotionRssFeedStack-NotionRssFeedHandler4C3C1ED2-WD0IieIRVroG \
--payload '{}' lambda_output.json && cat lambda_output.json

The output should look like this:

{
    "StatusCode": 200,
    "ExecutedVersion": "$LATEST"
}
"The AWS Cloud platform expands daily. Learn about announcements, launches, news, innovation and more from Amazon Web Services."

This last line indicates that our Lambda had been successfully executed and had pulled in RSS feed of the AWS latest news. With this output verified we’re now officially halfway through.

The Notion Part

We need to prepare Notion before we can actually start sending the RSS content. Essentially we need to do the following:

Create a so-called Private integration
Create a parent page or database to hold all our daily RSS entries
Share that page or database with our integration

| The above steps are also described here in much greater detail, so if you’re having troubles understanding the context — check it out.

But before — let’s figure out exactly what we are going to do with respect to displaying RSS entires in Notion. I see there are three ways to achieve what we need, and I am going to walk through all of them.

Option 1. Database

We can create a Notion database, define some schema for properties and add items as child pages — a page per RSS item. This approach would allow us to supply rich content for every page (aka RSS entry), like a textual preview of the feed item.

However, there are some considerable drawbacks that stem from Notion API limitations at the moment. These are the inability to clean up the database or delete the particular page. With no ability to clean up before every integration run, it means that the database would likely pile up duplicated pages.

Option 2. Page with nested table

The idea here is to establish a regular Notion page with a nested table, which we can simply delete at the beginning of each integration run and re-create with RSS items. However, at the moment Notion API beta does not allow you to work with tables.

Option 3. List with links to the RSS items

Finally, we can establish a root page to serve as a table of contents and create a child page for every integration run (with a page title of current date), and for that given child page simply create a bulleted list of links. We won’t have any preview apart of the link name — RSS item title, but in many cases that will be enough. This is the variant we’re going with here.

Create Integration

Assuming you’re the owner or admin of your workspace, simply click Settings & Members → Integrations → Develop your own integrations → New integration.

Give it some meaningful name, upload an RSS icon, and finally hit Submit.

Now open the integration we’ve just created and grab Internal Integration Token — we’ll need it later.

Create RSS page

Next — create a place to store RSS items in Notion. For that create a new page, set the title RSS, and that’s pretty much it for the page layout. Later on, our integration will be creating here child pages with RSS content inside, and the RSS page — as long as it’s empty — will display all the children pages as a sort of table of contents.

Share RSS page with integration

The last thing for the Notion part — allow our integration actually access the page we’ve just created. So go ahead select our RSS page, hit Share → Invite → Select your integration.

Relaying RSS feed to the Notion API

Switch back to the code repo. We’re about to start sending RSS items to the Notion API. For that, we need to pass the integration token to Lambda. I strongly advise against hardcoding it, and rather suggest putting it in .env file. Simply create .env file at the root of the project and fill it with the following entries:

NOTION_INTEGRATION_TOKEN=<your notion integration token value>
NOTION_PAGE_NAME=RSS
RSS_FEED_URL=https://aws.amazon.com/about-aws/whats-new/recent/feed/

You can replace the page name and RSS feed URL with values of your choice, however, bear in mind that NOTION_PAGE_NAME should have the exact page name, which you've created in the previous step. Later on, we'll use the Notion API Search endpoint to find your page unique ID.

Let’s now update lib/notion-rss-feed-stack.ts with these config values:

You can see that we simply pass these values as-is via environment variables.

Finally, the Lambda handler code:

And that’s pretty much it for the Lambda part. Deploy it:

notion-rss-feed % cdk deploy

And check it:

notion-rss-feed % aws lambda invoke \
--function-name NotionRssFeedStack-NotionRssFeedHandler4C3C1ED2-WD0IieIRVroG \
--payload '{}' lambda_output.json && cat lambda_output.json

| Again, don’t forget to provide your own specific function-name found in the CDK deploy output.

If everything is good — you should be able to see your daily RSS feed:

So every time you invoke Lambda, a page with the current date gets created and lists all the RSS items in it, linking out to the respective pages.

Scheduling RSS feed

We’ve got our RSS feed in Notion! However, we don’t want to manually use AWS CLI every day, so let’s go ahead and schedule our Lambda to be executed on a daily basis automatically.

For that, we will use AWS EventBridge, which provides the capability to define CRON expression for triggering Lambda.

First, need to add some related dependencies. Switch to the project root and run the following:

notion-rss-feed % npm install --save @aws-cdk/aws-events @aws-cdk/aws-events-targets

Next — create an event rule and make it invoke Lambda on schedule. Edit lib/notion-rss-feed-stack.ts as follows:

Pay attention to the cron expression — you should set it to value that works for you. In my case, I’ve set it to trigger Lambda every day at 5:30 am UTC. You can read more about AWS cron expressions here.

Deploy our changes:

notion-rss-feed % cdk deploy

And this concludes our sample implementation.

| As I said in the beginning, you may find the final version of the sources here

What’s left

There are many areas to improve upon which I didn’t cover here for the sake of simplicity, but you’re welcome to take on that:

As we’re using only a Beta version of Notion API, more features are expected to come over time: deletion, tables, maybe even HTML or Markdown support. Those will open the opportunity to build a more sophisticated solution, ideally, the one that allows to actually preview content.
The security of the solution is not ideal — Integration Token is stored in plain text in Lambda environment variable. If there is anyone else using this AWS account, you should secure tokens using something like AWS Secrets Manager.
There is just one single RSS endpoint we’re using here. We can expand the solution to aggregate several endpoints.
AWS is not a particularly friendly place for personal projects, so I typically establish some monitoring and alarms to make sure some mistake in deployment (say — error in cron expression resulting in much more frequent Lambda invocations) won’t cost me a fortune.
Keeping RSS config committed to the repo, albeit as a part of the config file, is nice, but not as nearly nice as be able to configure it on the fly with some sort of simple UI.

For now — that’s it!

I hope you enjoyed this guide, and if there is going to be enough interest — I’ll do the second part, where we will address some of the mentioned issues.

Originally published on Medium

DEV Community: Dmitry Kankalovich