DEV Community

Cover image for Updating my Reddit Workflow with Pipedream
Raymond Camden
Raymond Camden

Posted on • Originally published at on

Updating my Reddit Workflow with Pipedream

This was originally just going to be a tweet, but then I realized I wanted a bit more space to talk about it and figured I'd write it up as a post. And since this is my blog and I can do what I want to, you get to enjoy this little nugget of information.

Back almost exactly a month ago, I blogged about using Pipedream to build a Reddit email report. The idea was that I wanted a daily email of posts from my subscribed subreddits for the past 24 hours of content. The implementation was a bit complex. I used one workflow to handle "get a days worth of content from subreddit" as a general "API" and another workflow connected to my authentication. It handles getting my subscriptions, hitting the API, and then generating the email. Here's an example of how that looked:

Email report

This worked well, but after a while of actually getting the email, I noticed some problems. The email takes all of the posts from all of my subscriptions and sorts them together. I thought this made sense to me, but I noticed it made it harder to actually read the content. Sometimes I don't care about a subreddit and mentally it just felt weird going from the movies subreddit to the Acadiana one. Also, some subreddits get a huge amount of traffic in a day. The email was hard to read and just too long.

So I decided to fix that. I didn't want to edit my original workflow because I wanted it to still be a reference for the older post. Luckily Pipedream makes that simple. I turned off the CRON schedule on the workflow and just used the copy command.

Next, I edited the Node.js code step that combines and sorts my data. This:

for(let i=0;i<result.length;i++) {
    this.posts = this.posts.concat(result[i].data.result);

this.posts.sort((a,b) => {
    if(a.created_utc < b.created_utc) return 1;
    if(a.created_utc >b.created_utc) return -1;
    return 0;

Enter fullscreen mode Exit fullscreen mode

Became this:

for(let i=0;i<result.length;i++) {
  this.posts = this.posts.concat(result[i].data.result.slice(0,10));

Enter fullscreen mode Exit fullscreen mode

No more sorting together and I'm only getting the first ten entries from each subreddit. (Those posts should be date sorted already. Should be.)

Then I modified the step that formats the email. I added in code to notice when a new subreddit start and added HTML to make it more visibly separated.

let dotLeft = function(s, len) {
  if(s.length < len) return s;
  return s.substring(0, len)+'...';

let lastSub = '';

this.subject = `Daily Reddit Report`;
this.body = `
<h1>Daily New Posts for Your Reddit Subscriptions</h1>
steps.get_and_sort.posts.forEach(p => {
  // only show thumbnails when they aren't "self","default","nswf" - or more broadly, not a url
  if(p.thumbnail.indexOf('http') === -1) delete p.thumbnail;
  let text = '';
  if(p.is_self) text = dotLeft(p.selftext,200);
  if(p.subreddit !== lastSub) {
    this.body += `<hr/><h2>${p.subreddit}</h2>`;
    lastSub = p.subreddit;
  this.body += `
${ p.thumbnail ? '<img src="'+p.thumbnail+'" align="left" style="margin-right:10px">':''}
<b>Title: ${p.title}</b><br/>
URL: <a href="${p.url}">${p.url}</a><br/>
Reddit URL: <a href="${p.permalink}">${p.permalink}</a><br/>
Author: ${}<br/>
<br clear="left">

Enter fullscreen mode Exit fullscreen mode

Basically - notice when the subreddit changes and add a horizontal rule and header. Also notice I modified my code on when to show images. This seems to work much better.

And that's it. My new workflow may be found here: I'm trying my best to make good use of the Readme feature to document what I've done.

Top comments (0)