DEV Community: Shriji

Sumrise is a personal spend analyzer tool that parses credit card and banking transaction emails forwarded to an inbound processor. It extracts transaction data asynchronously and displays monthly expenditure summaries and transaction lists in a simple dashboard.

Key features: Dockerized, 100% Self hostable, Bank and currency agnostic (It is just the filtering logic with your mail provider), You control the emails, You (can) control the LLMs, You control the data.

Bonus: Mobile responsive, dark mode, no tracking, no ads, no signup, no login (of course you login only as a super admin in an environment you control), no data collection, no data sharing, no data selling, no data mining, no data profiling, no data anything.

tools: sveltekit, pocketbase, postmark, groq, docker, cloudflare, cloudflared.

Demo

Code Repository

peopledrivemecrazy / sumrise

Sumrise is a personal spend analyzer tool that parses credit card and banking transaction emails forwarded to an inbound processor. It extracts transaction data asynchronously and displays monthly expenditure summaries and transaction lists in a simple dashboard.

Sumrise Spend Analyzer - Developer Specification (MVP)

Overview

Sumrise is a personal spend analyzer tool that parses credit card and banking transaction emails forwarded to an inbound processor. It extracts transaction data asynchronously and displays monthly expenditure summaries and transaction lists in a simple dashboard.

Architecture

sequenceDiagram

    participant U as You

    participant B as Bank

    participant G as Gmail

    participant P as Inbound Processor

    participant DB as PocketBase

    participant CRON as Cron Job

    participant GROQ as Groq API

    participant UI as SvelteKit App

    U->>B: Make a transaction

    B->>G: Sends transaction email

    G->>P: Forwards email to inbound processor

    P->>DB: Saves email to raw_emails collection

    CRON->>DB: Fetches queued emails

    CRON->>GROQ: Sends email for parsing

    alt Groq succeeds

        GROQ-->>CRON: Parsed transaction data

        CRON->>DB: Updates transaction record

    else Groq fails or response is incorrect

        CRON->>DB: Marks record as failed with error_msg

        UI->>DB: Displays link to retry parsing

    end

    U->>UI: Views dashboard

    UI->>DB: Fetches processed

…

View on GitHub

https://github.com/peopledrivemecrazy/sumrise

How I Built It

This project is built with a focus on simplicity, rapid development, and leveraging powerful, free-tier-friendly tools.

Most email providers have a way to forward emails to any email address. I use Gmail and can use the filter feature to forward emails with a specific match pattern (Subject, from, body text, etc) to a specific email address. I forward these emails to my Postmark mail.

This in turn makes a call to the webhook I set up with Pocketbase (hooks), which is a simple script that parses the email and saves the data to the database, adds queued status to the record. A cron job is set up to run every minute and checks for records with queued status and makes a subsequent call to the groq api to parse the email and update the record with the parsed data to the transactions collection.

You make a transaction -> Your bank sends an email -> Your email provider (Gmail in my case)forwards the email to my inbound processor -> Pocketbase saves the email to the database -> Cron job calls Groq api to parse the email and updates the database -> SvelteKit app displays the data in a simple dashboard.

Also, sometimes Groq fails or sends an incorrect response. I have a simple fallback that adds a failed status to the record and a link to access the failed transaction, which the user can retry. It sets the status to queued, and the cron job will retry it.

Tech Stack

Backend: I chose PocketBase as the all-in-one backend. It's an open-source Go application that provides a SQLite database, real-time subscriptions, user management, and a full admin UI in a single file. It's incredibly fast and easy to set up. The asynchronous email processing is handled by a cron job written in JavaScript and executed by PocketBase's integrated JSVM.
Frontend: The UI is built with SvelteKit, a modern web framework that allows for building highly performant applications with a great developer experience. For styling, I used Tailwind CSS to quickly create a clean and responsive design without writing a lot of custom CSS.
AI-Powered Parsing: To extract structured data from unstructured email text, I'm using the Groq API. It provides blazing-fast LLM inference, which is perfect for the real-time nature of this application.
Development & Infrastructure: The entire stack is containerized using Docker and Docker Compose. This ensures a consistent development environment and simplifies setup. The stack includes three main services: the SvelteKit app, the PocketBase server, and an Nginx reverse proxy. Nginx is configured to serve the frontend and backend under local domains (sumrise.test, pocketbase.test) with locally-trusted SSL certificates generated by mkcert.
Inbound Email Forwarding: Postmark to forward emails to the inbound processor. I'm using the Postmark Inbound Webhook to forward emails to the inbound processor.
Cloudflare: to serve the frontend and backend. I'm using the Cloudflare tunnels to expose the frontend and backend and allow Postmark to webhook to reach my machine.

Important notes

This is a Docker-focused project; most of its orchestration can be found in the docker-compose.yml file.

Nginx is configured to serve the frontend and backend under local domains (sumrise.test, pocketbase.test) with locally-trusted SSL certificates generated by mkcert.
A custom domain for the app sumrise.test and pocketbase.test for the backend, and edited the /etc/hosts file to point to my local machine.
Gmail Specific Configuring filter + Forwarding instructions here
When adding your postmark inbound email to Gmail, you will get a verification email; you need to verify it to start forwarding emails.
I am using Cloudflare Tunnels (cloudflared) with ingress to expose the frontend and backend to the internet.
For demo purposes, the cron job is set to run every minute.
Tweak the SYSTEM_PROMPT to your needs.

# Cloudflare tunnel config
tunnel: tunnel_id
credentials-file: /<user_home>/.cloudflared/tunnel_id.json
ingress:
- hostname: pocketbase.domain.com
  originRequest:
    disableChunkedEncoding: false
  service: http://localhost:8080
- hostname: sumrise.domain.com
  service: https://sumrise.test
- service: http_status:404

Setup

Create and set up a filter to forward the emails to Postmark.
Configure webhook in Postmark

For more information, please see the repo.

Final Submission: Dev to Sapper

Shriji — Sun, 10 Jan 2021 08:21:27 +0000

What I built

A beautiful portfolio/personal site generator that can be deployed with this button.

Category Submission:

Personal Site/Portfolio

App Link

https://blog.shriji.xyz/

Screenshots

Description

Requires you to have DEV account
Needs DEV API
At least 1 article written

With the above list and the supplied env variables for both local development and digital ocean deployment

Link to Source Code

peopledrivemecrazy / dev-to-sapper

Portfolio/Personal blog generator from dev.to API.

DEV to Sapper Portfolio/Personal blog generator

You can check out the deployed Sapper app at https://blog.shriji.xyz/

Note: Following these steps will result in charges for the use of DigitalOcean services

Requirements

You need a DigitalOcean account. If you don't already have one, you can sign up at https://cloud.digitalocean.com/registrations/new

Development and Forking

Fork/clone this repo and create a new .env file with variables in the provided .envsample.

npm install
#yarn install

#develop like any other Sapper app

Issues and PRs are welcome :)

Deploying the App

Click this button to deploy the app to the DigitalOcean App Platform.

Note: Once you begin to deploy there is a step where you need to supply your environment variables check .envsample for the format

Making Changes to Your App

As long as you left the default Autodeploy option enabled when you first launched this app, you can now make code changes and see…

View on GitHub

Permissive License

MIT

Background

I always wanted to consume DEV API to do something and this hackathon was the perfect motivator for building this app. Also, the one-click deploy fun.

How I built it

Introducing: Dev to Sapper

Shriji — Sun, 10 Jan 2021 07:54:48 +0000

The documentation has all the extra information needed.

Sanity check before we proceed with the application

Make sure you have the API Key ready.
Have Postman or any other HTTP API tool.
Sapper app ready and running.

Local development

Important endpoints

https://dev.to/api/articles/me?per_page=${PER_PAGE}&page=${page}
https://dev.to/api/articles/${USERNAME}/${sluginput}
https://dev.to/api/comments?a_id=${data.id}
https://dev.to/api/users/me 
https://dev.to/api/articles/me?per_page=4

Using your postman you can experiment with the endpoints and set the API in the headers with api-key in key and the value being set to your DEV API.

I am in a hurry to see the project live!

If you clone the repo and try to run it should report Invalid value "undefined" for header "api-key" this means the .env file isn't created yet, use the template below.

ENV Variables for .env file.

DEV_USERNAME = username
DEV_API_KEY = supersecretkey #obtained via https://dev.to/settings/account
PER_PAGE = 10 #if you like pagination
COMMENTS = true #if you want comments in each post
SITE = https://yoursite/ #SEO/Social Share

These variables are set to generate the entire portfolio and all the fetch calls will use these variables.

Anatomy of the project

src
│
└───routes
    │   about.svelte
    │   index.json.js
    │   index.svelte
    │   store.js
    │   _error.svelte
    │   _layout.svelte
    │
    └───blog
            index.json.js
            index.svelte
            [slug].json.js
            [slug].svelte
            _post.js
            _posts.js

Sapper's preload function can pre-render the content from the API and this is the secret sauce of the project.

Looking at the blog directory it has .js files and .svelte files, the .js files are executed on the server side and can be imported to the .svelte components(files) and the files that start with _(underscore) is private and will never be executed or be visible to the client side.

//_posts.js
require("dotenv").config();

const API = process.env.DEV_API_KEY
const PER_PAGE = process.env.PER_PAGE

//https://dev.to/api/articles/me
import { send } from 'httpie';


let data = async function getPosts(page) {
    const res = await send("GET", `https://dev.to/api/articles/me?per_page=${PER_PAGE}&page=${page}`, {
        headers: {
            'api-key': API
        }
    });

    let data = res.data;

    return data.map(e => {
        let date = new Date(e.published_at);
        let tags = e.tag_list
        date = date.toLocaleDateString("en-GB", {
            year: "numeric",
            month: "short",
            day: "2-digit",
        })
        let title = e.title
        let slug = e.slug
        let id = parseInt(e.id)


        return { title, date,tags,id,slug,tags }
    })

}
export default data;

This is the code for populating the the /blog route. *Note, _posts.js acts as a backend and only the responses are available for the client app.

The magic, you see index.json.js has no _(underscore) prefixed so this means the file is visible to the client app and you can verify it by going to http://localhost:3000/blog.json and you should see the JSON response.

//index.json.js
import postData from './_posts.js';

require("dotenv").config();
const PER_PAGE = process.env.PER_PAGE
export async function get(req, res) {

    res.writeHead(200, {
        "Content-Type": "application/json",
    });

    let pageNum = req.query.page
    let data = await postData(pageNum);
    let postList = data;
    let hasNext = true;
    hasNext = postList.length < PER_PAGE ? !hasNext : hasNext


    postList = postList.map((post) => {
        let { title, slug, date, id, tags } = post

        return {
            title, slug, date, id, tags,
        };
    });
    res.end(JSON.stringify({ postList, hasNext }));

}

Now that we know the relationship of the underscore files and it's corresponding js files. We just need to execute the preload function to load data on the template.

<script context="module">
    import { nextCursor } from "../store";

    export function preload({ query }) {
        return this.fetch(`blog.json?page=${query.page}`)
            .then((r) => r.json())
            .then((pageData) => {
                nextCursor.update(() => pageData.hasNext);
                return { pageData, currentpage: query.page || 1 };
            });
    }
</script>

###### index.svelte

The similar approach is done for the home page, blog(all articles) and the article page.

peopledrivemecrazy / dev-to-sapper

Portfolio/Personal blog generator from dev.to API.

DEV to Sapper Portfolio/Personal blog generator

You can check out the deployed Sapper app at https://blog.shriji.xyz/

Note: Following these steps will result in charges for the use of DigitalOcean services

Requirements

You need a DigitalOcean account. If you don't already have one, you can sign up at https://cloud.digitalocean.com/registrations/new

Development and Forking

Fork/clone this repo and create a new .env file with variables in the provided .envsample.

npm install
#yarn install

#develop like any other Sapper app

Issues and PRs are welcome :)

Deploying the App

Click this button to deploy the app to the DigitalOcean App Platform.

Note: Once you begin to deploy there is a step where you need to supply your environment variables check .envsample for the format

Making Changes to Your App

As long as you left the default Autodeploy option enabled when you first launched this app, you can now make code changes and see…

View on GitHub

Sapper, a framework for JAMstack applications

Shriji — Wed, 30 Dec 2020 13:13:41 +0000

Alright, What the heck is Sapper?

Sapper is the companion framework of svelte, like how Next/Nuxt is for React/Vue.

JAMstack?

Javascript APIs and Markup is an architecture that is easy to build, maintain and scale.

The DEV API

Dev has great API documentation but there is still a lot of room for improvement.

Getting started with Sapper

This is the standard way to get a default starter template for docs head to Sapper site.

# for Rollup
npx degit "sveltejs/sapper-template#rollup" my-app
# for webpack
npx degit "sveltejs/sapper-template#webpack" my-app
cd my-app

npm install
npm run dev & open http://localhost:3000

Now we got the all ingredients for making a JAMstack application we can start building the app but first, this is the anatomy of Sapper directory

src
├───components
│       Nav.svelte
└───routes
    │   about.svelte
    │   index.svelte
    │   _error.svelte
    │   _layout.svelte
    └───blog
            index.json.js
            index.svelte
            [slug].json.js
            [slug].svelte
            _posts.js

The majority of the work is done inside the src folder. The next part will have details of how the blog is fetched from DEV API.

###### Pretext, [SvelteKit](https://svelte.dev/blog/whats-the-deal-with-sveltekit) is coming out and the same can be made using SvelteKit

One-click Portfolio/Personal blog generator from dev.to API

Shriji — Sun, 27 Dec 2020 20:45:11 +0000

What I built

Simple Sapper Personal site that also fetches your articles from DEV using the API and you only need to supply just four environment variables during digital ocean app deploy.

Not just that it also updates the site when you write a new article at dev, how cool is that?

Category Submission:

Personal Site/Portfolio

App Link:

https://dev-to-sapper-zfnzi.ondigitalocean.app/ also at https://blog.shriji.xyz/ which was easy to configure via DigitalOcean's app dashboard

Screenshots

Home page

Posts page fetches 10 articles (Can be tweaked via .env)

Description

Sapper is the companion framework of svelte, like how Next/Nuxt is for React/Vue. You have the benefits of svelte and all the SEO powerhouse and Server Side Rendering. You can read more about sapper here.

Options as of now that can be supplied via .env fork/clone for local development. The same variables need to be supplied at DigitalOcean. The template is below.

DEV_USERNAME = username
DEV_API_KEY = supersecretkey #obtained via https://dev.to/settings/account
PER_PAGE = 10 #if you like pagination
COMMENTS = true #if you want comments in each post
SITE = https://yoursite/ #SEO/Social Share

Link to Source Code

https://github.com/peopledrivemecrazy/dev-to-sapper

Go to the repo hit the Deploy to DigitalOcean button;

Raise issues in GitHub if something went wrong, PRs are always welcome :)

Permissive License

https://github.com/peopledrivemecrazy/dev-to-sapper/blob/main/LICENSE

Background

There aren't many examples showcasing the ease and joy of creating Sapper apps, I always wanted to consume DEV API to do something and this hackathon was the perfect motivator for building this app. Also, the one-click deploy fun.

How I built it

I have been working with Svelte and Sapper and I have written most of my articles about them and my previous app with Sapper which I had a lot of fun building Trello JAMstack and showcasing on one of the talks and this one-click app is a perfect example of how a personal portfolio/blog can be generated with Sapper.

This project uses Luke Edwards's httpie, A lightweight alternative to node-fetch which gets the job done.

In general, Svelte steers users in the direction of using lightweight libraries.

Additional Resources/Info

Svelte
Sapper
Repo

Svelte: change API URLs during Development and Build

Shriji — Tue, 24 Nov 2020 08:45:03 +0000

This is a frequent question that is asked how to detect whether the environment is either development or production also this is significantly helpful when apps are being frequently published and the client-side app needs API that may be different URL for local/development and during production builds.

Solution

Svelte's most adopted bundler is Rollup and the default rollup plugin config looks like this


    plugins: [
        svelte({
            dev: !production,
            css: css => {
                css.write('bundle.css');
            },
            preprocess: sveltePreprocess(),
        }),
        resolve({
            browser: true,
            dedupe: ['svelte']
        }),
        commonjs(),
        typescript({
            sourceMap: !production,
            inlineSources: !production
        }),
        !production && serve(),
        !production && livereload('public'),
        production && terser()
    ],

I have stripped out the comments

We need to add @rollup/plugin-replace as a dev dependency.

npm i -D @rollup/plugin-replace

General usage and more info check out the github

Now, this plugin is going to look for occurrence(s) of MYURL and destructively replace it. So consider having an unique string to be replaced.

For easier explanation here is a quick example.

import replace from 'rollup-plugin-replace';

export default {
    // ...
    plugins: [
        replace({
        'MYURL':production?'http://myhost:3000':'http://localhost:3000'
        })
    ]
};

Example App.svelte


<script>
  let url = "MYURL";
  let api = "/myendpoint";
  url += api;

//your normal fetch logic.

/* async function fetchData() {
        const response = await fetch(url)
        result = await response.json()
        return result
}
*/
</script>

<!--
check URL using your template to verify 
when your svelte app is run in dev mode your URL will be http://localhost:3000/myendpoint
when your svelte app is served after the build (production) your URL will be http://myhost:3000/myendpoint
-->


<p> {url} <p> 

<!-- template logic
{#await fetchData()}
   ...
{/await}
-->

Advantage of letting rollup replace the URLs.

Good news, we are not exposing the dev and prod API endpoints on the client-side app. We are leveraging the capabilities of rollup during the build process this makes sure that you not only have the URLs securely in your .env and has only one file you need to change the base URLs. Take a look here how to identify dev or prod the article also achieves using rollup-plugin-replace

Svelte, Javascript but coolest

Shriji — Mon, 02 Nov 2020 10:08:47 +0000

This Looks Strange

If you're a first-time Svelte user such as myself, you might be asking yourself, "What is this strange framework, everything seems not redundant, not confusing and EASY, is this even used?" No worries, I have some answers for you first-time Svelte users. Svelte is a compiler that takes your component code and converts it into a clean JS code. "Svelte makes it joyful and to create interactive UIs."

Remember all I am offering is the truth, nothing more.

Svelte does make it joyful once you understand some of the basics that make Svelte so great. Soon you'll be creating apps much sooner and learn along with the process, giving Agent Smith a much-deserved prescription of Svelte magic if you catch what I'm pitching. Svelte is straightforward and comes with batteries and the learning curve is much less and you understand much quickly with a lot less boilerplate code. Let's take a look at some of the basics of creating Svelte apps.

Creating a New Svelte Application

Know HTML, know Svelte.

Cybernetically enhanced web apps, and so are the creators, contributors and discord chat most friendly and welcoming members of Svelte have made it simple for anyone with HTML and Javascript experience to create Svelte applications. Just enter in these 4 commands (separately) in your command prompt to get started:

npx degit sveltejs/template my-coolest-svelte-project
cd my-coolest-svelte-project
npm install
npm run dev

You will find .svelte files and Svelte components are built on top of HTML. Also, we have a cool REPL and Examples.

Just add data. No boilerplate

Variables are reactive. No Virtual DOM. Don't battle with what's rendered. You write less code which means fewer errors.

Props and Imports

Zero worries components "But wait? where are the complex classes that are exported in React?" Just export variables that are going to be props as simple as that 😉.

State via Stores

Svelte's most simplistic approach needs no installation of external libraries and anything on the store can be manipulated by using $ before what is variable is in the store. You do not need Redux.

Please read more about how joyful svelte can be at Shawn's blog and also about Stores + Auth

Were you listening to me neo or looking at the woman in the red dress? *I was..* Look again! https://youtu.be/YgJ5ZEn67tk?t=38

Now you know how to start with Svelte you have become much cooler

Links

Community

Official Twitter
Svelte Society
Come join our discord You can find me active there and when you get stuck somewhere say hi!

Other favourite talks.

post inspired from here, React, Javascript but Cooler

Making ISS 🛰️ tracker using Leaflet and Svelte V2

Shriji — Sat, 10 Oct 2020 11:43:28 +0000

A few months ago I made an ISS tracker and it had too many lines of code although easy it wasn't beginner-friendly so a month ago I started to work on making a user-friendly svelte wrapper for leafletjs which reduces the complexity drastically! check out the demo here

Quick start guide

Install via npm i npm i @anoram/leaflet-svelte.

<script>
import Map from '@anoram/leaflet-svelte'
let options={
  center: [13,80],
  markers: [
    {
      lat: 13,
      lng: 80
    }
  ],
  mapID: "map"
}
</script>
<style>
  .map {
    height: 600px;
    width: auto;
  }
</style>
<div class="map">
  <Map {options} />
</div>

Svelte REPL: https://svelte.dev/repl/a6a5c969aaf544018e5a4a9d7ea9e031?version=3.29.0

An updated version of ISS Tracker https://iss.anoram.com/

Repo here https://github.com/peopledrivemecrazy/svelte-iss

Enjoy!

Trello + Sapper JAMstack, When not to use

Shriji — Thu, 24 Sep 2020 19:45:30 +0000

While this is a fun project for JAMstack there are some limitations.

Multiple authors are possible but the latest edit becomes your published date.
The edits and updates can be tracked in Trello to check if point 1 happens.
Acknowledge you are using Trello for experiment/fun.

This demo shows how Sapper could be used for any headless CMS or other API solutions even Shopify has a headless mode.

This demo is also hosted here https://trello.anoram.com/

Check the repo here: https://github.com/peopledrivemecrazy/Sapper-Trello

Trello + Sapper JAMstack (Part 3)

Shriji — Thu, 24 Sep 2020 18:46:04 +0000

Sapper setup

Setting up Sapper is very easy to install for more details check here. I personally prefer Rollup.

# for Rollup
npx degit "sveltejs/sapper-template#rollup" my-app
# for webpack
npx degit "sveltejs/sapper-template#webpack" my-app
cd my-app

npm install
npm run dev & open http://localhost:3000

Once you have opened the project in your editor you will notice a confusing directory structure that may look daunting at first but it is pretty straight forward when it comes to producing your SSG part of Sapper.

└───routes
│   │   index.svelte
│   │   about.svelte
│   │   _layout.svelte
|   |   _error.svlete
│   └───blog
│       │   [slug].svelte
│       │   index.svelte
│       │   [slug].json.js
│       │   _posts.js
│       │   index.json.js

In Sapper, any file with _ underscore at the beginning is a hidden/private file. Our logic to obtain blog posts via API from Trello happens in the _posts.js file which generates the right JSON files for the blog posts. [slug].svelte and index.svelte contains template responsible for list of posts and the post itself. [slug].json.js and index.json.js exports the JSON it is possible without these files and yet has SSG done but it is a major hit on the performance and is very much noticeable with my series on headless WordPress and Sapper.

Snippet for _posts.jswhere we export a JSON object that has data for the blog post that could be populated on index.svelte and [slug].svelte

//_posts.js

import fetch from "node-fetch";
let data = {};
let url = `https://api.trello.com/1/lists/5f538d3a842e0a3b6ce9f259/cards?key={key}&token={token}`; //this is from your previous post
let trello = async function getPosts() {
  const res = await fetch(url);
  data = await res.json();
  return (data = data.map((e) => {

    const months = ["jan", "feb", "mar","apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec"];
    let posted_date = new Date(e.dateLastActivity)
    let formatted_date = `${posted_date.getDate()} ${months[posted_date.getMonth()]} ${posted_date.getFullYear()}`
    let tags = e.labels.map(e=>{
      return {
        name:e.name,
        color:e.color
      }
    })

    return { title: e.name, slug: e.name.toLowerCase().replace(/[^\w ]+/g,'').replace(/ +/g,'-'), desc: e.desc, image:e.cover.scaled,time:formatted_date,tags };
  }));
};

export default trello;

After a slight modification to index.json.js and [slug].json.js we can automatically get Trello items as blog posts.


//index.json.js
import trello from "./_posts.js";

export async function get(req, res) {
  res.writeHead(200, {
    "Content-Type": "application/json",
  });
  let data = await trello();
  let posts = data;
  posts = posts.map((post) => {
    return {
      title: post.title,
      slug: post.slug,
      time: post.time,
      tags: post.tags
    };
  });
  res.end(JSON.stringify(posts));
}

//[slug].json.js
import trello from './_posts.js';


const lookup = new Map();


export async function get(req, res, next) {
    // the `slug` parameter is available because
    // this file is called [slug].json.js
    const { slug } = req.params;
    let data = await trello()
    data.forEach(e => {
        lookup.set(e.slug, JSON.stringify(e));
    });
    if (lookup.has(slug)) {
        res.writeHead(200, {
            'Content-Type': 'application/json'
        });

        res.end(lookup.get(slug));
    } else {
        res.writeHead(404, {
            'Content-Type': 'application/json'
        });

        res.end(JSON.stringify({
            message: `Not found`
        }));
    }


}