DEV Community: Ces-D

Starting a CLI Finance Tool

Ces-D — Thu, 05 Aug 2021 22:04:13 +0000

Motivation

This is a finance tool project based heavily on beancount but with less features and way less complexity. After reviewing the beancount repo and documentation, I noticed that there are many more features available that I will use. In an attempt to create a new project with a new language, C#, this project was born.

Outline

This is a console application. As of right now the only major dependency is the System.CommandLine package. The commands code is written in namespace Controllers.Commands and the parsing and formatting of entries is written in namespace Controllers.VaultAccess.

Commands

Here is an example of one of my commands.

using System;
using System.CommandLine;
using System.CommandLine.Invocation;
using Controllers.VaultAccess;
using Config;

namespace Controllers.Commands
{

    public class NewCommand : Command
    {

        public NewCommand() : base("new", "Create a new vault entry")
        {
            AddArgument(DetailsArgument());
            AddOption(DateOption());
            AddOption(DirectiveOption());
            AddOption(ValueOption());
            Handler = CommandHandler.Create((DateTime date, string directiveType, string details, double currency) =>
            {
                JournalEntry entry = new JournalEntry(date, directiveType, details, currency);
                Console.WriteLine(entry.ToString());
            });
        }

The logic is unfinished but clear. My command class inherits from System.CommandLine.Command and abstracts away all of its logic when initialized. Using this format I am able to encapsulate the Options and Arguments into their own private methods.

 private Argument<string> DetailsArgument()
        {
            Argument<string> details = new Argument<string>("details");
            details.Description = "The details of this journal entry";

            return details;
        }

        private Option<DateTime> DateOption()
        {
            Option<DateTime> date = new Option<DateTime>("--date");
            date.AddAlias("-d");
            date.Description = "Date of entry";
            date.SetDefaultValue(DateTime.Now);

            return date;
        }

Journal Entry

Beancount is a very mature program and they have tested and perfected the input format for their entries. I am matching their input formats and created the JournalEntry class to handle the input, parsing, formatting, and output of the journal entry.
Here is an example of the format that I am expecting to output into my journal files
<Date> <Directive> <Details> <Value><Currency>
08/05/2021 open Liability:Food:Restaurant -35 USD

public class JournalEntry
    {
        private readonly int DATE_MAX_WIDTH = 13;
        private readonly int DIRECTIVE_MAX_WIDTH = 7;
        private readonly int DETAILS_MAX_WIDTH = 45;
        private DateTime entryDate { get; set; }
        private string directive { get; set; }
        private string details { get; set; }
        private double currency { get; set; }
        public JournalEntry(DateTime date, string dir, string det, double cur)
        {
            entryDate = date;
            directive = dir;
            details = det;
            currency = cur;
        }

        public override string ToString()
        {
            return string.Format("{0}{1}{2}{3}",
            entryDate.ToShortDateString().PadRight(DATE_MAX_WIDTH),
            directive.PadRight(DIRECTIVE_MAX_WIDTH),
            details.PadRight(DETAILS_MAX_WIDTH),
            currency);
        }

Next Steps

The hardest feature that I have yet to implement are the accounting logic and maybe more importantly the parsing of each line in the journal to make the various accounting reports. I intend to add reports for all the open assets and liabilities, balances of accounts, and history of expenditures.
The benefit of this project is that it is not the first. I can copy over some of the logic from existing accounting tools and then build and better them.

Github Repo

Discord Bot Framework

Ces-D — Sun, 04 Jul 2021 21:50:16 +0000

In this post I am going to be explaining the fully extensible discord bot built using DiscordJs. I am going to skip the part where you connect the bot to a discord server because this is explained in many other discord related blog postings. I will be running through and explaining the core files of the project.

Launcher

const main = () => {
  const prisma = new PrismaClient();
  const client = new Client();
  const control = new CommandControl(client, prisma);

  client.once(Constants.Events.CLIENT_READY, () => {
    prisma
      .$connect()
      .then(() => console.log("Prisma Connected"))
      .finally(() => console.log("Application Ready"));
  });

  client.on(Constants.Events.DISCONNECT, () => {
    prisma
      .$disconnect()
      .then(() => console.log("Prisma Disconnected"))
      .finally(() => console.log("Application Disconnected"));
  });

  client.on(Constants.Events.MESSAGE_CREATE, (message) => {
    control.handleMessage(message);
  });

  client.login(process.env.Bot_Token);
};

main();

This project relies heavily on typescript to keep writing consistent. Important variables to note are the client and control. All commands come in as Messages and are handled when the message event is listened to. If you are unfamiliar with the EventListener in node check out this link.

CommandControl

This class delegates all commands to their respective handlers.

 handleMessage(message: Message) {
    if (this.isAdmissableMessage(message)) {
      const command = this.parseForCommand(message);
      switch (command) {
        case UserEconomyCmds.LIST:
          this.Economy.listCommands(message);
          break;
        case UserEconomyCmds.CREATE_WALLET:
          this.Economy.createCommand(message);
          break;
        case UserEconomyCmds.BALANCE:
          this.Economy.balanceCommand(message);
          break;
        case UserEconomyCmds.MERCHANDISE:
          this.Economy.getMerchandiseCommand(message);
          break;
        case UserEconomyCmds.POSSESSIONS:
          this.Economy.getPossessionsCommand(message);
          break;
        case UserEconomyCmds.PURCHASE:
          break;
        case AdminEconomyCmds.SUMMARY:
          this.Economy.summaryWalletCommand(message);
          break;
        default:
          console.log(command);
      }
    }
  }

This is just an example so is already populated with optional commands. Firstly, isAdmissableMessage checks to make sure that the message the bot is listening to is going into a guild channel, not coming from the bot, starts with the command prefix, and is coming from a member of the guild. Secondly, the message is parsed to get the string connected to prefix, optimistically that is one of the commands being handled, but if it is not then it is ignored.

Utils

The only util as of this moment is a parser that ensures consistency oin the command format. I wanted to make a bot that accepted commands similar to a cli and parser.ts is what holds all this. There are three functions in the file, parseForArguments, hasAdminPermission, and argumentsFulfilled. These functions ensure that the message comes in like cli arguments.

Here is an example of a proper command: $deposit amount=10 @person.

parseForArguments

The message is split based on the spaces. The function accepts an array of strings that will function as the keys that are searched for in each of the strings in the newly made array. When in the key is found then the string is split again based on the "=" and returns the second part, optimistically that would be the correct variable. The function also accepts a boolean indicating whether the command requires a mention. The mention is for commands where you need to identify a specific guild member.

argumentsFulfilled

This function accepts the same array of strings that are keys for the command. This function makes sure that each one of those keys is returned from the parseForArguments function.

Cogs

This directory would be where you can add aspects into the bot. For example, I started an Economy cog with the class Economy which has a method handling each of the accepted commands.

Github

Space-Invader-Bot

How I Built An Incomplete CMS

Ces-D — Wed, 23 Jun 2021 17:49:28 +0000

I began building a blog application for a site that I may or may not finish. The bulk of the project is based on the blog-starter-typescript example in the NextJs GitHub repo. Their example loads markdown files from a folder in the root directory and uses remark and gray-matter to convert the markdown into html and to get file metadata. The example code for these function is located in their lib folder as api.ts and markdownToHtml.ts.

The system they used works very well, however, if you plan on creating many posts and incorporating many images then that is when the system begins to breakdown. Since you store all of the markdown files and images in your root directory then it will perpetually grow and hosting costs for packages that large are affected. This is where I was looking for a solution that would be light weight and adaptable.

In my search for a solution I considered databases to store the content. I initially considered a postgres relational database because that is what the typical first choice is when considering databases, but maintaining flexibility in the kind of content that is included in the post (videos, images, etc.) was important to me. Next I considered non-relational databases, but opted for my third option.

AWS S3 buckets can hold a wide variety of objects in storage and can be accessed through Node APIs. This seemed like the best option because I would be able to scale the number of posts I make indefinitely while not affecting my file size and I could maintain the already existing and tested markdownToHtml conversion. It is possible to store markdown files in AWS S3 buckets and then stream the contents into my existing functions. Before going any further, here is are the dependencies from my package.json.

"dependencies": {
    "@aws-sdk/client-s3": "^3.18.0",
    "@fontsource/signika-negative": "^4.4.5",
    "@fontsource/source-code-pro": "^4.4.5",
    "gray-matter": "^4.0.3",
    "next": "10.2.3",
    "react": "17.0.2",
    "react-dom": "17.0.2",
    "rehype-document": "^5.1.0",
    "rehype-sanitize": "^4.0.0",
    "rehype-stringify": "^8.0.0",
    "remark": "^13.0.0",
    "remark-parse": "^9.0.0",
    "remark-rehype": "^8.1.0",
    "slug": "^5.0.1",
    "unified": "^9.2.1"
  }

I created a new file called awsHandler.ts to store the AWS related functions. In my index.tsx file, which is typically found in the src/pages/ folder of NextJs application I would list the excerpts and descriptions of the blog posts. Initially this was done by reading all the markdown files contained in the root directory but now was changed to reading all the objects in S3 bucket.

export async function requestAllBucketObjects(): Promise<RequestAllBucketObjectsOutput> {
  const command = new ListObjectsV2Command({ Bucket: BUCKET_NAME, MaxKeys: 10 });
  const objectsOutput = await client.send(command);

  let promises: Promise<GetObjectCommandOutput>[] = [];
  let contentKeys: string[] = [];

  objectsOutput.Contents?.forEach((content) => {
    if (content.Key) {
      promises.push(requestSingleBucketObject(content.Key));
      contentKeys.push(content.Key);
    } else {
      return; //skip
    }
  });
  let allBucketObjects: GetObjectCommandOutput[] = await Promise.all(promises);
  return { allBucketObjects: allBucketObjects, contentKeys: contentKeys };
}

Notice that I return an object with allBucketObjects and contentKeys. This is where the incomplete aspect comes in. Using the sdk I am only able to search for specific bucket objects using the key in the form {BUCKET_NAME}/{FILE_PATH}.{FILE_TYPE}. I use these keys to request the metadata that I previously grabbed using gray-matter.

export async function getPosts(fields: string[] = []) {
  let posts: Items[] = [];
  const { allBucketObjects, contentKeys } = await requestAllBucketObjects();
  let promises: Promise<string>[] = [];

  allBucketObjects.forEach((output) => {
    promises.push(getMarkdownBody(output));
  });
  const markdowns = await Promise.all(promises);

  markdowns.forEach((markdown, index) => {
    const key = contentKeys[index];
    posts.push(organizePostItems(markdown, key, fields));
  });

  return posts;
}

Interestingly I was only able to use Node streams when requesting Bucket Objects from S3 requiring me to ensure the stream is Readable then adding each data chunk into a string. I need more practice with streams because ideally I would be able to stream and convert the entire function as it is being read instead of in the following format which defeats the purpose of the stream.

async function streamToString(readable: Readable): Promise<string> {
  let data = "";
  for await (const chunk of readable) {
    data += chunk;
  }
  return data;
}

So why is this a CMS. It is a loose term in this instance because I am using AWS S3 as my storage and a few general functions to control requesting, handling, and presenting the incoming data.