DEV Community: Reaper

Writing cleaner state in React and React Native

Reaper — Mon, 30 Aug 2021 00:00:00 +0000

Ever since hooks got introduced in React, it made it a lot more easier to handle composition in react components and also helped the developers of react handle the component context a lot better. Also, as consumers of the library, we could finally avoid having to write this.methodName = this.methodName.bind(this) which was a redundant part of the code to which a few developers ended up writing their own wrappers around the component context.

But that's old news, why bring it up now?

Well, as developers there's always some of us who just go ahead follow the standard as is even when it makes maintenance hard and in case of hooks, people seem to just ignore the actual reason for their existence all together.

If you witnessed the talk that was given during the release of hooks, this post might be not bring anything new to your knowledge. If you haven't seen the talk

You should.
I'm serious, go watch it!

For the rebels, who are still here reading this, here's a gist of how hooks are to be used.

Context Scope and hook instances

If you've not seen how hooks are implemented then to be put simply, the hook will get access to the component it's nested inside and has no context of it's own, which then gives you ability to write custom functions that can contain hook logic and now you have your own custom hook.

Eg: I can write something like this

import { useEffect, useState } from "react";

function useTimer() {
  const [timer, setTimer] = useState(1);

  useEffect(() => {
    const id = setInterval(() => {
      setTimer(timer + 1);
    }, 1000);

    return () => clearInterval(id);
  }, [timer, setTimer]);

  return {
    timer,
  };
}

export default function App() {
  const { timer } = useTimer();

  return <>{timer}</>;
}

And that gives me a simple timer, though the point is that now I can use this timer not just in this component but any component I wish to have a timer in.

The advantages of doing this

I now have an abstracted stateful logic that I can reuse
The actual hook code can be separated into a different file and break nothing since the hook's logic and it's internal state is isolated.

This gives us smaller Component code to deal with while debugging.

What does any of that have to do with state!?

Oh yeah, the original topic was about state...
Now the other part of having hooks is the sheer quantity that people spam the component code with it and obviously the most used one is useState.

As mentioned above, one way is to segregate it to a separate custom hook but if you have like 10-20 useState because you are using a form and for some weird reason don't have formik setup in you codebase then you custom hook will also get hard to browser through.

And, that's where I really miss the old setState from the days of class components and there's been various attempts at libraries that recreate the setState as a hook and I also created one which we'll get to soon but the solution is basically letting the state clone itself and modify just the fields that were modified, not that hard right?

You can do something like the following

const [userDetails, setUserDetails] = useState({
  name: "",
  age: 0,
  email: "",
});

// in some handler
setUserDetails({ ...userDetails, name: "Reaper" });

And that works (mostly) but also adds that additional ...userDetails everytime you want to update state. I say it works mostly cause these objects come with the same limitations that any JS Object has, the cloning is shallow and nested states will loose a certain set of data unless cloned properly and that's where it's easier to just use library's that make it easier for you to work with this.

I'm going to use mine as an example but you can find more such on NPM.

import { useSetState } from "@barelyhuman/set-state-hook";
import { useEffect } from "react";

function useCustomHook() {
  const [state, setState] = useSetState({
    nested: {
      a: 1,
    },
  });

  useEffect(() => {
    /* 
      setState({
        nested: {
          a: state.nested.a + 1
        }
      });
    // or 
    */
    setState((draftState) => {
      draftState.nested.a = draftState.nested.a + 1;
      return draftState;
    });
  }, []);

  return { state };
}

export default function App() {
  const { state } = useCustomHook();
  return <div className="App">{state.nested.a}</div>;
}

and I can use it like I would with the default class styled setState but if you go through it carefully, I actually mutated the original draftState and that's because @barelyhuman/set-state-hook actually create's a clone for you so you can mutate the clone and when you return it still creates a state update without actually mutating the older state.

Summary

Use custom hooks to avoid spaghetti state and effect management code
Use a setState replicator if you are using way to many useState hooks

make it easier on your brain to read the code you write.

How I built a simple status site with go lang and vercel?

Reaper — Sat, 03 Jul 2021 15:59:18 +0000

By Status i mean this website barelyhuman/status

The reason I built it was simple, I needed a simple setup to see if the web services were all up and running, the list is quite small because those are the currrent one's I would like to keep a track off.

Now this was built in a quick 60 min - 80 min time span and I'm going to explain the things I did and why I did them.

Stack

Go lang (templates and http server)
Vercel

That's basically it. No seriously, that's about it.

Mental Model

The idea was to have a simple go binary that would render a html page with each website that needs to be checked and its particular status.
This is basically why the website takes a bit longer to load after the browser cache has expired

The go binary is the microservice that vercel will use as an entry point, so on a very basic scale, since I won't be using code to show how this is being done, we'll go through the list of things that it does.

Get the list of sites to be queried
Ping each site to check if it's up, in certain cases ping the backend api of the web service to check if the api is up.
Check for the status to be 200, if not then the site/api is probably down
Render the html with the above data
Upload this as a microservice endpoint to Vercel

Problems

Vercel doesn't allow a root endpoint without configuration so the endpoint for the micro service would be /api/pinger and the final url would be https://status.barelyhuman.dev/api/pinger , which while not a big deal, is a few more keystrokes than just https://status.barelyhuman.dev and thats one issue.
The html templates files are additional assets and those don't always work in the vercel api functions and that iffy deployment and checks would be a deal breaker for a service which is trying to show the current status of other services.
Vercel has a time limit on each request of 10 seconds , post the initial execution request, so about 12-13 seconds whole. If I ping a website on the Indian Server from a US server, the services that are hosted in EU / IN would add up about 2 seconds, thus making the website be shown as Timed Out if it accidentally crosses the 12 second mark.
Avoid showing people what the urls of each of these api's are, which while you can dig down a bit and find out, I'd prefer to have at least a layer of protection on top.

Solutions and Hacks

The first one was simple, i do a browser rewrite for the / url to /api/pinger and done, checked if vercel supported that and boom , done.

{
  "rewrites": [{ "source": "/", "destination": "/api/pinger" }]
}

The templates are just strings at the end of the day and go will build everything into a single binary so if I just write them as strings assigned to a variable, I can achieve the same result, so tried that and it worked as expected. I've explained about it in this discussion thread
Pinging each site in a linear fashion would be a bad idea but that was the initial prototype. I wanted to test if everything was already working. Checked if it was and then I went ahead and deployed the prototype. Right after I realised, that these would fail if the request is coming from the US server to the Indian hosted backend, thus making it slower, and that basically actually happened. It pinged barelyhuman's blog properly but failed to get the status for the Taco backend and crashed, a reload showed both the results but I don't want it to crash for something so trivial.
I then just setup a go routine to parallelize the fetching of each service's status and now the site handles 4 without any issue, not a big number but since it's parallel it theoretically should create any problems for more either.
I can optimize this better by limiting the time of each response to be under 4 seconds but then a busy server might not respond at once so I'm rethinking if I should/shouldn't do it.
The html render and the website links are all fetched from the environment variables so that's a secret on the vercel portal for now so as long as no one hacks into vercel, I'm a little safe, obviously you can look through each app but those are proxies and rate-limited proxies (not boasting, there's ways around each) but for now that's a simple barrier to keep the urls safe.

Announcing Taco

Reaper — Fri, 02 Jul 2021 04:43:22 +0000

I totally forgot about creating a mirror post when I announced this a that I'd get a little more exposure here.

Nope, I didn't forget about the announcement , nor was I busy doing something else. I wanted it to be a release at night so I can sleep and wake up and not worry about it all day or obsessively checking my email for feedback.

But night releases don't work!!, I know.

I'm not Elon musk with millions of people waiting for my one line tweet

Taco. It's up. check it on https://alpha.taco.barelyhuman.dev

I'm reaper, I talk full paragraphs, even if I can't do that in real life... anyway

Oh by the way, if you do just plan on clicking the above link and going on a test run instead of reading the whole thing

Username: demo@reaper.im
Password: Demo@123

You won't be able to run the signup, I don't want people spamming the test server with accounts they won't use

Taco

I'm going to answer a few questions regarding it and then you can go ahead and login with the test credentials to go through the alpha version.

Why Build another project manager?

I've been on it for a while actually, I built pending with the same thought and then never went ahead because then I'd be building what everyone else already has.

I'm not a kanban person, I prefer writing down notes and making checklists and marking them to manage my work.

I do have solutions for the checklists part which is todo and the notes are just me noting down on paper or a tablet device , whatever is closer and it's worked well for years but then I write these notes here and there and never find them when I really need them and have to do stuff off of memory, this is where the idea of just building one for myself came into the picture and also where I accidentally discovered that Linear existed.

Now linear does everything I expect an issue manager to handle and I can use it in the way I intended to build Taco and drop taco as a project altogether but then there's no BarelyHuman in the picture at all.

To become an indie developer supported by people so I can live off of projects I wish to build, there has to be a starting point and/or project. Right?

TillWhen was the prototype or experimental product to see if I'd be able to get users without any significant marketing and what that count is going to be, then I'd add up other metrics to see what I can improve on it and once I saw the hypothetical data of what I could do with a full fledged focused project , I sat down and finalised the designs for Taco.

I could just copy everything from linear and call it a day but the point is to always learn and experiment with things so I sat down to experiment with micro-services for a full app while avoiding any data cloning to be involved and how would I deploy this.

I came up with a go backend and a decent deployment plan but then I threw that away, started again and we ended up with a simple mono repo with a single stateless backend and frontend and obviously the concerns of UI Components and Shared Logical helpers as the other packages in the repo.

Also ended up designing a few new components and I now have a base css that I use with basically everything.

But none of that answers the questions, Why Project Manager? , I could've built anything, why project manager? Simple reason being that I've built versions of it before and it wouldn't be hard to build it as a proper product. My mentality was to build something simple enough to attract at least a small group of people and something I've worked towards enough to be able to build it to a bigger proper project in case it worked out.

And so Taco,

The current announcement you are reading is basically for the first alpha release of the product and not an MVP release, the alpha basically means that there's features pending, there's things that are going to keep getting added to it and that's the whole reason you get access to a testing account and not an account of your own. This is to understand and get feedback from you about how things can be improved in the product , because what you see in the product at this stage is good enough for me to work with other than the notes part which I plan to add next weekend.

Those are less features than what JIRA/ASANA/Linear/etc etc has!

You wouldn't be wrong in saying that.
My intention isn't to compete with any of the giants, they have teams, investors, other things that I might not even know off. Like most projects by me the first customer of the project is me and the only difference here is that I plan to make it a project where the customer isn't just a single user and hence the alpha stage release , so we carve out a product that's not just built on my thinking of the product.

You are wasting time.

Maybe, maybe not. I was happy when TillWhen got 3 users, I was happy when it got 150, that doesn't change for me, I built it. That's all that mattered to me back then and matters to me right now. Commitlog got 50 stars on GitHub, that was my goal for the entire year, I got lucky when it hit that goal in about a month.

I've got a lot of time, 16 hours every weekend. I spend that on multiple projects, This is just another addition to the list of things I'm going to enjoy building

What's special ?

Honestly, nothing during this release.

There's no USP (Unique Selling Point) , nothing that can get shark tank's attention.

But , just like every project by an indie developer, it's built to inspire more developers to build stuff on their own to learn, earn or maybe inspire more people.

What's special about it for the consumers, genius.

Let's see, you get to run the feedback through an actual person instead of a chat bot, you get to request help regarding issues with the person who's actually developing it, the problems get checked by a human (Even if the name is barely human) and not through a 10 step issue validation process.

And where do I send this feedback?

You can send this feedback on ahoy@barelyhuman.dev for the reaper.im fans or me@barelyhuman.dev for the people who'd like to be professional.

That's about it folks, hope you get through the app and like it, even in the shape that it is in right now.

Updates 15th and 16th May

Reaper — Mon, 17 May 2021 00:52:28 +0000

I think I should add more tutorials and tips than just post updates but I don't know, I just prefer using the blog as an Update Log, I will try writing more tips and considerations from now on.

As for now.

This update log is mostly focused on Taco cause that's all I worked on this weekend. There is a certain parser I wrote for a bit of work from office and it's a small update so let's start with that.

Mobile Version Sync

So, at Fountane we've been getting a lot of hybrid app requests for mobile development recently and it's a pain to keep the NPM, Android(Gradle) , iOS version and build numbers in sync and this causes an issues when things like an Update server are involved.

I did look up some NPM alternatives for the same but they weren't being maintained much and I didn't want to sit on a fork of a simple tool I could've written (or so I thought) and I got onto it. I already knew the iOS just needed updates in the Info.plist file so that would be a quick thing.

Read the plist file, parse the xml , edit the CFBundle properties for the version and build and write it back to the plist, easy.
Same goes for Android, read the build properties , update the versionName and versionCode, write it back and done.

Much to what I thought, there was no parser for gradle files in golang , so I ended up writing one which took me almost a day because I wanted to handle a few edge cases(not all) that would cause issues during the use case.

And it's a tightly coupled parser right now, as it's part of the version-sync cli right now, I'll decouple it as soon as I'm done with Taco, don't want to pause work on Taco and pick up new projects right now.

If you wish to understand the logic behind the parser, I did write a small prototype for the same in JS before I wrote the actual go lang parser so you can find the JS version on barelyhuman/gradle-parser-proto and the go lang cli tool on barelyhuman/mobile-version-sync , again it's in development and being used for a very specific use case, generalising it is a plan but not till I'm done with the existing project.

Taco

Trust me, I try to keep these posts short, they just end up being big since I unconsiously add in stuff that I think people should know.

Back to the topic, we've got the

Auth (will be changing to passwordless before release)
Projects
Tasks and Tasks Status
Settings and Plans
Integration with TillWhen (This one might come as a shock)

Auth

I'm basically done with the Auth part and it used the traditional email and password but I'll be moving it to the magic link setup that TillWhen has once I'm done with the rest of the features, this was done because it's quick and I didn't want to use TillWhen's code because I wanted the code to be concise for this project and TillWhen has patchy code that I didn't want to just copy over.

Projects

Simplest one to implement, creation and assignment and visualising of projects is done, here's a preview

What you see is the light version of the screen, the creation is using keyboard shortcuts for now because this is a developer preview and not what the end version will have. You'll have proper buttons to create projects but the keyboard shortcuts are most likely going to come bundled as well.

The alert icon you see is actually a pulsing icon and I can't really show it well in a picture but it pulses when you are approaching a project deadline.

The menu on the bottom allows you to go to the project details to check on members and tasks as needed.

Tasks and Status

Already implemented this quite a few times now with todo and TillWhen also has a beta task list so I don't think I need to explain what and how this is but it's not Kanban, I really don't see the need for someone working to have to go through moving items here and there in a board instead of just marking it as done or any other status, but that's just my thinking and Asana does it well but get's very bloaty in terms of things you can do and you accidentally end up clicking on something you didn't want too. Not what I want from a project manager.

Here's what the tasks look like right now.

Again, the creation is left on keyboard for now and the assignment to users and projects is still being done since I need to add filters for the same in a way that doesn't add to the bulk of what you see visually. I'm still working on the UX for the same.

The squircle dots on the left are the visual representation of the current status of the task and are what you click on to move it up or down, since the concept of backlog doesn't exist right now, it's not in the menu, I do plan to add that later since I use the backlog more than I use the actual task list.

Settings and Plans

Nothing huge here, just simple notification settings for now and obviously the ability to update your username, the Plan defaults to Hobby for everyone right now since the payment gateway etc will need a little more work and that's not going to be a part of the initial release anyway.

Also since payments are going to be involved I'll have to prepare legal documents for the same and hence the whole billing and card addition is going to part of future releases and not the initial one.

Integration with TillWhen

Sad to say but I might shut down TillWhen after this particular feature is done and added to the release, this is still in works since both apps need to be updated to be able to do this. The overall plan is for you to be able to export all data from TillWhen and import it on Taco and then taco is where you also handle time logs, so yes while this a big thing to do it is something I do have in mind. Reason being it'll be hard to maintain and work on 2 decently sized projects as an Indie developer.

Or i can move everything that Taco does into Tillwhen but then that's more patches here and there which I don't want too. Why is TillWhen Patchy? It was built in 2 days, what do you expect ?

Also I didn't plan scaling, integrations anything, I got lucky with them because of how I write but that's about it. It's got enough hacky and patchy implementations. The slack integration is a joke on it, exactly why Multi Workspace Slack integration on it has not been done yet. I don't want to break the existing implementation thats working well.

That was a long post...

Fortunately, that's all.

Adios!

The most preferred way to add CI/CD to React Native

Reaper — Tue, 04 May 2021 07:20:41 +0000

I've got a few projects where I need to add in CI/CD for React Native and while I've got fastlane setup and React Native has the environment variables working, my main issue is with the bootstrapping of a new project and having to set all this again.

Service Accounts for Play Store
Gitlab Environment files
Apple Account

While I know the above can't be given up , I was thinking something on the line's of a bash script that opens up the needed pages during the initial setup so I remember everything that needs to be taken care of before I let gitlab execute everything.

Would like to know if there's other solutions that people are following.

Why I use Next.js for everything and why you shouldn't!

Reaper — Tue, 20 Apr 2021 05:19:27 +0000

You can directly read the summary if you'd like to avoid the explanation, Summary

Context

I became a Vercel (formerly known as Zeit) fanboy and someone who wanted to join their team somewhere around April 2019 when I first discovered Zeit.co for app deployments and also found out that most of what I used in terms of libraries and even tools were actually built by them ¹

And I've attempted writing to them a few times in terms of joining the team but with no success of any response so I'm just going to assume it's a No, getting back to the point.

Next.js

Talking about the foundational framework of TillWhen and a lot of my web apps, mini tools and also the reason I mentioned Vercel first. Vercel is responsible for the Open source framework Next.js which is a SSG (Static Site Generator) based on React much like Gatsby. The reason I picked Next.js was simple

Easy base app scaffolding (yarn create next-app <app-name>)
Don't have to add a router as the page structure defines routes (an inspiration for the statico generator and also ftrouter)
In built API handlers which are written as modules and are pretty scalable if you understand how to structure the routes²
The Generated HTML works decently with or without JS unless you decide to handle routing programmatically, which you shouldn't but in case you do it might not work without JS.

These points aren't unique to Next.js anymore but then the newer frameworks don't really offer anything significant to make me shift to them, If I had to jump out of Next.js I'd jump to Nuxt which is for Vue and just use React as is for other projects, I've given Gatsby a try but I guess I just prefer Next.js.

Why shouldn't I use Next.js for everything like you do ?

Well, I always say that you decide on tech stack based on intention of what you are building and the target you wish to achieve while building it.

If the target is to learn, you're better off choosing a tech you haven't used ever. This is how you understand where the tech prevails and where it's going to be a bad option.

If you choose to build a quick prototype, you choose the tech you are most familiar with, this can be a really old tech and probably obsolete at this point but if you are just testing out an idea or working on seeing if the product is going to gain any traction, you still use what you know has worked for you in the past.

If you choose to build for production, you take the prototype or concrete requirements and figure out and experiment with tech that was built for this stuff.

Eg: I built a static generator from scratch for my blog, doesn't mean I'd do it for a client. I'll pick up a battle tested (Wordpress, Ghost, etc) to be my base and hand them that, this can be a bit heavier but is a lot more stable in the long run. On the other hand while doing this you also get ideas as to how you can improve your own scratch built tools to fit more and more scenarios.

But you use next for everything

True, now the reason for that is most of what I build is to act as quick prototypes of a certain idea that I had or something I wish to see and re-implement just to understand what's going on in the code when a dev actually built it and if I can improve it. Good examples of this are Hen , Colors, Pending, other mini tools that I've built over the years. I do use each of them but they aren't unique concepts and just exist because something from some app inspired me and I just wanted to get down and build a clone to understand it and this is where it's fine to use a fallback default stack, which in my case happens to be Next.js

The stack may differ for everyone. You might be a Angular + Phoenix or React + Koa or a RoR + React/Angular monolith person, doesn't matter.

Example

Let's take Hen into consideration here, it's a simple live preview for component code written in React.

Do I need a SSG for this? Nope.
Do I need an inbuilt router ? Nope.
Do I need n number of imports that come with the added framework ? Nope.

I could build Hen with just vanilla JS or just React and that's more than enough. As a matter of fact Hen is built with just React, Hen Experimental on the other hand is the repository that holds the unstable and testing code for the original Live Preview attempt and was written in Next.js.

Point to take from this?

Prototype with a fallback stack and build/re-build it with the shortcomings and advantages with a stack that finds a nice middle ground

You may or may not have to do this with every app you build but it's generally a good idea to find out what the app requirements need , build a dummy version with tech you are quick with, this can even use existing codebase from previous projects etc etc etc, you get the idea.

Again, you might not have to switch the entire stack and as the experience grows with various technologies you might not even have to build the prototype for a certain set of functionalities, you just know what will work well and what won't but to get there you'll have to experiment first and not take people's word for it (conditional but generally helps to increase your own knowledge and build an independent opinion)

Summary

Have a fallback stack
Don't use it for everything and anything just because you can, unless it fits with the requirement.
Have a Prototype First, Production Later mindset to make sure you build something that's a lot more scalable while avoiding monkey patching stuff in the future.
Experiment with different technologies to build both knowledge and independent opinion, which can then help you make better decisions (I'm saying this with regards to programming, but you can apply this to life in a way as well)

I used to have Hyper as my primary terminal, pkg to create executable node binaries, ms to handle millisecond formatting and no I wasn't sponsored to say this, I actually did use all these libs a lot back then and I had already tried next.js at this point but it was still not production level as a few issues stopped me from using it for prod. Oh and there was release and also I did get inspired by their design system a few times ↩
You can create a huge mess if you write long api paths and don't follow entity based standard while writing routes. ↩

This Weekend in BarelyHuman Dev Labs - March 10th and 11th

Reaper — Mon, 12 Apr 2021 06:01:10 +0000

Project Updates

statico

Mentioned that we had a static generator written specifically for reaper.im and all I did was pull it out into a separate repository and is now under the name statico and while it can be used for basic markdown to html page generation while maintaining folder structure¹ it's still a very simple generator and not as generic as it should be, it will be as the project progresses.

commitlog

There was a little progress in the release command addition to the cli, not a lot and no it's not been pushed, the older release command is still there in the codebase and while functional it doesn't increment the tag as I expected it to be so the cases are still being handled.

vscode-monoclean-theme

I started writing a vscode theme long time back, just completed it this weekend and also added a light variant to it. It's only been tested on Go and JS so I'm not sure about the other languages while it shouldn't cause many issues in other languages/workflows , do raise an issue if you do see one. You can find the install instructions and screenshots on github

Life Updates

Wasn't able to post last week since there wasn't any progress, I've been sick the past week and wasn't able to pick up anything so there was nothing to post last weekend. Somehow I've caught covid while staying home all day, anyway I don't see any direct symptoms so I'm not that worried but we'll see how it goes.

In terms of business setup, Tillwhen codebase is to be cleaned and I need a good project management workflow to be setup so I don't treat TillWhen like a side project and just abandon it, which will definitely happen if I don't setup a good workflow forcing me to sit and work on it.

That's about it for now,

Adios!

You can go through the reaper.im codebase to see what I mean by a simple structure ↩

Handling heavy memory tasks asynchronously

Reaper — Tue, 16 Mar 2021 02:53:45 +0000

This is going to be both a post to point you to a library I made and also point you to a solution for handling heavier tasks.

Disclaimer: This is one of many solutions that you can use, I'm just talking about this one since I end up using this a lot more than other solutions, I will mention other solutions though

As programmers we are given problems that need to be solved or we discover a problem and then sit down to solve it. Once such problem has always been load management, we've got various tools that handle it for us at different levels.

You've got load balancers for network requests, you've got the ability to scale vertically and horizontally in terms of parallel containers or k8 nodes but let's come to the small section of this, handling load in your actual code.

I'm not sure if you've come across this but a lot of times you've got like 100s of functions that need to be run parallely or sequentially or even cases where the memory can't handle this run.

Context

A few days back I was working on a small dummy replication server on my Raspberry Pi which hardly had about 100-250mb ram left with all the other hosted services running on it. The replication server was to run my spotify , get all playlists, go through each playlist to get all the tracks and then match these on Apple Music and then sync them to my Apple Music. Why do this? I accidentally deleted my Apple Cloud library (My bluetooth keyboard malfunctioned and instead of deleting a single track, I deleted the entire library) and I need all the tracks back so I wasn't going to do it one by one and I could use SongShift but then it's slow and I'd have to setup multiple runners on it for each playlist and then run them manually so I figured I'd just write a small server for it.

Problem

Limited amount of RAM: 100-250MB
Number of tracks to process: 2500
Matching each track takes about : 1s
total time to be 2500 seconds if done sequentially
memory usage per track search = 500kb , so about 125MB total to import all

The problem? I have close to no ram left on the rasp when this runs and using the other hosted services becomes hard cause they need to shift to swap and then it lags.

Solution?

There's quite a few solutions here

Message Queues
In Memory Batches
DB Queues

Message queues

Good solution but then I need to add RabbitMQ or redis + msg queue, setup a worker to listen to the queue and a server sending the instructions to the queue, limit the queue to not process more than 10-20 tracks at once.

DB Queues

Similar to the message queues but instead I add rows to a queued table of any existing database on the device and the worker keeps checking it after 10-20 mins to see if it has any new queued tasks and mark them complete when done with , this is simpler to implement and would be a viable solution if you need a solution that's crash resistant. Message queues though might loose it's messages on crash in certain cases so this might be the most reliable solution to go with.

In Memory Batches

Now, as the name says, its similar to the above two but we do it in-memory and yes, I'll loose all progress on a crash but this particular use case doesn't need to know if it's started again so we can go ahead with this and it's simpler if you are doing the same operation again and again.

Implementation

You cut down the array of items into slices of certain amount, let's say I want to process only 20 tracks at once, then I run the async process and wait for the first 20 to complete. We then add the next 20 and process them and continue in the same manner but do it only in memory and this reduced the usage down to about 10mb of usage while this is running, leaving the remaining memory free to be used by other hosted apps.

For synchronous you could just slice the array and run a while loop waiting for one to complete before the other starts but incase of async this can be a little more code to write than normal so you might want to use a library.

I would recommend using one of the following

Though, p-map and bluebird's promise.map will allow you to maintain a queue that always have the mentioned number of promises waiting to be completed so it works like an in memory queue, on the other hand conch will maintain batches and will process it one batch after the other instead of a continous queue, both work well in the case , I just prefer the batching.

Another live example of this being used would be music's import logic, which does something similar but to search youtube with the provided spotify link

what other ci/cds can do this?

Reaper — Mon, 15 Mar 2021 13:14:16 +0000

So, I was going through a few CI/CD alternatives since Gitlab was down for quite a while today and 2nd time this week so wanted to have a fallback runner for the same.

I came across buddy.works and I kinda like it but then the memory is kinda limiting on the free plan and the actual reason I chose buddy was that it allows maintaining a mirror of the repository and I can push to it's remote to trigger a build but the it failed on all the builds due to memory limitations and I was thinking if there's other CI/CDs that provide this kind of functionality where I can push a mirror repo to the service.

It can be a self hosted solution as well

Weekend Updates - March 6th and 7th, 2021

Reaper — Mon, 15 Mar 2021 04:36:32 +0000

TillWhen

This week, I wasn't really feeling like developing or working on TillWhen so there are no specific updates there other than a small patch that adds to the authentication check during first render of the app.

New Additions

The past week was mostly me working on themer and making fixes and making the code more cleaner and readable in case someone else wants to fix something that isn't working as they expected it to.

Another project that was created yesterday was conch, a micro library to handle sequential batch processing of promises for systems that can't handle processing like a 100 promises at once, and also something I needed to regulate for cases where I have to preserve memory usage, though the current logic can be improved a lot.

Modified / Feature Upgrades

Other than adding themer to most maintained projects, music got a UI upgrade and now supports importing spotify playlists into the queue, you can either replace the entire queue with the playlist or add the tracks from the playlist to the existing queue. The keyboard shortcuts it had still work but are not listed on the UI right now , since I haven't figured a good way to show them on the current UI style without making it look odd.

Minor work includes template updates to the ftrouter-templates repo, and this includes an experimental cli command that was added to the current master branch of ftrouter, it now allows you to init a folder though this is not in any of the official releases tags, also I finally plan to sort the of ftrouter using some cli tool framework first before I release ftrouter onto the npm registry. You can still install it using the git repo as the repo consists of the compiled source.

mkdir -p new-project
cd new-project
ftrouter --init # which is a shorthand for `ftrouter --init -d .`

For an example on how ftrouter works you can check the music-server or the minimal template's example which includes both query and param cases as well.

That's all that I was able to do this weekend, though I plan on focusing a little more on TillWhen next week, let's see if I do.

Themer and how you can handle dark mode a lot more gracefully.

Reaper — Thu, 11 Mar 2021 06:23:51 +0000

Mirrored post from reaper.im/blog

A few days back I was basically redesigning the long lost todo app from my repositories and
I ended up liking my selected color scheme and the dark variant of it. This lead to a simple dark and light toggle that I wrote in about 20 lines of JS, by simply changing a key in the local storage and handling that change and edge case accordingly.

10 mins after this, I realised the the commitlog-web could take advantage of the new color scheme and the web version of it is written in golang and html templates so I needed something vanilla so I just ended up using the above code from the todo implementation. At this point, it's all good, but then a small issue. It'd take the stored theme instead of the system preferred theme only and for someone who's theme changes automatically over the course of the day , this was a problem.

Now most people would be fine with just the prefers-color-scheme media query but now I don't assume what scheme the user would want to use for my particular app so I want him to be able to choose between system, light, dark and now this is where themer got created.

It's like 200 lines and you can probably understand by reading the source code , but I'll get through the algorithm just in case.

Source Code

Also, you can just install themer and use it if you'd find that easier but here goes.

Requirements

Ability to switch between system,light,dark.
As a developer, the developer experience to just add in one button , point the library to it and have it work seamlessly.
As a developer, the ability to customize the toggles when needed so a function export that can handle the same context.
Permanent storage of the selected theme.

The Plan

Since there's a need for context, we are going to use a Prototype Function declaration for this library (more on that in a few mins).
Ability to customize the button, so the button won't be created dynamically but picked from the config provided to the library, though I wanted a quick setup so the library will handle the icons inside the button, just not the button creation and styling.
Write a function that can be exposed to the instance so that if needed, the person can create custom toggles programmatically.

Code Flow

We define a prototype function first. A prototype function is basically the vanilla js way of making/writing classes , give you the ability to add pre-defined methods to an instance created via the function as a constructor, an example of this would be Date

So, first piece of code.


function Themer(){

}

We need it to accept a config so that we can select if we want to handle the toggle ourselves or we want the user to handle it for us. Also, we will see if there's an existing theme value the user has or not.

function Themer(config){
 let element = config.trigger;
 if(element){
   // Check if the trigger was passed a class string or an id string and convert it to a proper html node ref
   if(typeof config.trigger === 'string'){
    element = document.querySelector(config.trigger);
   }
 }

// existing state for the theme , fallback to system if nothing is found
let defaultState = localStorage.getItem('theme') || 'system';

}

Now, for the actual toggle, all we do is set the body tag to have an attribute called data-dark-mode and if this is present, your css can over-ride the default light mode variables or you can write custom css with this as a selector.

body[data-dark-mode] button {
 background: white;
 color:#121212;
}

though, just resetting the variables would be easier, you can find an example here

All that's left is to find out which theme we are on and which the next one is supposed to be and this is done on the click on the trigger, also, remember we have to expose the function so we have to isolate that logic and also we need to make sure the same functions are also executed when the system preference changes if the set theme is on system

No use posting the snippet cause that's the whole index.js which you can read.

Hope you liked the post,

Adios!

Improving the UI of an old mini tool

Reaper — Mon, 08 Mar 2021 02:24:20 +0000

Mirrored post from reaper.im

I'm not really good at post titles, so going to keep them short from now.

A really old project of mine started gaining a bit of attention the past few days and I thought that I've been planning on making it look good for quite a while, I should just sit and redesign it once and for all.

So, when I woke up at 4 , unable to go back to sleep, I decided to make a design change.

Old Todo , this is what it looked like and here's the new one, Todo

I'll add an image for everyone else to have a look at

So let's go through everything that's there.

Features

the obvious one first, Dark Mode Support
Addition of tasks, it's a todo list (what were you expecting?)
Filter by state, you toggle between, All, Completed, Pending
Power menu to change theme and filter by state(coming soon)
PWA, you can install it if you're using chrome on desktop and Safari (Add to Home) / Chrome (Add to Homescreen) on iOS and Android respectively
Offline support (cause pwa), and your tasks stay in your browser, there's no sync, though you can share an encoded link to people who you'd want to share the tasks with.
Import tasks from shared URL into your task (coming soon)

And as always, it's free. Though you should try supporting the DEV!

I did end up implementing the same style on other tools as well though.

What's one project that you think was dead but then someone at random just tells you that they are using it and like it?